Jeffrey R. Young's Chronicle article, "Reaching the Last Technology Holdouts at the Front of the Classroom," has apparently struck a nerve among professors, particularly those who are critical of educational technology. As I write this, the article has 59 comments on the Chronicle site, which is far more than most articles receive. Even the graph accompanying the article has received 13 comments!

Since clickers are mentioned in the article and in many of the comments, I thought I would weigh in here on the blog...

First, it's worth noting that Chris Dede, the Harvard University learning technologies professor interviewed for the article, doesn't make the argument that professors who don't use technology are shirking their duties. Several of those who left comments seem to think so, however. For example, here's comment #33 from Emily in NY:

"Dede does nothing in this article but set up a false dichotomy between professors committed to outdated, boring and irrelevant teaching methods and those eagerly embracing the modern technologies that contemporary students crave."

Here's the closest Dede comes to that argument, in the National Educational Technology Plan he helped draft for the US Department of Education in March:

"The challenge for our education system is to leverage the learning sciences and modern technology to create engaging, relevant, and personalized learning experiences for all learners that mirror students' daily lives and the reality of their futures."

Dede's arguments in the Chronicle argument are focused on motivating professors to tap into the latest research on learning and continue to improve their teaching practices over time. From the report he drafted, it's clear he thinks that technology can help with that, but he doesn't seem to be making the argument that professors who don't use technology are irresponsible, just those who stick with the same teaching methods you'd find in a classroom circa 1900. Sure, technology can be a big part of change, but many of the teaching innovations mentioned in the article (such as David Pace's work on enhancing history teaching) don't involve any technology.

Speaking of false dichotomies, however, here's one from comment #18 by user "tee_bee":

"What matters is that students learn--and a skilled teacher with a blackboard is still going to do a far better job than a bozo with some clickers and powerpoint slides."

True, a skilled teacher is going to do a better job than a bozo any day, regardless of technology. But comparing a skilled teacher to a bozo isn't really important here. Might technology (including clickers) help a skilled teacher be even more effective? Yes, that happens. And might technology help a relatively novice teacher become more effective? Yes, that happens, too. Those are the kinds of changes in teaching that are worth thinking about and encouraging, and I think that's a point that Chris Dede would agree with.

How might teaching with clickers help a good teacher be even more effective? Several comments on the Chronicle article were skeptical of clickers' potential for doing this. For example, here's what user "ikant" said in comment #21:

"I'm young, tech-savvy, and pretty unconvinced by this article. I can't speak for all fields, of course, but I'm pretty skeptical that good class discussions and quality writing in the humanities are particularly improved by clickers etc... the heart of what I do is in trying to educe questions, critical thought and excitement about books which students might previously have thought were utterly irrelevant to them, and (my evaluations indicate that) I do this very well with no particular technological bells and whistles in the classroom. Am I missing something?"

I'm glad that this instructor is capable of leading effective class discussions, foster critical thinking, and increase student motivation in the classroom. Let me clear: Doing so is entirely possible without clickers! However, not all instructors are as skilled as "ikant" appears to be and even for instructors like "ikant," it's possible that clickers would enhance an already productive classroom environment. Some examples from past blog posts:

• Case Study: Using clickers to promote critical thinking in text-based, political science courses
• Case Study: Using clickers to teach ethics in a history class
• My Thoughts: Using clickers to foster creativity and collaboration
• My Thoughts: Using clickers to teach critical thinking in the humanities

Here's a similar comment (#26 on the Chronicle site) from user "csgirl":

"The reason I don't use blogs and clickers is that they simply are not appropriate to the material I teach. Clickers in particular are useless to me - I care about the strategies my students are using to solve problems, not whether they can click the right answer in a quiz."

This is a common misconception about clickers, that they're just good for quizzing students basic conceptual understanding and recall. Here's another formulation of it, from user "chewy18" in comment #53:

"They might work well for understanding basic concepts or in preparation for recognition/recall examinations where the test question is a line long and the answer a word or two in length. What about those of us who teach upper division courses where we struggle with students who have not, until they reach senior status, even been exposed to the analytical reasoning process. Suddenly they discover that life is, after all, not a multiple choice test and developing an argument that could go either way, is a requirement. How does that appeal to the clicker technology?"

Sure, clickers work well for assessing basic conceptual understanding and factual recall, but they're useful for teaching at the higher levels of Bloom's Taxonomy, too. Here are some more examples from past blog posts that demonstrate this:

• Case Study: Using clickers to teach difficult material in upper-division physics
• Case Study: Using clickers to teach students about plagiarism in library instructional settings
• Case Study: Using clicker questions without single correct answers in mathematics
• Case Study: Using clickers to teach nursing students to make evidence-based decisions

And for "csgirl," here's a great collection of resources on using clickers and peer instruction in computer science from Daniel Zingaro.

Finally, you can imagine how this comment (#37) from user "fizmath" made me feel:

"The teacher/physician analogy is lousy. We have real data to show that new medical tech benefits patients. You can't say the same about blogs, videoconferencing and those stupid clickers."

(This is a response to Chris Dede's analogy that teachers who don't update their teaching methods over time are akin to physicians who don't update their medical practices over time.)

Want some research? Try these studies, all of which are well designed and support the claim that clickers used in appropriate ways enhance student learning:

• Stowell & Nelson (2007) - Clickers provided instructors with more accurate assessment of student learning during class than other response methods, including a show of hands.
• Yourstone, Kraye, & Albaum (2008) - The use of clickers for end-of-class quizzes improved student exam scores by four points over the use of pencil-and-paper quizzes discussed the next day in class, likely because of the immediate feedback clickers provided to students on their learning.
• Hoesktra (2008) - Clickers helped students be more attentive during class (since they know clicker questions could be asked at any time) and participate in more meaningful ways (both before votes are submitted and after results are displayed).
• Smith et al. (2009) - Students actually learned from each other when discussing clicker questions in pairs prior to voting. They don't "simply choose the answer most strongly supported by neighbors they perceive to be knowledgeable."
• Mayer et al. (2009) - Clickers made it easier for instructors to ask their students questions during class and for students to respond to those questions, leading to improved student learning through better class discussions.

My summary for those skeptical of using clickers in the classroom: Read the literature, find out how those in your discipline are using clickers effectively, and see (preferably by experimentation) if those methods might help you to enhance your teaching, regardless of how effective you are currently as a teacher. If a classroom response system doesn't help you do your job better, then don't use one. They're not for everyone. However, don't write clickers off without first investigating their potential. They're far more useful and versatile that you might think at first.

Image: "Innovation" by Flickr user thinkpublic, Creative Commons licensed

The summer meeting of the American Association of Physics Teachers (AAPT) wraps up in Portland, Oregon, today. There were several talks on teaching physics with clickers at the meeting, including one by Ian Beatty of the University of North Carolina at Greensboro physics education research group. Ian was the subject of my first podcast interview, and he’s been doing great work helping science instructors at the K12 and post-secondary levels teach effectively with clickers.

In Ian’s presentation, he identified and addressed several common concerns instructors express about teaching with clickers. For each concern, Ian identifies a belief about teaching and/or learning that likely underlies the concern, as well as an alternate belief that can be adopted to address the concern productively. Ian also includes some practical strategies and example clicker questions for each of these alternate beliefs.

For example, when many instructors hear about teaching with clickers, they’re concerned with having sufficient class time to cover what they need to cover in their courses given the time required by having students discuss and respond to clicker questions. Ian notes that this concern is likely a result of the following belief: “I must explicitly address in class everything students will be held accountable for.” Ian then presents an alternate perspective on this idea: “I can use class time to focus on core ideas and big-picture understanding, and charge students with filling in the details outside class.” This alternate perspective is, perhaps, non-intuitive to many instructors, but it’s a reasonable and useful perspective to have. Adopting this perspective leads to a shift from what Ian calls an understanding of class as a place to present content to an understanding of class as a place to help students digest content. Ian then shares five tips and techniques for implementing this shift in the classroom.

Ian addresses other concerns in a similar manner, including concerns about having enough time to write good clicker questions, concerns about poor student participation during class, and concerns about changing one’s teaching style. His visuals, which use the online presentation tool Prezi, are included below and are well worth checking out.

I’ve scheduled this post to appear on the blog just as I’m starting my keynote at the University of Louisville clickers conference in Louisville, Kentucky.

• For those of you not at the conference, you can get a sense of what I’m talking about right now by checking out my Prezi below.  You’re welcome to weigh in on Twitter about these ideas.  Just tag your tweets with #ULclickers so I’ll see them.
• For those of you at the conference, you’ll find below links to a few resources mentioned in my talk.  Feel free to explore these after the keynote!  (Or during… I’m cool with that.)

My talk is titled “Connecting with Participatory Culture: Clickers and Deep Learning.”  Here’s the abstract:

Today’s students vote for their favorite contestants on American Idol, “like” a friend’s wall post on Facebook, comment on news and events on Twitter, and engage in robust online discussions about World of Warcraft.  We live in a participatory culture, one in which voting, commenting, creating, and sharing are the norm and people prefer being contributors to being consumers.  Teaching with clickers is one way to tap into this culture, engaging students in ways that motivate them to participate during class in meaningful ways.  In this talk, Derek Bruff will explore ways that using clickers connects with our students’ participatory culture and how those connections can be leveraged to promote deep learning.

And here’s my Prezi:

Finally, some relevant resources:

Like Buttons / Student Perspective Questions

• Matthew Freeman’s perspective questions come from this article: Campt, D., & Freeman, M. (2009). Talk through the hand: Using audience response keypads to augment the facilitation of small group dialogue. The International Journal of Public Participation, 3(1), 80-107.  Here’s my summary.

Text-to-Vote / Peer Assessment Questions

• A description of Kori Street’s use of clickers for peer assessment can be found on pages 94-96 of my book.

Serious Fans / Misconception Questions

• The research on World of Warcraft forum discussion can be found here: Steinkuehler, C. & Duncan, S. (2009). Informal scientific reasoning in online virtual worlds. Journal of Science Education & Technology. DOI: 10.1007/s10956-008-9120-8.  See also Constance Steinkuehler’s other research.
• Derek Hansen’s visualizations of edits to Lostpedia, a wiki for the TV show Lost
• The “time for telling” idea comes from this article: Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475-522.
• My misconception question about lie detectors is adapted from Teaching Statistics: A Bag of Tricks by Andrew Gelman and Deborah Nolan (Oxford University Press, 2002).

Event TV / Critical Thinking Questions

• The connection I made between social media (especially Twitter) and “event” TV comes from “Twitter and TV: How Social Media Is Helping Out Old Media,” an essay by James Poniewozik in the March 22, 2010, issue of Time.
• Here’s my post about critical thinking questions in my recent cryptography course.
• Here’s my post about how to adapt Rob MacDougall’s “counterfactual questions” for use with clickers.
• The terms absolutist, relativist, and evaluative come from this article: Kuhn, D. (1992). Thinking as argument. Harvard Education Review 62(2), 155–178.  Similar ideas are found in the work of William Perry and others, captured well in this essay by Richard Felder.

Volunteerism

• I heard about Ushahidi on the World’s Tech Podcast #274, a production of BBC’s The World.  Here are the current Ushahidi reports from Haiti.
• I also heard about Matter of Trust‘s efforts to collect hair and nylons to make hair booms in the World’s Tech Podcast, episode 287.  I highly recommend the World’s Tech Podcast (hosted by Clark Boyd) as a way to learn about “undercovered” tech news.
• The Alphachimp Studio blog has some information on how graphic designers in Nashville have been responding to the recent floods here.  My “We Are Nashville” t-shirt comes from Cool People Care.  And it was Hands on Nashville that’s done a tremendous job of coordinating volunteer efforts in town.

For more on the notion of a participatory culture, read Henry Jenkins’ white paper, “Confronting the Challenges of Participatory Culture: Media Education for the 21st Century” [PDF].  Also, here’s my blog post that got me started thinking along these lines, the one that references the Campt and Freeman article.

What are your thoughts on the ideas in my keynote?  Do we, especially our students, live in a participatory culture?  What consequences does that have for how we teach?

Image: “skates” by Flickr user marythom / Creative Commons licensed

Sunday night, I delivered the opening keynote at Central Michigan University’s Great Lakes Conference on Teaching and Learning.  My presentation was titled “Class Time Reconsidered: Motivating Student Participation and Engagement.”  My goal was to share some frameworks and strategies for engaging students in the classroom by taking a few common assumptions about teaching and learning and flipping them on their heads.  Here’s my Prezi, complete with much flipping of things on their heads:

Some thoughts on the presentation:

One of my first clicker questions asked participants to identify a key challenge in motivating students to engage meaningfully during class.  Strangely, the even-numbered answer choices were by far the most popular-students are hesitant to speak up in front of their peers, students focus too much on grades and not enough on learning, and students don’t prepare adequately for class.  These results worked well for me, since I had been planning on addressing ways to reach students who are risk-averse or grade-focused and ways to motivate students to prepare for class in useful ways.

Participants engaged in a Think-Pair-Share activity in which they tried to identify six steps in a typical process their students might undertake to learn in their course.  This followed an introduction to the idea of a “time for telling,” so I asked participants to make sure that “telling” wasn’t the first step on their lists.  I also encouraged participants to force themselves to come up with six steps.  Coming up with three-step plans (take notes during class, figure things out in the homework, regurgitate on exams) is too easy.  Identifying a six-step process means you have think a little more intentionally about how your students learn.

Given the clicker question results indicating that lack of student preparation is a big challenge, we camped out for a while on the idea of a pre-class assignment.  I made two important points about these assignments: they should be graded, if only on effort, so that students will take them seriously and you should make use of these assignments in some way during class.  Otherwise students will see them as busywork, not connected with the “real” work of the course.  One participant shared her approach-she has students create outlines of their pre-class readings, then share and compare their outlines in small groups during class.

Monday morning (the day after my presentation), I saw on the book raffle table that there’s a new book on Just-in-Time Teaching, Just in Time Teaching: Across the Disciplines and Across the Academy,  edited by Scott Simkins and Mark Maier (Stylus, 2010).  I wish I had known that Sunday night-I would have mentioned it during that section of my presentation!

My third and final clicker question asked participants to identify one of five in-class engagement strategies they wanted to try soon.  While I wasn’t intending the presentation as a pitch for clickers, perhaps my biases couldn’t be hidden-clickers was the number one answer!  This result might have also been because clickers are new and different, but not so different as to require a complete rethinking of one’s teaching approach.  I’m convinced that the return on investment for teaching with clickers is high-one can make small changes in one’s teaching methods that yield significant results.

At the end of the presentation, I had the participants generate questions for me at their tables.  Most of the tables had at least one person with a Web-enabled device (such as the iPads several of the CMU staff hosting the event were sporting).  They used these to submit their tables’ questions via Google Moderator.  I asked them to vote on other tables’ questions, as well, providing me with a ranked list of the most popular questions.  This served as a reasonable demo of Google Moderator as a backchannel tool, but unfortunately I didn’t have time to address the questions that emerged through this process.  My plan is to address the more popular questions with Google Moderator since, as the creator of this Moderator session, I can leave comments on individual questions.  You can see the questions submitted by the group here.

The conference continues through Tuesday morning.  I was able to attend most of the conference on Monday, and I live-tweeted a couple of the sessions.  You can read my tweets here.  Joy Mighty of Queen’s University in Ontario delivered the Monday lunch keynote, and she made a strong case that by not paying attention to matters of diversity in our classroom, we run the risk of fostering inequity.  It was a thought-provoking keynote for me.

Thanks to Central Michigan University for having me as part of their conference and for some great conversations about student engagement!

Over on the new Active Class blog, Sidneyeve Matrix recently discussed the idea of turning the college lecture into something like what the television industry calls “event programming.”  She did so in the context of encouraging students to come to class when the lecture is captured for later listening or watching by students.  She suggested instructors who are capturing their lectures incorporate a couple of elements to the in-class experience that aren’t replicated in the lecture capture: the use of video clips that help students remember and make sense of course content and the use of clickers for content and opinion questions.

Sidneyeve makes some great points about how the use of clickers can give students a sense of ownership over the in-class learning experience:

Clicker polls effectively personalize, customize, and socialize the class. Students know that poll results depend on who is in class that day and it is that indeterminacy that lends energy and anticipation to the lecture. Moreover, if the students see that their polling feedback is valued by the professor, and is connected to assessment, they too will value the activity of in-class participation as worthwhile.

Sidneyeve has hit upon a subtle, but important point about one of the roles that clickers (and more general classroom response systems, like backchannel tools) play in the classroom.  Using clickers turns students into co-creators, along with the instructor, of the in-class learning experience.  When the results of a clicker question are shown on-screen, something happens that would not have happened were those particular students not in the room participating that day.  Many instructors who use clickers practice what I call “agile teaching,” using the results of clicker questions to directly inform how they lead discussion or how they spend their class time.  Even when instructors don’t practice agile teaching, the results of a clicker question are still unique to the particular collection of students in the room at that time, and those results have at least some small impact on the students who view them.

Imagine the opposite, what some call “ballistic teaching,” a lesson plan that, once launched, cannot be altered by feedback from students in the room.  A video capture of this kind of lecture would be almost, if not more, valuable for students as attending this lecture in person.  Instructors considering lecture capture are often worried, as Sidneyeve points out, that students won’t come to class if lectures are placed online.  But if there’s virtually no difference between watching a lecture online and watching it from a seat in the classroom, why should students come to class?  What’s the point?

As Sidneyeve points out, including clicker questions in a lecture means that there is necessarily a difference between watching a lecture online and participating in the “live” version.  At the very least, watching a clicker question on video after the fact means that the student’s vote isn’t included in the results of the clicker question.  And if the clicker question is used to facilitate small-group or classwide discussion, then there’s even more difference–and more reason for the student to come to class and participate live!

I would like to take the television metaphor of “event programming” that Sidneyeve uses, and push it one step further.  A colleague of mine shared with me a recent Time essay by James Poniewozik titled “Twitter and TV: How Social Media Is Helping Old Media.”  In the essay, Poniewozik points out that many television viewers DVR their favorite shows to watch them later and skip the commercials.  This latter point is of particular interest to the television industry, since their revenue depends largely on advertisers.  Poniewozik argues, however, that social media like Facebook and Twitter can make certain television shows into “events” that viewers want to watch live.  He points to the live discussions that occurred online during the recent Winter Olympics and Academy Awards.  Participating in these live discussions was an incentive for people to watch these programs live.  I’ll attest to that–one of the reasons I watch Lost live as it airs each Tuesday night is so I can participate in online discussions about the show during and immediately after it!

Where am I going with this?  Well, to continue with the metaphor, a lecture that includes no interactive component and is captured for later viewing by students is like a television show that you DVR and watch when you get around to it.  A lecture that includes interactive components like the use of clickers and backchannel discussion (which is very much like the kind of online discussions Poniewozik describes) is more like an “event” television show that you just have to watch live as it airs.  Wouldn’t it be great if students refused to skip class (and watch the lecture video or borrow a friend’s notes instead) because they’ll miss their best opportunity for learning?

Image: “Empty” by Flickr user Shaylor / Creative Commons licensed

Here are a few interesting ideas shared during the first set of talks about teaching with clickers at the Joint Mathematics Meetings earlier today.  (This post’s title inspired by the following books seen at the exhibits: Calculus Gems, Mathematical Diamonds, More Mathematical Morsels, and Biscuits of Number Theory.)

Kathryn Ernie (University of Wisconsin-River Falls) shared ways that she and her colleagues use clickers in their college algebra courses.  One use that she mentioned was to warm students up before “traditional” in-class quizzes.  By asking students a clicker question or two, then discussing those questions prior to a graded quiz, the students are able to approach the quiz with a little more confidence.

Ben Galluzzo (Shippensburg University) also talked about graded quizzes.  However, in his case, clicker questions aren’t warm-ups for the quiz; the clicker questions are the quiz.  Clickers allow Ben to turn his quizzes into learning experiences for his students.  After each quiz question, he discusses the question with the class before moving on to the next question.  This can work particularly well when he has more than one quiz question of the same type.  Students who miss the first one can learn from the discussion of that question and apply what they’ve learned to the subsequent question.  Students like this because they appreciate the chance to redeem themselves.

Aprillya Lanz (Virginia Military Institute) mentioned that teaching with clickers help students stay awake and engaged during class.  This is particularly important for her since many of her students are freshmen (“rats” as they’re called at VMI) who are required to participate in all kinds of strenuous physical activities, particularly on Sunday nights.  This can make for some very sleepy students in Monday morning classes.

Daniel Joseph (also VMI) described a problem that those who teach calculus often see: His calculus students often struggle because of pre-calculus misconceptions.  They can’t tackle the calculus because they get tripped up by algebra and other pre-calc topics.  He described several methods he’s tried to combat this, but he finds that the students’ over-confidence trips keeps these methods from working.  The students say they “know” all the pre-calculus material because they’ve studied it in the past.  Daniel appreciates how clickers provide his students with frequent evidence that they don’t know it as well as they think they do.

Daniel shared one approach to attacking this problem–using clicker questions in a pre-semester pre-calculus course for incoming freshmen.  He’s interested in hearing ideas for hitting this issue, with or without clickers, in the calculus course itself.  Any ideas?

(By the way, Daniel used the phrase “attack this problem” at least five times in his presentation.  Given that he teaches at a military institute, I figured that was language that comes naturally to him.  Thus my use of the verbs tackle, combat, attack, and hit above!)

A little while ago I reviewed an article by David Campt and Matthew Freeman describing the use of clickers in dialogue facilitation.  I was impressed by their nuanced use of participant perspective questions–the kinds of questions I call “student perspective questions” in my book, questions that surface students’ (or dialogue participants’) opinions and experiences.  The idea of using what appears to be a factual question (such as, “What percent of US citizens are people of color?”) to surface participants’ assumptions and impressions was particularly interesting.  Prior to reading their paper, I hadn’t thought of such questions as perspective questions.

More recently, Campt and Freeman wrote a short article for the Web site Religious Conference Manager describing ways clickers might be used in religious settings (sermons, conferences, and so on).  The article describes the kinds of perspective questions I mentioned above, as well as strategies for making the most of clickers in these settings.  The following line stood out to me:

Internet experiences and television shows are creating the expectation that people will be co-creators rather than mere passive vessels.

I hear this perspective expressed frequently in educational technology discussions, particularly in regard to students’ participation in Web 2.0 online tools like Facebook and YouTube, tools where students are not only consumers of content but producers.  I hadn’t thought about this notion in the context of teaching with clickers, however.

Certainly, if you’re giving your students a quiz that requires them to recall a few facts, then you’re not using clickers to help students become “co-creators” of knowledge in the classroom.  However, if you’re asking clicker questions designed to generate discussion (like tough conceptual understanding questions, application questions, or critical thinking questions), then as students involve themselves in that discussion, they are, indeed, become co-creators in their learning.  And if you’re using clickers for perspective questions in the ways that Campt and Freeman do, then the students’ responses (their opinions and experiences) are integral to the learning process, making them co-creators.

When used just for voting, clickers do indeed bring in the interactive element that students are accustomed to having in their daily life–”liking” a friend’s wall post in Facebook, rating a book on Amazon, voting for a favorite contestant on American Idol.  We live in a participatory culture where everyone seems to get a vote, a way to provide feedback.  Clickers provide a way to connection with that culture in the classroom.

Moreover, when used to generate discussion, clickers help motivate an even more participatory culture in the classroom, one similar to that of a blog with an active commenting community.  The author of a blog post gets the conversation going, but everyone weighs in–just like an instructor gets a conversation going by posing an engaging clicker question and the students weigh in via votes and discussion.

What do you think?  Does teaching with clickers work particularly well with students used to a participatory culture?  Do you buy the argument that today’s culture is more participatory than, say, 15 years ago?

Clyde Herreid of the University of Buffalo’s National Center for Case Study Teaching in Science has received a $500,00 grant from the National Science Foundation to study the use of clicker questions in case studies used in biology classes, with a particular focus on students’ emotional engagement with science.  (The University of Buffalo seems to be a happening place for clickers.  I mentioned their School of Dentistry’s use of classroom response systems on Monday.)  I haven’t blogged about the use of clickers in case studies yet, but I mentioned the practice in my book, citing Herreid’s paper on the topic (Herreid, 2006) as well as Peggy Brickman’s clicker-enhanced case study on DNA fingerprinting.

I’m glad to see an investigation of this pedagogy receive such a sizable grant, particularly given its emphasis on the affective domain.  However, the press release describes students using clickers like a game show buzzer, which is a bit misleading, now that I think about it.  On Jeopardy, for instance, only the fastest responder is allowed to answer, but when using clickers all students have a chance to weigh in.

Meanwhile, the Faculty Technology Center at Louisiana State University recently hosted a presentation on teaching with clickers by biological sciences professor Steve Pomarico.  Pomarico noted that before using clickers in his 250-student course, attendance would vary from 60% to 30% on any given day.  Now that he uses clickers and awards participation points for students responding to his clicker questions, attendance is never below 65%.  He notes that merely using clickers to take attendance is a poor choice, however.  Asking questions that let students test their understanding and provide instructors with useful feedback on student learning is a better use of the technology.

• Herreid, C. F. (2006). “Clicker” cases: Introducing case study teaching into large classrooms. Journal of College Science Teaching, 36(2), 43-4.

University of British Columbia professor of earth and ocean sciences Roland Stull recently gave his popular course on the science of storms a clicker makeover.  Persuaded by research from Carl Wieman’s Science Education Initiative at UBC, he now structures his class sessions around conceptual understanding clicker questions, using a version of the standard peer instruction technique.  Stull has his students read their textbook and respond to online quiz questions the night before class.  He has a TA analyze their answers for common areas of confusion, then adjusts his plans for class to address those areas.  Stull notes a variety of benefits to this teaching approach:

“It’s a lot more fun for me to teach the class,” Stull said in an interview in his UBC office. “Not only are the students interacting with themselves, but they are much more willing to ask me questions during class.”

The Georgia Straight article about Stull’s use of clickers quotes Alan Webb, a University of Waterloo accounting professor who published a study ostensibly showing that teaching with clickers actually decreases student participation in class.  However, as I noted in my review of this study, what Webb actually showed was that indicating the correct answer to a clicker questions prior to class discussion of the question decreases student participation.

At the University of Buffalo School of Dentistry, instructors John Maggio and Chester Gary have students respond to questions during class using their laptops as response devices.  The school requires students to have laptops so they can access electronic textbooks, so using “virtual clicker” software on student laptops makes sense.  Maggio finds that his students have rather short attention spans, so he uses clicker questions to keep them engaged during his 90-minute classes, asking as many as twelve questions per class.  The frequent questions and the fact that some are graded on accuracy (not just effort) keep his students from using their laptops to distract themselves.

Just like Roland Stull at UBC, John Maggio says that his clicker questions have increased participation in his class:

“They raise their hands much more often, they’re discussing things much more, they’re participating more than they ever have,” [Maggio] says, noting that his classes featured very little discussion or debate before the introduction of the audience-response technology.

One of the criticisms I often hear about teaching with clickers is that doing so gives shy students an excuse not to summon the courage to speak out in class.  These two news articles would indicate that’s not the case, after all.

Reference: James, Mark C., Barbieri, F., & Garcia, P. (2008). What are they talking about? Lessons learned from a study of peer instruction.  Astronomy Education Review, 7(1).

Summary: This study is a follow-up to James (2006), a study comparing the effect on student discourse during clicker questions of grading incentive.  In the earlier study, student conversations were audio-recorded in two different astronomy course, each taught by different instructors and each involving different grading schemes for clicker questions.  The main finding of the earlier study was that a low-stakes grading scheme (one in which incorrect answers counted as much as correct answers) encouraged richer student-to-student discussions prior to voting on clicker questions than a high-stakes grading scheme (one in which incorrect answers counted only a third as much as correct answers).

A key drawback to James’ earlier study was that the two courses being compared were different in significant ways other than the grading scheme used for clicker questions.  In particular, they had different topics and different instructors.  That drawback has been mostly eliminated in the current study by James, Barbieri, and Garcia.  In this study, the same instructor taught the same course, an introduction to astronomy course with about 180 students, in two consecutive semesters.  In both semesters, clicker questions contributed 12.5% of the students’ overall course grades.  In the first semester, incorrect answers counted one-third as much as correct answers, but in the second semester, incorrect answers counted 90% as much as correct answers.

The instructor used a version of the standard peer instruction technique.  Students were not asked to vote on clicker questions independently, but were instead asked to discuss the questions in pairs prior to voting.  Random samples of students in each semester were audio-recorded during these pair discussions throughout the two semesters.  The audio-recordings were analyzed in two different ways to measure “discourse bias,” “the difference between the fractional contributions to a conversation between partners.”  For instance, if one partner contributed 70% of the time and the other contributed 30% of the time, then the pair’s discourse bias would be 40%.

First, each idea shared by the students during the discussions was coded according to ten categories, including categories such as restating question elements, stating answer preferences, and providing justifications.  (One side finding was that there was no correlation between “type” of clicker question and the nature of the ideas shared by students during the discussion.)  Second, the total number of words produced by each student during the discussions was counted.  Both techniques provided measured of each student’s contribution to the discussions.

The results strongly indicated that the low-stakes grading scheme encouraged more balanced participation by students during pair discussions.  For example, when using the first measure of discourse bias (counting ideas), the average bias for the high-stakes class was 33.2%, whereas the average bias for the low-stakes class was 19.5%.  That is to say that each pair of students engaged in a conversation more dominated by one of the students in the high-stakes class.  The second measure of discourse bias (counting words) provided similar results-an average bias of 39.8% in the high-stakes class and 26.6% in the low-stakes class.

The authors also note that the low-stakes grading scheme promoted more independent student responses to clicker questions following pair discussions.  In the high-stakes class, only 7.6% of the time did two partners submit different answers to clicker questions, whereas in the low-stakes class, this occurred 17.1% of the time.  The authors conclude from this that in the high-stakes class, students’ concern for earning points motivated them to submit their partners’ answers to clicker questions even when they didn’t really believe those answers.

Comments: This study improves on James’ earlier study and provides persuasive evidence that low-stakes grading schemes for clicker questions promotes more meaningful student participation in small group discussions prior to voting.  True, this wasn’t a double-blind, randomized control group experiment (in which students were randomly assigned to the two grading schemes and the instructor didn’t know which grading scheme would be used with each group of students), but such experiments are practically impossible to implement in educational settings.  Short of that “gold standard,” this is a very well-designed and persuasive study, in part because many of the possibly confounding variables in the earlier study were eliminated and in part because of the use of direct, qualitative measures of student participation.

Willoughby and Gustafson (2009) conducted a similar study in physics courses, audio-recording student discussions in some sections and not in others.  They found that students in the sections that were not audio-recorded “block-voted” more when high-stakes grading schemes were used for clicker questions and less when low-stakes grading schemes were used.  In the sections where audio-recorders were used, there was no statistically significant difference in block-voting rates.  They concluded that the presence of the audio-recorders might have influenced student voting behaviors (an example of the Hawthorne effect).  If true, it’s possible that the difference James, Barbieri, and Garcia found in block-voting behavior might have been even greater had audio-recorders not been used-all the more reason to use low-stakes grading schemes when using clickers for formative assessment.

I would have liked to have seen a little additional information provided in the article about the use of clicker questions in these courses.  Were students asked to vote on their own before discussion the clicker questions in pairs?  (I don’t think they were, but this isn’t stated in the article.)  Also, what instructions were given to students prior to the peer discussion times?  I’ve seen some evidence (Lucas, 2009) and heard some advice (from Doug Duncan) that the instructions given to students prior to peer instruction can affect the quality of the discussions.  And while it was found here that the type of clicker question did not correlate with the kinds of ideas shared during peer instruction, it would have been informative to know what kinds of clicker questions were used.

What’s not directly address in this article, however, is the assumption that more student participation during small-group discussion of clicker questions leads to greater student learning.  This was an issue I raised in my comments on James’ earlier study, which included data on student performance on final exams.  In response to that study I asked if greater class participation led to greater student learning or if students who knew the material better simply dominated class discussions.  While it’s possible that the latter is true, evidence from non-clicker studies strongly suggests that more active participation in class discussions leads to greater student learning.  I wish this assumption (that participation leads to student learning) had been stated as such in the article.

The takeaway here is that low-stakes grading schemes for clicker questions leads to greater student participation and clicker questions results that more accurately reflect students’ actual understanding (or lack of understanding).  These results have important implications for instructors using clickers to motivate student participation and inform agile teaching choices.