Reference: Webking, R., & Valenzuela, F. (2006). Using audience response systems to develop critical thinking. In Banks, David A. (Ed.), Audience Response Systems in Higher Education: Applications and Cases. Hershey, PA: Information Science Publishing.

Summary: Webking and Valenzuela describe ways they use classroom response systems in their political sciences courses at the University of Texas-El Paso to foster critical thinking through active participation and class discussions. After noting some commonly cited advantages of teaching with clickers—easier attendance and participation record-keeping, greater participation through anonymity and accountability, and the collection of data to inform agile teaching decisions—the authors provide several concrete examples of clicker questions they have found valuable for developing their students’ critical thinking skills.

The authors’ first example is a sequence of clicker questions that serve to guide students through a close reading of a few passages in the play Antigone. At one point in the play, Antigone makes a statement that seems to very clearly express her belief that obedience to the gods trumps obedience to the king. At another point, however, she makes a somewhat cryptic statement that calls this previous assertion into question. Webking and Valenzuela start with an understand-level question that asks students to clarify this second statement. They follow this with an application-level question asking students to identify a logical consequence of her cryptic statement, one which seems to run counter to her earlier statement about serving the gods. Their third question is an analysis-level one, and it asks students to reconcile the two seemingly contradictory statements by Antigone by identifying a hidden motivation of hers that makes her statements consistent.

Webking and Valenzuela also describe how they use a particularly challenging, analysis-level question about Plato’s Euthyphro. The question asks students to identify the central argument of a particular passage, one that deals with the relationship between justice and piousness. The question is one that Jean McGivney-Burelle would call a “horizontal question” since students answering the question are typically split evenly among three answer choices. Webking and Valenzuela note that one of the three popular responses can’t be supported by the text. Students who argue for this answer choice quickly realize that they were projecting their own perspectives on the text, not arguing from the text. This is a useful metacognitive moment for these students. The class discussion then focuses on the remaining two popular answer choices. Making sense of these two choices requires the students to grapple with categorical logic, the kind that is well-represented by Venn diagrams. Once the students have discussed their way to the correct answer, they realize the value of categorical logic in making sense of arguments like the ones Plato makes—another metacognitive moment.

The Plato example comes from one of the authors’ smaller, upper-level courses, and they assert that “it is in a smaller class that the [classroom response] system is at its best in encouraging discussion and precise argument.” They reach this conclusion, in part, because of the ability of their classroom response system to report to the instructor individual student responses to clicker questions as those responses are submitted. The authors use these individual, real-time results to guide their post-vote discussions, focusing on “groups which had difficulties in reaching consensus, students or groups which answered particularly quickly or particularly slowly, students who disagreed with their groups, students who changed their minds, and so on.” They argue that the ability to see individual, real-time results is important in leading effective post-vote discussions since it allows instructors to analyze “each student’s rational odyssey with each question.”

Also in the article are two examples of student perspective questions the authors use to motivate particular topics in their courses. In one example, they ask students to identify questions they aren’t likely to ask someone they’ve just met. Invariably, students identify the questions about religion and politics. The authors point out to students that one reasonable conclusion from this is that religion and politics are the least important things to know about when getting to know someone. This motivates students to want to learn why this social phenomenon exists.

Comments: This would be a great article to give a faculty member in political science or philosophy who’s interested in getting started teaching with clickers. Webking and Valenzuela provide a concrete, interesting example of a guided close reading of a text (Antigone) using clicker questions of increasing difficulty. This is a great model for instructors in the humanities and social sciences interested in helping their students develop critical thinking and close reading skills. I wish, however, that they had included some voting data in this example and had discussed how they use the results of these questions to guide discussions, as they did with their Plato example.

The Plato example is a great model of clicker use in text-based courses, too. One reason is that the approach Webking and Valenzuela use leads students to appreciate the nature of argument in their discipline. They write, “In time, and actually not very much time, students learn to care more about the strength of the argument than about having their initial position defended as right.” The authors present a useful list of options for leading these kinds of class discussions—focusing on groups that were conflicted, students who answered quickly or slowly, students who changed their minds, etc.

The authors assert that the quality of discussions they can foster depends on the availability to the instructor of real-time, individual voting data. Not all classroom response systems have this feature and, in my experience, instructors who have the option of looking at individual results as they come in don’t frequently take advantage of this option. I think that perhaps the availability of real-time, individual results isn’t as critical as Webking and Valenzuela assert. I’ll often have my students vote on a question individually, then discuss it in groups, then vote again. I’ll sometimes ask for a student who changed his or her mind from the first vote to the second vote to explain his or her reasoning. I can also see asking for a student who disagreed with his or her group to contribute to the post-vote discussion.  (That’s a nice idea, one that I’ll have to try soon!)

My approach, using the aggregate and not individual voting data, relies on students who fit certain profiles volunteering to share their perspectives with the class. Webking and Valenzuela’s approach doesn’t rely on volunteers, but it isn’t quite cold-calling, either, since they select students only after the students have had a chance to consider and respond to the clicker question. I’d like to call this “warm-calling” since the students have had a chance to warm up to the question and since the instructors aren’t calling on students without any knowledge of what those students might contribute to the discussion. I’m not familiar with many instructors who practice warm-calling.  If you do, I’d love to hear from you in the comments about your experiences doing so.

Image: “Coffin Sculpture of Antigone” by Flickr user Xuan Rosamanios / Creative Commons licensed

Although relatively few instructors in the humanities use clickers, if there’s one discipline in the humanities where clickers are starting to get some traction, it would be philosophy. I interviewed a couple of philosophy faculty members for my book (including Ron McClamrock of SUNY-Albany), and I’ve recently found a few online resources for using clickers in philosophy, listed below.

Why the particular interest in clickers among philosophy instructors? Perhaps it’s because some teach courses in logic, and these courses are often more like math courses (where clickers are more mainstream) than typical humanities courses. Perhaps it’s because some philosophy instructors teach relatively large classes–larger than is typical in English and language instruction, certainly–and clickers excel in large classes. However, I suspect the primary reason clickers have been adopted in philosophy is because philosophy instructors like to ask what I call “student perspective questions” in my book. These opinion and experience questions work beautifully in ethics courses, and I imagine they work well in other philosophy courses, as well.

On the Teaching Philosophy 101 site, John Immerwahr provides an introduction to teaching with clickers in philosophy courses.  He suggests a few uses of clickers that are of particular use in teaching philosophy.  For instance, he suggests asking students a few opinion questions at the beginning of a unit to surface their perspectives on the topic, helping them have a great stake in the discussion that follows.  He also suggests asking the same questions before and after a topic is discussed as a way to show students that “serious discussion of issues actually matters to how people think (a point which they sometimes don’t get initially).”

Immerwahr also stresses a point about clickers that is sometimes subtle: They can be used to generate “meta-conversations,” as he calls them.

Interestingly, the wording of the questions themselves often creates prompts for discussion. Student like to discuss why the class voted as it did, and people will sometimes make interesting distinctions (e.g., a student might say “If the question has said ‘can’ make a difference instead of ‘will’ make a difference, I would have voted differently,” which can then lead into another interesting discussion).

In my talks on teaching with clickers, I’ll often mention that the results display itself can generate useful discussion.  Asking students why the class voted as it did can often lead to productive discussions of assumptions students make about themselves and each other.

Immerwahr’s example also reminds me of another point I often make, that the wording on clicker questions need not be as precise as the wording on exam questions.  One reason is that if the question isn’t worded exactly right, an instructor can still make it work during the discussion of the question.  Another is that clicker questions can be modified and asked again based on student comments during discussion.  In Immerwahr’s example, for instance, the instructor could easily change “will” to “can” in the question and re-poll the students.

For an expanded version of Immerwahr’s introduction to clickers, read his Teaching Philosophy article, “Engaging the ‘Thumb Generation’ with Clickers.”  The article includes more discussion of the clicker uses mentioned above, as well as other uses, and features several sample questions.

And for even more resources on using clickers in philosophy instruction, visit the Peer Instruction in the Humanities project out of Monash University in Australia.  This site features a step-by-step guide to PI, advice on designing a PI lecture, a description of a sample PI lecture, examples of various types of clicker questions appropriate for this teaching context, and even a question bank organized by topic!  I’m very glad to know that there’s a humanities clicker question bank out there to complement existing question banks in the sciences.

Image: “Portrait of Erasmus Desiderius“, Andreas Praefcke, Wikimedia Commons

Reference: DeBourgh, G. A. (2008). Use of classroom “clickers” to promote acquisition of advanced reasoning skills. Nurse Education in Practice, 8(2), 76-87.

Summary: Gregory DeBourgh provides a useful introduction to using clickers in nursing education, focusing on pedagogical strategies that use clickers to promote critical thinking.  His exploration of critical thinking in the context of nursing education is particularly interesting.  Here’s a sample:

“Reasoning is about using intellectual power to draw conclusions, form judgments, and make inferences based on evidence, education, and experience… The practical significance of acquiring skill in advanced reasoning is to move to the level of predictive clinical reasoning which enables one to anticipate both ideal and likely outcomes given a set of data.”

DeBourgh argues that using classroom response systems to engage students in high-level questions is an effective strategy for developing their critical thinking skills.  He supports this assertion by drawing on the literature on the roles of feedback and questioning in learning and by sharing concrete examples of clicker uses in nursing education.

Included are three sample questions, including a “one-best-answer” question that asks students to identify the likely cause of a particular symptom shown by a patient in a case study.  DeBourgh endorses the use of such questions since they better represent situations students are likely to encounter in clinical settings where they must deal with ambiguity.  He also suggests asking question sequences based around patient cases that “change the focus to add new variables,” noting that doing so also reduces the cognitive load students experience when familiarizing themselves with a new case.

DeBourgh makes a good argument for using clicker questions to model critical thinking skills for students:

“Anticipate likely incorrect responses and prepare ‘talking points’ for discussion as this facilitates ‘thinking on your feet’ and makes more visible to students how an expert uses heuristics, reasoning, and refined problem-solving skills to gain command of a clinical situation.”

Asking questions designed to provide an opportunity for the instructor to model critical thinking is one instance of many DeBourgh describes of crafting questions to meet particular teaching and learning objectives.  In doing so, DeBourgh draws on articles by Ian Beatty on good question design, transferring Ian’s advice to the context of nursing education.

DeBourgh also points out that clicker questions embedded in PowerPoint can be particularly useful in nursing, a field which frequently uses pictures, diagrams, sound clips, and video–media that can also be embedded in PowerPoint.  He also notes that nursing courses often involve discussion of nursing ethics and student opinions about ethical decisions, topics that lend themselves well to clicker questions.

The article also includes results from a study survey about clicker use.  Student responses to rating questions are summarized, and student responses to open-ended questions are presented, as well.

DeBourgh ends with a few challenges involved in teaching with clickers, two of which are particularly significant.  He notes that since instructors can track student performance in a class on a daily basis, expectations for students are raised, which is not popular with all students.  DeBourgh also speaks to the increased expectations for instructors:

“The greatest challenge is the new role for faculty to plan the curriculum and instruction around ‘deep comprehension’ rather than ‘covering content’ using a traditional lecture format.”

Comments: I read this article in advance of my presentation at the Health Professional Educational Research Symposium earlier in the month, and I was particularly impressed with Gregory DeBourgh’s eloquence in describing critical thinking in the context of nursing education and in describing ways that clicker pedagogies can foster those critical thinking skills.

As I’ve tried to capture above, DeBourgh describes a variety of ways of using clickers in nursing education, and he included one approach that was entirely new to me, one inspired by the 50-50 option in the television game show Who Wants to Be a Millionaire? DeBourgh suggests that before the correct answer to a clicker question is revealed to students, an instructor might ask the students which answer choices should be eliminated.  It’s a little unclear how DeBourgh implements this, but I can even imagine setting up a multiple-mark question with four answer choices, then asking students not to select the one correct answer but to select two incorrect answers.  This would offer a nice change of pace in question format and would help students focus on more than just the correct answer.  It’s often useful for students to consider why some answer choices are plausible on the surface but actually incorrect.

Hopefully it’s also above that DeBourgh puts an emphasis on teaching with case studies (multimedia case studies, at that) in his article.  I understand that case study methods are perhaps more common in nursing than they are in other disciplines, and I appreciated reading this article as a way to better understand why that was the case.  DeBourgh’s comments about using clickers for discussing ethics also helped me better understand the disciplinary context here.

If you’re a nursing educator, please share a thought or two about using clickers in your field in the comments section!

Update: Greg DeBourgh emailed me and clarified his 50-50 technique.  Here’s what he said:

I display the potential four-answers to a given question, then before the students “vote” with their clickers, I ask for a volunteer or select a student at random (my clicker system has this feature) and ask the student to eliminate 2 of the 4 potential answers and to explain why they are eliminating these two. This speaking out loud of their rationale for eliminating two of the potential answers that are not related to the question strengthens the students’ reasoning skills. They actually get quite good at it. If the student I called upon to answer hesitates or is reluctant to speak, I invite them to choose a “consultant” in the room to help them out. I hope this clarifies a bit for you.

I asked Greg what he does if the student eliminates the correct answer.  Here’s his response:

If the student eliminates one of the correct answers, it is still learning, and so I ask “does everyone agree with the 50/50 elimination?” If someone objects, I ask for their rationale. If no one objects, I just let the process go and during the “reveal and rationale” we talk about why each answer is incorrect or correct.

Thanks, Greg, for this clarification, and for this great use of clickers.

Here are a few interesting ideas shared during the first set of talks about teaching with clickers at the Joint Mathematics Meetings earlier today.  (This post’s title inspired by the following books seen at the exhibits: Calculus Gems, Mathematical Diamonds, More Mathematical Morsels, and Biscuits of Number Theory.)

Kathryn Ernie (University of Wisconsin-River Falls) shared ways that she and her colleagues use clickers in their college algebra courses.  One use that she mentioned was to warm students up before “traditional” in-class quizzes.  By asking students a clicker question or two, then discussing those questions prior to a graded quiz, the students are able to approach the quiz with a little more confidence.

Ben Galluzzo (Shippensburg University) also talked about graded quizzes.  However, in his case, clicker questions aren’t warm-ups for the quiz; the clicker questions are the quiz.  Clickers allow Ben to turn his quizzes into learning experiences for his students.  After each quiz question, he discusses the question with the class before moving on to the next question.  This can work particularly well when he has more than one quiz question of the same type.  Students who miss the first one can learn from the discussion of that question and apply what they’ve learned to the subsequent question.  Students like this because they appreciate the chance to redeem themselves.

Aprillya Lanz (Virginia Military Institute) mentioned that teaching with clickers help students stay awake and engaged during class.  This is particularly important for her since many of her students are freshmen (“rats” as they’re called at VMI) who are required to participate in all kinds of strenuous physical activities, particularly on Sunday nights.  This can make for some very sleepy students in Monday morning classes.

Daniel Joseph (also VMI) described a problem that those who teach calculus often see: His calculus students often struggle because of pre-calculus misconceptions.  They can’t tackle the calculus because they get tripped up by algebra and other pre-calc topics.  He described several methods he’s tried to combat this, but he finds that the students’ over-confidence trips keeps these methods from working.  The students say they “know” all the pre-calculus material because they’ve studied it in the past.  Daniel appreciates how clickers provide his students with frequent evidence that they don’t know it as well as they think they do.

Daniel shared one approach to attacking this problem–using clicker questions in a pre-semester pre-calculus course for incoming freshmen.  He’s interested in hearing ideas for hitting this issue, with or without clickers, in the calculus course itself.  Any ideas?

(By the way, Daniel used the phrase “attack this problem” at least five times in his presentation.  Given that he teaches at a military institute, I figured that was language that comes naturally to him.  Thus my use of the verbs tackle, combat, attack, and hit above!)

Note: I’ve had a couple of problems with this blog recently–the RSS feed stopped working and I haven’t had any time to post.  I’ve now fixed both problems.

I’m a huge fan of the World’s Technology Podcast from the BBC and PRI.  Clark Boyd puts together a great collection of technology stories from around the world every week–not the usual stories about the latest gadgets, but stories about how technology is impacting society and culture around the world.  Great stuff.

A couple of months back Clark ran a story on Chinese students using cell phones to cheat.  I tweeted Clark and suggested that he take a look at some of the positive ways teachers are using cell phones in the class room.  I recommended he take a look at what Greg Kulowiec has been doing in his 9th grade history classes in Massachusetts.  I’ve blogged about Greg’s use of Poll Everywhere, and Greg’s posted a great video showing how he uses this text-messaging-based response system to ask his students ethical questions about the Holocaust.

Well, Clark Boyd took me up on my recommendation and interviewed Greg about his use of technology in his classes.  The interview ran near the beginning of episode 256 of Clark’s podcast.  Greg talks about how he had his students use their cell phones during class to call people they knew and quiz them about the US Constitution and how he uses his students’ cell phones as part of a classroom response system to engage them during class.

Greg also describes a couple of ways to leverage his students’ cell phones’ camera functions.  He had his students take photos during a class trip to the New England Aquarium and send them to Greg’s Evernote account for later use in an Animoto video, for instance.  He also plans to have his students create their own Evernote accounts so they can take photos of references as they do research, send those photos to their Evernote accounts, and use Evernote’s tagging ability to organize their research notes.  Very cool stuff.  I think it’s time I signed up for an Evernote account.

Props to Clark Boyd for being open to listener suggestions and to Greg Kulowiec for being willing to share his innovative uses of technology.  You can follow Greg’s continuing experiments on his blog, The History 2.0 Classroom.

Here are a couple more clicker questions I tried out in my cryptography class recently.

“Cryptography was the decisive factor in the Allied victory in World War Two.”

1. Strongly agree
2. Agree
3. Disagree
4. Strongly disagree

Surprisingly, most students agreed or strongly agreed with this statement.  Arguments for this statement focused either on the tactical role of information gained via code-breaking in particularly important battles (e.g. Midway) or the more generally important role of military intelligence.  Arguments against this statement included the assertion that cryptography may have shortened the duration of the war but did not change its outcome.

Given the lopsided results, I asked for those who agreed with the statement to provide some reasons for their answers.  After hearing from a few students, I then asked those who disagreed with the statement to argue their side.  This didn’t quite generate the discussion I had hoped it would, so I asked those who strongly agreed with the statement to state their reasons.  This back and forth went well enough, but it probably would have worked better had I asked the students to respond more directly to each other’s arguments.  Had the results been more split, this approach would likely have worked very well.  As it was, it probably would have helped for me to play devil’s advocate more actively by defending the “disagree” response.

As you can tell, the question above reads like an opinion question but really calls for critical thinking!

Here are two more clicker questions that go together and worked very well:

Singh writes on page 149 that “the creative codebreaker must ‘perforce commune daily with dark spirits to accomplish his feats of mental ju-jitsu.’”  Which is more important to successful codebreaking–logic or creativity?

1. Logic
2. Creativity
3. Both are equally important.

How would you have answered the previous question before you took this course?

1. Logic
2. Creativity
3. Both are equally important.

I only had a couple of minutes for these two questions, which didn’t provide much time for discussion.  However, the results of the two questions were very different, which was fascinating!  For the first question, most students said that both logic and creativity are equally important in breaking codes.  For the second question, most students indicated they would have said logic was more important than creativity.  The shift was fairly dramatic–over 60% of students chose “both” on the first question and similar numbers chose “logic” on the second question.

I asked my students why their perspectives changed since the course began.  They indicated that learning about the invention of ciphers as well as understanding how difficult it has been and is to break ciphers, their appreciation of the role of creativity has grown.  Inventing a new, secure cipher takes a creative act, so breaking that cipher also takes creativity.  Their own experiences breaking codes this semester in the problem sets have reinforced the notion that creativity is essential.

I hope that this question provided my students with a “metacognitive moment”–a chance to step back and consider what (and, to some extent, how) they have learned in this course.  The dramatic difference in the results of this question likely enhanced this “learning about learning” to the extent that it occurred.

Have you used clickers to help create “metacognitive moments” in your teaching?

It’s the start of a new semester at most colleges and universities, so I thought it would be interesting to hear how instructors use clickers on the first day of class.  I’ll share what I did on the first day last week, and I’ll invite you to share your creative first-day uses of clickers in the comments below.

I’m teaching an undergraduate linear algebra course this spring.  For an application-oriented course like this one, I like to spend the first day of class showing students examples of the kinds of problems they’ll be able to solve by the end of the semester.  I find that this helps students see value in studying the subject at hand.

One of the problems we looked at last week dealt with modeling a population of spotted owls.  (I adapted this problem from David Lay’s linear algebra textbook.)  The population consists of juveniles (owls less than one year old), sub-adults (owls between one and two years old), and adults (owls at least two years old).  Each year some proportion of adults produce juveniles as offspring, some proportion of juveniles survive to become sub-adults, some proportion of sub-adults survive to adulthood, and some proportion of adults survive to live another year.  The relative sizes of the proportions I just mentioned determine what happens to the overall population level over time.

I had my students consider various birth and survival rates and respond to the clicker question, “Given these rates, in the long run does the population increase, decrease, or level off?”  In the first scenario, it was clear to most students after a few minutes of small-group discussion that the high birth and survival rates meant that the population was likely to increase over time.  Having figured out the first scenario, the students responded very quickly to the second scenario, one in which low birth and survival rates meant that the population decreased over time.

Then I presented a third scenario, one in which it was not so clear what would happen to the population given a particular set of birth and survival rates.  Although one student quipped that the answer must be “level off” since that was the only answer choice we hadn’t used, most students acknowledged that it wasn’t obvious from the given rates what would happen to the population in the long term.  This allowed me to make the point that we’ll have to develop some mathematical modeling tools in order to analyze this situation accurately and make valid predictions.  This, I hope, provides students with some motivation to learn about those tools later in the semester.

So what kind of clicker questions do you ask on the first day or in the first couple of weeks of class to get your students off to a good start?

An article in last Friday’s edition of the journal Science on peer instruction, a pedagogy often used with (and enhanced by) clickers, has received some attention.  I thought I would comment on it here…

Reference: Smith, M. K., Wood, W. B., Adams, W. K., Wieman, C., Knight, J. K., Gulid, N., & Su, T. T. (2009). Why peer discussion improves student performance on in-class concept questions.  Science, 323(5910), 122-124.

Summary: The peer instruction teaching method popularized by Eric Mazur (Mazur, 1997) involves having small groups of students discuss and respond to questions, usually multiple-choice, conceptual understanding questions asked via clickers.  There’s strong evidence (Crouch & Mazur, 2001; Fagen, Crouch, & Mazur, 2002) that it is an effective teaching method, but some have questioned the extent to which learning occurs during such peer discussions.  Perhaps students “simply choose the answer most strongly supported by neighbors they perceive to be knowledgeable,” write the authors.  This study provides fairly convincing evidence that this is not the case and that real learning occurs during in-class peer discussions.

The authors, most from the University of Colorado-Boulder, looked at data from sixteen pairs of clicker questions asked in an undergraduate genetics course.  Each pair of questions were considered to be “isomorphic,” that is, questions about the same concept with different “cover stories.”

Students answered the first question in each pair on their own, and, on average, 52% of students answered these questions correctly.  Then students discussed the same question in small groups and answered it again, increasing the success rate to 68%.  Then students answered the second, isomorphic question on their own without discussion, and 73% of students answered these questions correctly.

Furthermore, the authors point out that of the 48% of students who answered the first question in each pair incorrectly on their own, 42% of them answered the question correctly after discussion.  Of these students, 77% were then able to answer the second question correctly on their own, indicating that they had learned something useful from the peer instruction time.

Additionally, the authors categorized the sixteen question pairs as easy, medium, or difficult, depending on the percentage of students who answered the initial question correctly.  For the difficult questions, the gains from the first question to the second question were even more dramatic: 16% of students answered the first questions correctly on their own, 32% answered the first questions correctly after discussion, and 54% answered the second questions correctly on their own.

These results indicate that students learned something during peer instruction time, something they could apply toward answering the second question in each pair on their own.  The effects of peer instruction appear to be even greater for more difficult questions.

Comments: These are very interesting results and provide fairly compelling evidence that peer instruction has real value in facilitating student learning.  I’ve seen some unpublished (to my knowledge) research by Dennis Jacobs, a chemist at the University of Notre Dame, along these lines showing the value of peer instruction.  I’m glad to see some published work in this direction, published in Science no less!

The comments on the Inside Higher Ed post on this story point out a couple of important points.  One is that knowing how to answer a question on a particular concept when that concept is isolated from other course concepts does not mean that a student will be able to answer a similar question on a test covering multiple concepts.  That is, applying a particular problem-solving approach does not imply the ability to select the correct approach when confronted by a new problem.  This is a good point, and it also raises the issue of long-term retention.  To what extent might learning gains from peer instruction be evident weeks later on midterm or final exams?

It’s also worth noting that in this study, the role of the instructor was minimized.  I’m guessing that the instructor gave the students some instructions for discussing the clicker questions, but beyond those instructions, the instructor did not provide any explanations that would affect student performance on these questions.  The gains seen here derive from peer instruction alone.  Peer instruction supplemented by instruction by the teacher might show even greater gains.  That might be something worth investigating in future studies.

Additionally, the authors point out that “there may also be a learning benefit to considering successive clicker questions on the same topic.”  This gets at the role of question sequences seen, for instance, in Reay, Li, and Bao (2008).  A study using a control group methodology might ask some students to discuss the first question in each isomorphic pair and other students just to spend more time on their own thinking about it.  That would provide some potentially useful data.

Finally, one of the comments on the Chronicle‘s Wired Campus blog entry on this study pointed out that the study doesn’t really say anything about the role of clickers in facilitating peer instruction.  Would similar gains in student performance be seen were clickers not used?  I would argue that clickers enhance peer instruction in a few ways–by creating the expectation that all students should participate and commit to answers thereby increasing engagement and preparing students to participate more actively in peer instruction.  Further study of the role of clickers in creating the gains seen in this study would be interesting.

Having led a number of workshops for faculty new to using clickers at my home institution and elsewhere, I was looking forward to the chance to lead a workshop for faculty with more experience teaching with classroom response systems at the clickers conference in Louisville the other weekend.  I knew I wanted an interactive session that helped participants share their experiences using clickers with each other and helped them see new ways they might make use of clickers.

I based the main activity in my session on one that Shelley Smith of the University of Minnesota at Duluth led at a session at the 2007 POD Network conference.  Since I developed something of a taxonomy of clicker questions during the process of writing my book, one featuring types of questions as well as types of activities commonly used with clickers, I had participants in my session at Louisville first write about a particularly effective use of clickers from their own teaching experience.  Then I described my taxonomy using examples drawn from my book.  (Thanks to Edna Ross, whom I interviewed for my book, who was present to share her story about using clickers to generate “times for telling.”)  Next I had participants use my taxonomy to categorize the clicker uses they had earlier described in writing.  I then asked a couple of participants share their uses of clickers with the whole group.  After each of these participants did so, I asked the other participants to categorize what was described according to my taxonomy and submit their categorizations using clickers.

As happened with Shelley Smith’s session at POD, this activity generated useful discussion about the ways in which instructors use clicker questions.  This was the first time I used this activity in a workshop, and it went very well.  (Shelley used Bloom’s Taxonomy in her activity, and she brought with her questions written by faculty members in various disciplines at her home institution instead of having participants write about their own uses of clickers.)

One participant shared a use of clickers she heard about from a colleague in the English Department at her institution.  He first posed a clicker question that asked his students to voice their opinion on the death penalty–did they support it or not.  Then he read a poem about a prisoner on death row.  Following the poem, he asked his students to respond again to the opinion question about the death penalty.  The results of the second question were different from the results of the first question, which not only provided the instructor and students with a view into student opinions, but also showed to the students the power of poetry to affect change in people’s perspectives.  (I didn’t catch the name of the participant who shared this story.  If you know who it was, please let me know.)

Ken Jones of the University of Texas at San Antonio shared a way in which he has used clickers in the past.  He first asked the students in his course if they would be willing to commit some particular breach of business ethnics.  (I forget the particular breach.  It was something about sharing private information, I think.)  Most of the students said they wouldn’t.  He then asked them if they would do so for some amount of money.  A few students said they would.  Ken then asked them the same question several times, each time increasing the amount of money.  More and more students said that they would take the money.

As with the poetry example above, Ken’s use of clickers revealed to him and his students something about his students’ perspectives and set the stage for a productive discussion of the topic at hand.  Both examples also generated productive discussion at my session regarding the multiple purposes instructors often have for using clicker questions.

After this activity, I asked participants which types of questions and activities they used most frequently in their teaching.  Conceptual understanding and application questions were the most popular, each chosen by about a third of the participants.  Student perspective questions came in third, chosen by 10% of the participants.

The activities most frequently used with clickers were uncovering student learning (chosen by 32% of participants), evaluating student learning (21%), generating small-group discussion (21%), and generating classwide discussion (16%).  Disciplinary differences were rather strong on this question.  Of those in the STEM fields (science, technology, engineering, mathematics), 88% indicated they used clickers foremost to uncover or evaluate student learning.  None indicated that their primary use was to generate small-group or classwide discussion.  The responses from those in the humanities were just the opposite–none used clickers primarily to uncover or evaluate student learning and all of them indicated that they use clickers mainly to generate small-group or classwide discussion.

This was a non-scientific survey, of course, but I found these disciplinary differences interesting.  I know that a few of the larger campus studies of clicker use looked at the various ways that faculty use clickers.  I’ll have to see if these discipline-specific findings are consistent with those larger studies.

Thanks to those who participated in my session last weekend.  My only regret is that I didn’t collect what the participants wrote about their uses of clickers.  Had I done so, I could have shared those with the participants over email.  Next time!

Doug Duncan, an astronomer at the University of Colorado and director of the Fiske Planetarium, gave the afternoon keynote address at the clickers conference in Louisville last Saturday.  Much of Doug’s talk, like Tim Stelzer’s talk in the morning, was directed at faculty relatively new to clickers, and his advice on using clickers effectively was all very sound.

One piece of advice Duncan shared stood out to me.  Several times in his talk, Duncan stressed the importance of telling students why you have them engage in peer instruction via clicker questions (if that’s how you use clickers).  It’s not always obvious to students what they should be getting out of these kinds of learning experiences.  Sometimes students think the point of these activities is to punch the correct button on their clickers.  These activities are usually intended instead to develop students’ conceptual understanding, problem-solving skills, and critical thinking skills, but this only happens if students engage in meaningful discussions with their peers.  As Duncan said, “No brain, no gain.”

Explaining your pedagogy to them (in ways that make sense to them) can help them get more out of the experience and develop metacognition (learning about their own learning).  Plus, it can help them appreciate these activities more, which reduces negative perceptions of the use of clickers.

Duncan shared a sequence of clicker questions he uses to help his students see the value of engaging in peer discussion.  First, “Do you prefer I tell you the answers to clicker questions or give you hints?”  At least half of his students typically say they prefer him to tell them the answers.  Second, “Which leads to longer lasting learning?”  Most students admit that Doug giving them hints leads to deeper learning.  Finally, just to make sure the point is clear, “Do you expect to get a job where you have to think for yourself?”  Almost all his students say yes to this one!

These are great examples of what I call “syllabus reminder questions” because they remind students of the kinds of messages often communicated via syllabi.  Students aren’t often in a great position to receive these messages on the first day of class, so reminding students of them throughout the semester via clicker questions can be an effective use of clickers.  (I describe these questions in the “Monitoring Questions” section of my book.)

Conversations between instructors and students about the teaching and learning process can play an important role in helping students learn more effectively.  I’m glad Duncan stressed this point in his talk and provided us with ideas for how to start these conversations.