Back in May 2010, I led a webinar on teaching with clickers as part of the CIRTLcast series for the Center for the Integration of Research, Teaching, and Learning (CIRTL), an NSF-sponsored network of six universities interested in preparing future science, engineering, and mathematics faculty. The full webinar was 60 minutes, and you can access the audio recording and my slides in the CIRTLcast archive. However, CIRTL has done a great job taking some excerpts from the session and packaging them as a 10-minute YouTube video, complete with a transcript!

In the video, you’ll hear me talk about using clickers to generate small-group and classwide discussion, create “times for telling,” encourage metacognition, facilitate peer assessment, structure class time, turn quizzes into learning experiences, and make class more fun. Clickers can be used very effectively to engage students in the learning process during class, and this short video is a nice introduction to these uses of clickers.

Thanks to CIRTL for giving me the opportunity to present this webinar and for putting together this great video!

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Preservice Elementary Teachers’ Perceptions of Clicker Use in their College Mathematics Course,” Travis K. Miller, Millersville University of Pennsylvania [Slides]

In my last post, I mentioned that Janet White first used clickers in her courses for pre-service teachers at Millersville University of Pennsylvania. Another speaker in the contributed paper session back in January was her colleague, Travis Miller, who shared results of a student survey he conducted in the pre-service teacher course he taught. Travis used clickers for only six lessons during that course in each of the four sections he was teaching. His clicker questions weren’t graded, and he followed the “classic” peer instruction model each time, having students vote individually, then discuss the question in small groups, then vote again.

Travis’ students overwhelmingly (96%) liked using clickers in the course. Travis mentioned that there are very few things he does as a teacher that are as uniformly popular with his students! Almost as many students (89%) believed that the clicker activities helped them learn the material in those six lessons. Travis drilled down on this, asking students to say why the clickers were useful. The number one answer (59% of students) was that the clicker questions provided students with an opportunity to discuss and think about course content. The number two answer (23%) was that the clickers provided a sense of accountability and involvement.

Travis didn’t stop there, either. He asked his students which topics they understood better because of the clicker activities. Of the six topics that Travis addressed using clickers, sets and Venn diagrams was cited by 52% of the students as the one that most benefited from clickers. Numeration / base arithmetic was a distant second with 15%, and deductive reasoning came in third with 13%. When sharing these data, Travis floated a very interesting hypothesis. He wondered if the fact that the number one topic (sets and Venn diagrams) was a visual one led to the students selecting it as most benefited by clicker questions. I’m a big fan of visual thinking, so this comment caught my attention. Is there something special about peer instruction with clickers and visual thinking?  I’d appreciate your thoughts in the comments.

Travis’ other interesting hypothesis was that his more competitive students liked the competitive aspects of clickers (being the first to answer, answering correctly more frequently than other students, and so on), while the non-competitive students didn’t mind those aspects since they were essentially opt-in. That is, the students who didn’t want to compete could still participate fully with the peer instruction and voting process without feeling any pressure to treat it like a game. Graham, Tripp, Seawright, & Joeckel (2007) found that most students who are hesitant to participate in class liked clickers as well as those who were fine with participating, but I don’t think I’ve seen any research that compared competitive students with non-competitive students. That would make for an interesting research question.

Travis also taught some sections of his pre-service teacher course without using clickers, and he surveyed students in these sections about the potential advantages and drawbacks of using clickers. What concerns did they have about using clickers? They worried about the cost of the devices, that clickers weren’t necessary in small classes, that clicker activities take up too much class time, and that the technology might not be reliable. I found it interesting that these are among the common concerns of faculty members not already using clickers, too!

Image: “Happy Pi Day!” by Flickr user Mykl Roventine / Creative Commons licensed

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Using Personal Response Systems (Clickers) in Liberal Arts Mathematics Courses to Support a Lecture Format,” Janet A. White, Millersville University of Pennsylvania [Slides]

Just like Jean McGivney-Burelle and Kimberly Burch, Janet White shared her experiences teaching with clickers in a “liberal arts” mathematics course taken by non-majors. Unlike Jean and Kimberly, who teach relatively small sections of this kind of course, Janet teaches in a large lecture hall with 75 students per section. Janet had used clickers in courses for pre-service math teachers in the past and found them useful, so when it was her turn to teach this larger course, she decided to use them again. A classroom response system was hardly the only technology Janet used in this course: She also had students complete online homework and quizzes and she annotated her PowerPoint lecture slides using an Interwrite Mobi.

Janet used clickers on a daily basis in her course, usually either to assess students’ prior knowledge or to assess their understanding of a topic taught during lecture.  Her questions came from a bank of multiple-choice questions provided by her textbook publisher.  She counted the clicker questions as part of her students’ participation grades, but in a low-stakes manner.  Given her use of the questions as well as the source of the questions, many were on the lower levels of Bloom’s taxonomy, aimed at recall and application of procedural knowledge.  She shared an example of a prior knowledge question that asked students to find the measure of an angle that complements a 36 degree angle.  A slightly harder question aimed at assessment of something taught during the course asked students to identify the cut edge in a given graph (or to assert that the graph had no cut edge).

Student survey results indicated that 85% of Janet’s students who used clickers regularly liked using them, and 71% said that using clickers helped them learn the material.  Students who used clickers regularly during the course ended up with higher grades in the course than students who didn’t, but, of course, that can’t necessarily be attributed to the use of the clickers.  (And since clicker questions were factored in the course grade, students who participated more frequently in clicker questions would almost certainly have higher grades in the course anyway.)

Student comments about the clickers were generally positive.  My favorite one was, “I liked getting the wrong answer anonymously.”  Other comments addressed the usual points that students like about clickers: They liked the interactivity, they liked discussing questions with classmates, they liked seeing where they stood relative to their peers, and they liked the feedback on their own learning the clicker questions provided.  The only significant negative aspect for the students was the cost, about $50 in Janet’s case.

Janet found that having students discuss clicker questions in small groups led to very engaged students, even in the large auditorium environment.  In the future, she plans to write more of her own questions, instead of relying on ones from the textbook’s question bank.  She hopes to write more difficult questions that will generate even more engaged discussion during class.  She’s also hoping to find ways to reduce the technology cost to the students, either by selecting a different vendor or facilitating the resale of clickers after each semester to students taking the course the next semester.

Also, Janet mentioned that the earth science faculty at Millersville are big users of clickers.  Earth science instructors looking for advice on using clickers might want to investigate!

Image: “Recursive Daisy” by Flickr user gadl / Creative Commons licensed

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Clickers in the Classroom,” Kimberly J. Burch, Indiana University of Pennsylvania [Slides]

Kimberly teaches a “Math 101″ survey course called “Foundations of Mathematics.”  Topics covered include set theory, graph theory, and counting methods (among others), and Kimberly shared several interesting clicker questions on each of these topics.  For example, here’s one of her questions from the unit on graph theory:

How many vertices are there in a tree with 19 edges?

1. 19
2. 18
3. 20
4. Not enough information given

Kimberly practices the “classic” peer instruction technique of having students vote individually first, then discuss the questions in small groups, then vote again.  She finds that students often converge to the correct answer on the second vote.

In the example above, her students were split between 18 and 20 on the first vote, but after the peer discussion time, most students went with the correct answer, 20.  I found this interesting because the “Not enough information given” seemed to be the obvious wrong answer to this question.  A graph with 19 edges might have any number of vertices, but a tree with 19 edges can only have 20 vertices.  Students who don’t realize that trees are graphs with very specific properties might be tempted to go for the “Not enough information given” option.

I suspect that Kimberly used this question after the students learned the relationship between the number of edges and number of vertices in a tree and that this question was meant to assess whether students remembered that relationship.  Some students likely remembered that one of these numbers was one more than the other but weren’t sure which one was higher.  That would account for the split vote between 18 and 20.  Had this question been asked as an exploratory question and not a review question, I’m betting the split would have been between 20 and “Not enough information given.”

Kimberly also mentioned that she uses her clicker system’s priority ranking questions to have her students decide what topics should be emphasized during exam review sessions.  Kimberly gives her students a list of 8-10 exam topics, and the students indicate the top three or four toughest topics in order.  Kimberly said that this helps her make good use of limited exam review time by focusing on the topics the students find the most difficult.

Kimberly also shared some data from a quasi-control group experiment she conducted.  She taught two sections of this survey course and alternated which topics she covered with clickers in the two sections.  For example, section A might cover topic 1 with clickers while section B covered topic 1 without.  Then for topic 2, section B used clickers and section A didn’t.  She then compared test scores for the two sections by topic.  For some topics, students using clickers performed better on exams but for other topics, the students not using clickers performed better.  And for other topics, there was no difference.  The data was generally favorable to using clickers, but the “quasi” part of this quasi-control group experiment made it difficult to draw firm conclusions.

Image: “Point Marian Bridge” by Flickr user timmenzies / Creative Commons licensed

File this under “Better Late Than Never.”  Back in January 2010, I coordinated (with Kelly Cline and Kien Lim) a contributed paper session on teaching with clickers at the Joint Mathematics Meetings in San Francisco.  Shortly after the conference, I blogged about some clickers talks that didn’t fall in our session, but I never got around to blogging about the talks in our session!  Five months later, I’m finally getting around to sharing my notes from those talks…

“The Evolution of Classroom Voting in Contemporary Mathematics,” Jean McGivney-Burelle, University of Hartford [PowerPoint Slides]

Jean teaches a “math for the liberal arts” course called Contemporary Mathematics taken by music and arts majors among others.  She finds her students come to the course with relatively little interest or self-reported ability in mathematics, so it’s a tough crowd to teach.  A few years ago, she started teaching with clickers in order to appeal to what she calls the “thumb generation”–students used to spending a lot of time sending text messages.

Jean interspersed some clicker questions throughout her lectures and encouraged students to discuss them in small groups before voting.  She and the students liked this, but she found that most of her questions were answered correctly by most of her students  and that the small group discussions didn’t involve much debate among students.  The next year, Jean decided to ask tougher questions.  She calls them QEDs–Questions to Encourage Discussion.  She aimed for the analysis, synthesis, and evaluation levels of Bloom’s Taxonomy.

For example, in Jean’s first year using clickers she gave her students a preference schedule–a list of how each voter in an electorate (only four of them to keep things simple) ranked all of the candidates.  She then asked her students to determine which candidate would win the election using the instant run-off voting scheme.  This is a straight-forward application of a particular algorithm.

The next year, Jean asked another question in which she shared a preference schedule with her students and asked them to analyze it.  However, this time, she asked the question at the beginning of the unit on voting schemes and asked her students to indicate which of the candidates had the best case for winning the election.  There’s no single correct answer to this question since winner of an election (well, one involving at least three candidates) depends on what scheme you use to count the votes.  This is a great example of a one-best-answer question (since students are asked to select the one answer they think is best among multiple reasonable answers) used to create a time for telling (since it’s used to make the point that which voting scheme you use matters).

Jean found that these more challenging and ambiguous questions generated longer and more engaged small group discussions as well as more “horizontal” bar graphs–ones indicating significant disagreement among the students.  Looking ahead, she plans to build on this success by writing questions designed to develop mathematical habits of mind–an important goal of this course.  For example, here’s a sample question she shared aimed at pointing students towards the notions of proof and counterexamples:

Suppose there is a majority winner in an election. Will all of the voting methods we have studied thus far always pick that winner?  Yes or no?  If you answer yes, prepare to defend your answer. If you answer no, have a counterexample ready.

I really like this question.  It has a degree of ambiguity that students often find disconcerting, but it also reminds students of how they’ll need to defend their answers, which should help put their minds at ease.  As Jean noted in her talk, in a course like this one, it’s more important students develop mathematical habits of minds (like the ones surfaced by this question) than learn particular math content areas.  I hope this kind of question helps with this objective.

Stay tuned to the blog for more notes on these talks over the coming days…

Image: “Deep Down Inside, We All Love Math” by Flickr user Network Osaka / Creative Commons licensed

I’ve often said that those teaching in the social sciences have the most options for using clickers.  Both content and opinion questions are typically on-topic in a social science course, giving these instructors the ability to use clickers in just about any way imaginable.  Case in point: The video below by Russell James, who teaches in the housing and consumer economics program at the University of Georgia.

James covers a lot of ground in this video.  He shares examples of several types of clicker questions he uses, including student perspective questions (sometimes used to connect student opinions with results from national opinion polls), experiment questions (in which students participate in experiments designed to illustrate certain economic behaviors), and prediction questions (in which students predict the outcomes of research experiments from the literature).  James moves very quickly in this video, so be ready to pause it in order to read his sample questions.

James mentions other uses of clickers, too, such as taking a minute at the end of each class to ask students the kinds of rating questions that typically appear on end-of-semester course evaluations.  He says this is the “number one” use of clickers that has transformed his teaching, since it generates regular data on his teaching effectiveness.  James mentions a use I would call a monitoring question–asking students to click in when they’ve finished a particular task.  He notes that this lets him know when it’s time to move on after an activity and that the count of students who have finished displayed on-screen sends a message to students who aren’t keeping up with their peers.

James also describes a game he calls “clicker wars.”  In this game, often used to review for exams, he divides his students into groups, perhaps based on gender or class year.  Each group is then divided into teams of two or three students each, and each team is given a single clicker.  James then poses questions to his students, and each team must come to consensus on its answer.  If a team misses a question, they’re out of the game as a team, but can still help other teams in their group.  The winning team gets some kind of prize at the end of the game, and the winning group gets a prize, too, although a lesser one.  James says this gives students a lot of incentive to stay engaged in the game throughout.

James also suggests a few ways to handle students who cheat with clickers by bringing their absent friends’ clickers to class, making it appear that those friends are present.  Most of James’ suggestions I’ve mentioned here on the blog before, but he had a novel one, too.  He suggests taking a digital photo of the class as a deterrent.  If a student’s clicker says that student was present but the student isn’t in the photo, that becomes an honor code violation.  James says that telling students you’re doing this will prevent some cheating.

Thanks to Russell James for sharing his creative ideas for teaching with clickers!

Just over a year ago, I shared a story here about a clicker question I used in one of my math courses that didn’t go as planned during class.  I titled my post “Flexible Clicker Questions” because I wanted to make the point that clicker questions that seem to be poorly written can turn into real learning opportunities for one’s students.  If you put a poorly worded multiple-choice question on an exam, you’re in for a lot of student complaints and regrading.  However, in class, a poorly worded multiple-choice clicker question can, with a little agility on your part, turn out great.

I mention this because Mitch Keller recently described a similar incident in his math course over on his blog, Partially Ordered Thoughts.  He posed a particular clicker question with what he thought had a single correct answer.  His “correct answer” was indeed the most popular student response to the question, but more than 60% of students selected other answers.  Mitch wisely had his students discuss the question in small groups and then led a classwide discussion of the question.  Not only did he surface the correct reasoning for the “correct” answer, but he discussed the other answer choices, too.  It turned out that there were reasonable arguments for not one, but three of his answer choices.  Mitch writes:

A natural first reaction to a slip-up in a clicker question is almost always “Drats! I thought I’d done that perfectly.” However, it became a teachable moment. In reality, we were able to discuss far more aspects of generating functions than I intended with the question.

Have you used a clicker question that turned out to be poorly worded, yet resulted in valuable class discussions?  Please share below!

Image: “Untitled” by Flickr user Maurizio Polese / Creative Commons licensed

I was looking over my notes from the 2010 Health Professions Educational Research Symposium (HPERS) hosted back in January by Nova Southeastern University, and I was reminded of a couple of interesting points Jim Vanides of Hewlett-Packard made in his closing keynote.  Jim had shared several fascinating ways that educators receiving HP grants have been making use of tablet PCs in the classroom.  I had been familiar with ways in which laptops are often used in the classroom, but I hadn’t heard as much about uses for tablet PCs.

Jim described several examples in which the “digital ink” that tablets provide through their stylus interfaces played key educational roles.  For example, students in an engineering class can draw or design various diagrams on their tablet PCs, then send those to the instructor to be shared and discussed with the whole class.  Sure, students could prepare their diagrams before class for sharing, but sometimes having students work on “deliverables” during class–when the instructor and fellow students are around to provide feedback during the design process–is advantageous.

During the Q&A after Jim’s keynote, I asked him a question I’ve posed here on the blog several times: If students are submitting responses to free-response questions during class, how can an instructor process and use 20 or 30 or more student responses in a timely fashion during class?  With multiple-choice clicker questions, the bar chart is the perfect way to display the results.  How to handle free-response questions?

Jim made two very good points in response to my question:

1. Since humans process visual information very quickly, having students submit diagrams, drawings, photographs, and other visual responses means instructors can often make sense of dozens of student responses very quickly during class–more quickly than with textual responses.  As a big fan of using visual thinking tools in my work, I really liked this answer!  It also argues for using devices with touch interfaces, like tablets and smart phones, as part of classroom response systems used with open-ended questions.  (Jim also noted that in science, engineering, and mathematics, where people often think with pencils in hand, digital ink can be very useful.)
2. Instructors practicing the “usual” version of peer instruction, in which students respond to a clicker question on their own, then discuss it with peers, then vote again, can start processing student responses to open-ended questions during the peer discussion phase–while students are engaged in talking with each other.  This gives the instructor a little more time to make sense of student responses and decide how to discuss them with the class.  This is not unlike taking a “backchannel break” during class in which students brainstorm questions in small groups and submit them to the instructor via backchannel.  (I used this approach in a recent workshop on lecturing.)

As regular readers of this blog know, I’m very excited by the educational possibilities of having a class full of students with Web-enabled, app-enabled, touch-screen devices.  I was glad to hear a few new and very useful ideas on this topic from Jim at the HPERS Conference.  I think it’s often helpful to hear from people somewhat out of the usual academic circles I run in for great new ideas!

Image: “40+86 Tablet” by Flickr user bark

I was quoted this morning in “At Universities, Is Better Learning a Click Away?“, an Associated Press story on the future of classroom response systems by AP reporter Eric Gorski.  The story features Michael Dubson, who teaches physics with clickers at the University of Colorado-Boulder.  CU-Boulder, and its physics education research group in particular, has been very active in the world of clickers (including contributing to these great videos), and I was glad to hear Michael Dubson’s perspectives on the technology in the AP piece.

CU-Boulder is an i>clicker campus, and Dubson makes the case in the AP story that a simple, dedicated clicker device is preferable in most instances to more flexible systems based on smart phone apps.  Indeed, i>clicker devices have only six buttons–an on/off button and buttons labeled A, B, C, D, and E.  This is a very simple system, but, as inventor Tim Stelzer argued at the Louisville clicker conference back in 2008, multiple-choice questions with five answer choices work very well for the kinds of formative assessment and peer instruction many instructors use clickers to implement.

Gorski places me on the other side of a somewhat-artificial divide:

Derek Bruff, assistant director of Vanderbilt University’s Center for Teaching, said simple clickers are great at multiple choice questions. But he’s more excited about using smart phones, which allow students to ask questions of instructors, hold back-channel discussions with each other and respond in their own words.

Regular readers of this blog know that I’m definitely excited by the possibilities of using smart phones as “super-clickers” or to facilitate backchannel discussion in the classroom.  It’s true that I’m more excited by smart-phone systems than I am by simple clickers like i>clicker, but that’s largely because I’ve been involved in teaching with clicker with several years and I’m eager to leverage that experience to consider new kinds of technology-facilitated classroom dynamics.  (For one thoughtful perspective on those potential dynamics, consider Sean Seepersad’s recent post on moving away from clickers.  I hope to blog about Sean’s post soon!)

I’ve spent plenty of time thinking about the pedagogy of multiple-choice questions (while writing my book, blogging about clickers here, and giving talks on the subject around the country), and I think the multiple-choice format is often underrated.  I even have an article coming out (soon, I hope!) titled, “Multiple-Choice Questions You Wouldn’t Put on a Test: Promoting Deep Learning with Clickers.”  So I definitely get where Michael Dubson is coming from: Five-answer multiple-choice clicker questions are incredibly useful in all kinds of courses.

All this to say that one of the principles I attempted to uphold when writing my book was that everyone’s teaching context is different–different students, different disciplines, different institutions, different teaching styles and experiences.  I’m interested in helping instructors think more intentionally about their teaching choices, exploring the pros and cons of choices both traditional and innovative.  So while I may be more excited myself about smart phone systems, I always encourage instructors to select technologies and teaching practices that make the most sense in their particular teaching contexts.

I’m glad for clickers to receive the attention of the Associated Press.  The story has been all over Twitter today, and I hope it makes its way into print and online newspapers across the country.  And I’m glad that I could help Eric Gorski out as he was researching this story.  Eric also contributed to a short video piece to accompany his article, and he blogged about the story on the AP’s Facebook page.

Thoughts on the AP story?

Image: “The Nabla System (Forgotten Seed)” by Flickr user Syntopia

Although relatively few instructors in the humanities use clickers, if there’s one discipline in the humanities where clickers are starting to get some traction, it would be philosophy. I interviewed a couple of philosophy faculty members for my book (including Ron McClamrock of SUNY-Albany), and I’ve recently found a few online resources for using clickers in philosophy, listed below.

Why the particular interest in clickers among philosophy instructors? Perhaps it’s because some teach courses in logic, and these courses are often more like math courses (where clickers are more mainstream) than typical humanities courses. Perhaps it’s because some philosophy instructors teach relatively large classes–larger than is typical in English and language instruction, certainly–and clickers excel in large classes. However, I suspect the primary reason clickers have been adopted in philosophy is because philosophy instructors like to ask what I call “student perspective questions” in my book. These opinion and experience questions work beautifully in ethics courses, and I imagine they work well in other philosophy courses, as well.

On the Teaching Philosophy 101 site, John Immerwahr provides an introduction to teaching with clickers in philosophy courses.  He suggests a few uses of clickers that are of particular use in teaching philosophy.  For instance, he suggests asking students a few opinion questions at the beginning of a unit to surface their perspectives on the topic, helping them have a great stake in the discussion that follows.  He also suggests asking the same questions before and after a topic is discussed as a way to show students that “serious discussion of issues actually matters to how people think (a point which they sometimes don’t get initially).”

Immerwahr also stresses a point about clickers that is sometimes subtle: They can be used to generate “meta-conversations,” as he calls them.

Interestingly, the wording of the questions themselves often creates prompts for discussion. Student like to discuss why the class voted as it did, and people will sometimes make interesting distinctions (e.g., a student might say “If the question has said ‘can’ make a difference instead of ‘will’ make a difference, I would have voted differently,” which can then lead into another interesting discussion).

In my talks on teaching with clickers, I’ll often mention that the results display itself can generate useful discussion.  Asking students why the class voted as it did can often lead to productive discussions of assumptions students make about themselves and each other.

Immerwahr’s example also reminds me of another point I often make, that the wording on clicker questions need not be as precise as the wording on exam questions.  One reason is that if the question isn’t worded exactly right, an instructor can still make it work during the discussion of the question.  Another is that clicker questions can be modified and asked again based on student comments during discussion.  In Immerwahr’s example, for instance, the instructor could easily change “will” to “can” in the question and re-poll the students.

For an expanded version of Immerwahr’s introduction to clickers, read his Teaching Philosophy article, “Engaging the ‘Thumb Generation’ with Clickers.”  The article includes more discussion of the clicker uses mentioned above, as well as other uses, and features several sample questions.

And for even more resources on using clickers in philosophy instruction, visit the Peer Instruction in the Humanities project out of Monash University in Australia.  This site features a step-by-step guide to PI, advice on designing a PI lecture, a description of a sample PI lecture, examples of various types of clicker questions appropriate for this teaching context, and even a question bank organized by topic!  I’m very glad to know that there’s a humanities clicker question bank out there to complement existing question banks in the sciences.