Jeffrey R. Young's Chronicle article, "Reaching the Last Technology Holdouts at the Front of the Classroom," has apparently struck a nerve among professors, particularly those who are critical of educational technology. As I write this, the article has 59 comments on the Chronicle site, which is far more than most articles receive. Even the graph accompanying the article has received 13 comments!

Since clickers are mentioned in the article and in many of the comments, I thought I would weigh in here on the blog...

First, it's worth noting that Chris Dede, the Harvard University learning technologies professor interviewed for the article, doesn't make the argument that professors who don't use technology are shirking their duties. Several of those who left comments seem to think so, however. For example, here's comment #33 from Emily in NY:

"Dede does nothing in this article but set up a false dichotomy between professors committed to outdated, boring and irrelevant teaching methods and those eagerly embracing the modern technologies that contemporary students crave."

Here's the closest Dede comes to that argument, in the National Educational Technology Plan he helped draft for the US Department of Education in March:

"The challenge for our education system is to leverage the learning sciences and modern technology to create engaging, relevant, and personalized learning experiences for all learners that mirror students' daily lives and the reality of their futures."

Dede's arguments in the Chronicle argument are focused on motivating professors to tap into the latest research on learning and continue to improve their teaching practices over time. From the report he drafted, it's clear he thinks that technology can help with that, but he doesn't seem to be making the argument that professors who don't use technology are irresponsible, just those who stick with the same teaching methods you'd find in a classroom circa 1900. Sure, technology can be a big part of change, but many of the teaching innovations mentioned in the article (such as David Pace's work on enhancing history teaching) don't involve any technology.

Speaking of false dichotomies, however, here's one from comment #18 by user "tee_bee":

"What matters is that students learn--and a skilled teacher with a blackboard is still going to do a far better job than a bozo with some clickers and powerpoint slides."

True, a skilled teacher is going to do a better job than a bozo any day, regardless of technology. But comparing a skilled teacher to a bozo isn't really important here. Might technology (including clickers) help a skilled teacher be even more effective? Yes, that happens. And might technology help a relatively novice teacher become more effective? Yes, that happens, too. Those are the kinds of changes in teaching that are worth thinking about and encouraging, and I think that's a point that Chris Dede would agree with.

How might teaching with clickers help a good teacher be even more effective? Several comments on the Chronicle article were skeptical of clickers' potential for doing this. For example, here's what user "ikant" said in comment #21:

"I'm young, tech-savvy, and pretty unconvinced by this article. I can't speak for all fields, of course, but I'm pretty skeptical that good class discussions and quality writing in the humanities are particularly improved by clickers etc... the heart of what I do is in trying to educe questions, critical thought and excitement about books which students might previously have thought were utterly irrelevant to them, and (my evaluations indicate that) I do this very well with no particular technological bells and whistles in the classroom. Am I missing something?"

I'm glad that this instructor is capable of leading effective class discussions, foster critical thinking, and increase student motivation in the classroom. Let me clear: Doing so is entirely possible without clickers! However, not all instructors are as skilled as "ikant" appears to be and even for instructors like "ikant," it's possible that clickers would enhance an already productive classroom environment. Some examples from past blog posts:

• Case Study: Using clickers to promote critical thinking in text-based, political science courses
• Case Study: Using clickers to teach ethics in a history class
• My Thoughts: Using clickers to foster creativity and collaboration
• My Thoughts: Using clickers to teach critical thinking in the humanities

Here's a similar comment (#26 on the Chronicle site) from user "csgirl":

"The reason I don't use blogs and clickers is that they simply are not appropriate to the material I teach. Clickers in particular are useless to me - I care about the strategies my students are using to solve problems, not whether they can click the right answer in a quiz."

This is a common misconception about clickers, that they're just good for quizzing students basic conceptual understanding and recall. Here's another formulation of it, from user "chewy18" in comment #53:

"They might work well for understanding basic concepts or in preparation for recognition/recall examinations where the test question is a line long and the answer a word or two in length. What about those of us who teach upper division courses where we struggle with students who have not, until they reach senior status, even been exposed to the analytical reasoning process. Suddenly they discover that life is, after all, not a multiple choice test and developing an argument that could go either way, is a requirement. How does that appeal to the clicker technology?"

Sure, clickers work well for assessing basic conceptual understanding and factual recall, but they're useful for teaching at the higher levels of Bloom's Taxonomy, too. Here are some more examples from past blog posts that demonstrate this:

• Case Study: Using clickers to teach difficult material in upper-division physics
• Case Study: Using clickers to teach students about plagiarism in library instructional settings
• Case Study: Using clicker questions without single correct answers in mathematics
• Case Study: Using clickers to teach nursing students to make evidence-based decisions

And for "csgirl," here's a great collection of resources on using clickers and peer instruction in computer science from Daniel Zingaro.

Finally, you can imagine how this comment (#37) from user "fizmath" made me feel:

"The teacher/physician analogy is lousy. We have real data to show that new medical tech benefits patients. You can't say the same about blogs, videoconferencing and those stupid clickers."

(This is a response to Chris Dede's analogy that teachers who don't update their teaching methods over time are akin to physicians who don't update their medical practices over time.)

Want some research? Try these studies, all of which are well designed and support the claim that clickers used in appropriate ways enhance student learning:

• Stowell & Nelson (2007) - Clickers provided instructors with more accurate assessment of student learning during class than other response methods, including a show of hands.
• Yourstone, Kraye, & Albaum (2008) - The use of clickers for end-of-class quizzes improved student exam scores by four points over the use of pencil-and-paper quizzes discussed the next day in class, likely because of the immediate feedback clickers provided to students on their learning.
• Hoesktra (2008) - Clickers helped students be more attentive during class (since they know clicker questions could be asked at any time) and participate in more meaningful ways (both before votes are submitted and after results are displayed).
• Smith et al. (2009) - Students actually learned from each other when discussing clicker questions in pairs prior to voting. They don't "simply choose the answer most strongly supported by neighbors they perceive to be knowledgeable."
• Mayer et al. (2009) - Clickers made it easier for instructors to ask their students questions during class and for students to respond to those questions, leading to improved student learning through better class discussions.

My summary for those skeptical of using clickers in the classroom: Read the literature, find out how those in your discipline are using clickers effectively, and see (preferably by experimentation) if those methods might help you to enhance your teaching, regardless of how effective you are currently as a teacher. If a classroom response system doesn't help you do your job better, then don't use one. They're not for everyone. However, don't write clickers off without first investigating their potential. They're far more useful and versatile that you might think at first.

Image: "Innovation" by Flickr user thinkpublic, Creative Commons licensed

Reference: Webking, R., & Valenzuela, F. (2006). Using audience response systems to develop critical thinking. In Banks, David A. (Ed.), Audience Response Systems in Higher Education: Applications and Cases. Hershey, PA: Information Science Publishing.

Summary: Webking and Valenzuela describe ways they use classroom response systems in their political sciences courses at the University of Texas-El Paso to foster critical thinking through active participation and class discussions. After noting some commonly cited advantages of teaching with clickers—easier attendance and participation record-keeping, greater participation through anonymity and accountability, and the collection of data to inform agile teaching decisions—the authors provide several concrete examples of clicker questions they have found valuable for developing their students’ critical thinking skills.

The authors’ first example is a sequence of clicker questions that serve to guide students through a close reading of a few passages in the play Antigone. At one point in the play, Antigone makes a statement that seems to very clearly express her belief that obedience to the gods trumps obedience to the king. At another point, however, she makes a somewhat cryptic statement that calls this previous assertion into question. Webking and Valenzuela start with an understand-level question that asks students to clarify this second statement. They follow this with an application-level question asking students to identify a logical consequence of her cryptic statement, one which seems to run counter to her earlier statement about serving the gods. Their third question is an analysis-level one, and it asks students to reconcile the two seemingly contradictory statements by Antigone by identifying a hidden motivation of hers that makes her statements consistent.

Webking and Valenzuela also describe how they use a particularly challenging, analysis-level question about Plato’s Euthyphro. The question asks students to identify the central argument of a particular passage, one that deals with the relationship between justice and piousness. The question is one that Jean McGivney-Burelle would call a “horizontal question” since students answering the question are typically split evenly among three answer choices. Webking and Valenzuela note that one of the three popular responses can’t be supported by the text. Students who argue for this answer choice quickly realize that they were projecting their own perspectives on the text, not arguing from the text. This is a useful metacognitive moment for these students. The class discussion then focuses on the remaining two popular answer choices. Making sense of these two choices requires the students to grapple with categorical logic, the kind that is well-represented by Venn diagrams. Once the students have discussed their way to the correct answer, they realize the value of categorical logic in making sense of arguments like the ones Plato makes—another metacognitive moment.

The Plato example comes from one of the authors’ smaller, upper-level courses, and they assert that “it is in a smaller class that the [classroom response] system is at its best in encouraging discussion and precise argument.” They reach this conclusion, in part, because of the ability of their classroom response system to report to the instructor individual student responses to clicker questions as those responses are submitted. The authors use these individual, real-time results to guide their post-vote discussions, focusing on “groups which had difficulties in reaching consensus, students or groups which answered particularly quickly or particularly slowly, students who disagreed with their groups, students who changed their minds, and so on.” They argue that the ability to see individual, real-time results is important in leading effective post-vote discussions since it allows instructors to analyze “each student’s rational odyssey with each question.”

Also in the article are two examples of student perspective questions the authors use to motivate particular topics in their courses. In one example, they ask students to identify questions they aren’t likely to ask someone they’ve just met. Invariably, students identify the questions about religion and politics. The authors point out to students that one reasonable conclusion from this is that religion and politics are the least important things to know about when getting to know someone. This motivates students to want to learn why this social phenomenon exists.

Comments: This would be a great article to give a faculty member in political science or philosophy who’s interested in getting started teaching with clickers. Webking and Valenzuela provide a concrete, interesting example of a guided close reading of a text (Antigone) using clicker questions of increasing difficulty. This is a great model for instructors in the humanities and social sciences interested in helping their students develop critical thinking and close reading skills. I wish, however, that they had included some voting data in this example and had discussed how they use the results of these questions to guide discussions, as they did with their Plato example.

The Plato example is a great model of clicker use in text-based courses, too. One reason is that the approach Webking and Valenzuela use leads students to appreciate the nature of argument in their discipline. They write, “In time, and actually not very much time, students learn to care more about the strength of the argument than about having their initial position defended as right.” The authors present a useful list of options for leading these kinds of class discussions—focusing on groups that were conflicted, students who answered quickly or slowly, students who changed their minds, etc.

The authors assert that the quality of discussions they can foster depends on the availability to the instructor of real-time, individual voting data. Not all classroom response systems have this feature and, in my experience, instructors who have the option of looking at individual results as they come in don’t frequently take advantage of this option. I think that perhaps the availability of real-time, individual results isn’t as critical as Webking and Valenzuela assert. I’ll often have my students vote on a question individually, then discuss it in groups, then vote again. I’ll sometimes ask for a student who changed his or her mind from the first vote to the second vote to explain his or her reasoning. I can also see asking for a student who disagreed with his or her group to contribute to the post-vote discussion.  (That’s a nice idea, one that I’ll have to try soon!)

My approach, using the aggregate and not individual voting data, relies on students who fit certain profiles volunteering to share their perspectives with the class. Webking and Valenzuela’s approach doesn’t rely on volunteers, but it isn’t quite cold-calling, either, since they select students only after the students have had a chance to consider and respond to the clicker question. I’d like to call this “warm-calling” since the students have had a chance to warm up to the question and since the instructors aren’t calling on students without any knowledge of what those students might contribute to the discussion. I’m not familiar with many instructors who practice warm-calling.  If you do, I’d love to hear from you in the comments about your experiences doing so.

Image: “Coffin Sculpture of Antigone” by Flickr user Xuan Rosamanios / Creative Commons licensed

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Using Personal Response Systems (Clickers) in Liberal Arts Mathematics Courses to Support a Lecture Format,” Janet A. White, Millersville University of Pennsylvania [Slides]

Just like Jean McGivney-Burelle and Kimberly Burch, Janet White shared her experiences teaching with clickers in a “liberal arts” mathematics course taken by non-majors. Unlike Jean and Kimberly, who teach relatively small sections of this kind of course, Janet teaches in a large lecture hall with 75 students per section. Janet had used clickers in courses for pre-service math teachers in the past and found them useful, so when it was her turn to teach this larger course, she decided to use them again. A classroom response system was hardly the only technology Janet used in this course: She also had students complete online homework and quizzes and she annotated her PowerPoint lecture slides using an Interwrite Mobi.

Janet used clickers on a daily basis in her course, usually either to assess students’ prior knowledge or to assess their understanding of a topic taught during lecture.  Her questions came from a bank of multiple-choice questions provided by her textbook publisher.  She counted the clicker questions as part of her students’ participation grades, but in a low-stakes manner.  Given her use of the questions as well as the source of the questions, many were on the lower levels of Bloom’s taxonomy, aimed at recall and application of procedural knowledge.  She shared an example of a prior knowledge question that asked students to find the measure of an angle that complements a 36 degree angle.  A slightly harder question aimed at assessment of something taught during the course asked students to identify the cut edge in a given graph (or to assert that the graph had no cut edge).

Student survey results indicated that 85% of Janet’s students who used clickers regularly liked using them, and 71% said that using clickers helped them learn the material.  Students who used clickers regularly during the course ended up with higher grades in the course than students who didn’t, but, of course, that can’t necessarily be attributed to the use of the clickers.  (And since clicker questions were factored in the course grade, students who participated more frequently in clicker questions would almost certainly have higher grades in the course anyway.)

Student comments about the clickers were generally positive.  My favorite one was, “I liked getting the wrong answer anonymously.”  Other comments addressed the usual points that students like about clickers: They liked the interactivity, they liked discussing questions with classmates, they liked seeing where they stood relative to their peers, and they liked the feedback on their own learning the clicker questions provided.  The only significant negative aspect for the students was the cost, about $50 in Janet’s case.

Janet found that having students discuss clicker questions in small groups led to very engaged students, even in the large auditorium environment.  In the future, she plans to write more of her own questions, instead of relying on ones from the textbook’s question bank.  She hopes to write more difficult questions that will generate even more engaged discussion during class.  She’s also hoping to find ways to reduce the technology cost to the students, either by selecting a different vendor or facilitating the resale of clickers after each semester to students taking the course the next semester.

Also, Janet mentioned that the earth science faculty at Millersville are big users of clickers.  Earth science instructors looking for advice on using clickers might want to investigate!

Image: “Recursive Daisy” by Flickr user gadl / Creative Commons licensed

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Clickers in the Classroom,” Kimberly J. Burch, Indiana University of Pennsylvania [Slides]

Kimberly teaches a “Math 101″ survey course called “Foundations of Mathematics.”  Topics covered include set theory, graph theory, and counting methods (among others), and Kimberly shared several interesting clicker questions on each of these topics.  For example, here’s one of her questions from the unit on graph theory:

How many vertices are there in a tree with 19 edges?

1. 19
2. 18
3. 20
4. Not enough information given

Kimberly practices the “classic” peer instruction technique of having students vote individually first, then discuss the questions in small groups, then vote again.  She finds that students often converge to the correct answer on the second vote.

In the example above, her students were split between 18 and 20 on the first vote, but after the peer discussion time, most students went with the correct answer, 20.  I found this interesting because the “Not enough information given” seemed to be the obvious wrong answer to this question.  A graph with 19 edges might have any number of vertices, but a tree with 19 edges can only have 20 vertices.  Students who don’t realize that trees are graphs with very specific properties might be tempted to go for the “Not enough information given” option.

I suspect that Kimberly used this question after the students learned the relationship between the number of edges and number of vertices in a tree and that this question was meant to assess whether students remembered that relationship.  Some students likely remembered that one of these numbers was one more than the other but weren’t sure which one was higher.  That would account for the split vote between 18 and 20.  Had this question been asked as an exploratory question and not a review question, I’m betting the split would have been between 20 and “Not enough information given.”

Kimberly also mentioned that she uses her clicker system’s priority ranking questions to have her students decide what topics should be emphasized during exam review sessions.  Kimberly gives her students a list of 8-10 exam topics, and the students indicate the top three or four toughest topics in order.  Kimberly said that this helps her make good use of limited exam review time by focusing on the topics the students find the most difficult.

Kimberly also shared some data from a quasi-control group experiment she conducted.  She taught two sections of this survey course and alternated which topics she covered with clickers in the two sections.  For example, section A might cover topic 1 with clickers while section B covered topic 1 without.  Then for topic 2, section B used clickers and section A didn’t.  She then compared test scores for the two sections by topic.  For some topics, students using clickers performed better on exams but for other topics, the students not using clickers performed better.  And for other topics, there was no difference.  The data was generally favorable to using clickers, but the “quasi” part of this quasi-control group experiment made it difficult to draw firm conclusions.

Image: “Point Marian Bridge” by Flickr user timmenzies / Creative Commons licensed

File this under “Better Late Than Never.”  Back in January 2010, I coordinated (with Kelly Cline and Kien Lim) a contributed paper session on teaching with clickers at the Joint Mathematics Meetings in San Francisco.  Shortly after the conference, I blogged about some clickers talks that didn’t fall in our session, but I never got around to blogging about the talks in our session!  Five months later, I’m finally getting around to sharing my notes from those talks…

“The Evolution of Classroom Voting in Contemporary Mathematics,” Jean McGivney-Burelle, University of Hartford [PowerPoint Slides]

Jean teaches a “math for the liberal arts” course called Contemporary Mathematics taken by music and arts majors among others.  She finds her students come to the course with relatively little interest or self-reported ability in mathematics, so it’s a tough crowd to teach.  A few years ago, she started teaching with clickers in order to appeal to what she calls the “thumb generation”–students used to spending a lot of time sending text messages.

Jean interspersed some clicker questions throughout her lectures and encouraged students to discuss them in small groups before voting.  She and the students liked this, but she found that most of her questions were answered correctly by most of her students  and that the small group discussions didn’t involve much debate among students.  The next year, Jean decided to ask tougher questions.  She calls them QEDs–Questions to Encourage Discussion.  She aimed for the analysis, synthesis, and evaluation levels of Bloom’s Taxonomy.

For example, in Jean’s first year using clickers she gave her students a preference schedule–a list of how each voter in an electorate (only four of them to keep things simple) ranked all of the candidates.  She then asked her students to determine which candidate would win the election using the instant run-off voting scheme.  This is a straight-forward application of a particular algorithm.

The next year, Jean asked another question in which she shared a preference schedule with her students and asked them to analyze it.  However, this time, she asked the question at the beginning of the unit on voting schemes and asked her students to indicate which of the candidates had the best case for winning the election.  There’s no single correct answer to this question since winner of an election (well, one involving at least three candidates) depends on what scheme you use to count the votes.  This is a great example of a one-best-answer question (since students are asked to select the one answer they think is best among multiple reasonable answers) used to create a time for telling (since it’s used to make the point that which voting scheme you use matters).

Jean found that these more challenging and ambiguous questions generated longer and more engaged small group discussions as well as more “horizontal” bar graphs–ones indicating significant disagreement among the students.  Looking ahead, she plans to build on this success by writing questions designed to develop mathematical habits of mind–an important goal of this course.  For example, here’s a sample question she shared aimed at pointing students towards the notions of proof and counterexamples:

Suppose there is a majority winner in an election. Will all of the voting methods we have studied thus far always pick that winner?  Yes or no?  If you answer yes, prepare to defend your answer. If you answer no, have a counterexample ready.

I really like this question.  It has a degree of ambiguity that students often find disconcerting, but it also reminds students of how they’ll need to defend their answers, which should help put their minds at ease.  As Jean noted in her talk, in a course like this one, it’s more important students develop mathematical habits of minds (like the ones surfaced by this question) than learn particular math content areas.  I hope this kind of question helps with this objective.

Stay tuned to the blog for more notes on these talks over the coming days…

Image: “Deep Down Inside, We All Love Math” by Flickr user Network Osaka / Creative Commons licensed

In spite of including experiences from not one, but two language instructors in my book, I still haven’t found any studies exploring the use of clickers in language classrooms for my bibliography.  And, if you check out the column to the right of this post, you’ll see the various disciplines I’ve covered here, and language instruction is not well represented.  (This very post will be only the second in that category.)  Since I’m pretty sure clickers have incredible potential in language instruction, you can imagine how glad I was to see a recent blog post about clickers in a Spanish class at Georgetown University!

The post is a report from Ellen Johnson, a PhD student in applied linguistics, who teaches and coordinates Spanish courses at Georgetown.  After hearing about clickers at a workshop hosted by Georgetown’s Center for New Designs in Learning and Scholarship (CNDLS), Johnson experimented with clickers in her language classroom.  Not only did she experiment with clickers, she collected some data useful in helping her judge their effectiveness.  Here’s what she did:

In a nutshell, 58 students enrolled in Beginning Spanish courses participated in the study on ser and estar. They were introduced to their uses in context, practiced answering questions using clickers with around 20 slides while viewing their performance in relation to their peers, and then completed posttests and reflection questionnaires.

Johnson also had colleagues observe the classes, and the feedback from both students and instructors about the use of clickers was very positive.  The students were particularly enthusiastic about getting immediate feedback on their learning, and Johnson’s fellow instructors thought the clickers had potential for helping them target their feedback to their students “in a more coherent manner.”

Students raised a couple of concerns in their feedback, however.  They thought that the clicker questions made it difficult for them to take notes during class.  Making clicker questions available to students after class is something I’ve mentioned here before, and there’s a little evidence that doing so is, in fact, very important since it allows students to review clicker questions later.  Knowing that clicker questions will be available online after class also frees students from having to take as many notes during class, which is likely to help them spend more time actually thinking during class.

Johnson’s students also noted that clicker questions don’t allow students to practice their speaking skills in a language class.  That’s a good point, but given the experiences of the language instructors I interviewed for my book, it would seem that clicker questions work very well for listening, reading, and writing skill development.

The main concerns raised by language instructors in Johnson’s study were logistical ones.  They worried that the technology would be difficult to start using or might fail during class.  I don’t know what system they use at Georgetown, but I know there are easy-to-use and reliable systems out there.  Also, Georgetown doesn’t seem to have a full-scale clicker implementation, one where students could be expected to purchase clickers at their bookstore, as is the case at many US colleges and universities.  That creates a logistical barrier, as well, since clickers would have to be distributed and collected each class session.

Thanks to Ellen Johnson for sharing her experiences with clickers.  I would be interested to hear more uses of clickers in language courses.  What kinds of questions and activities work well with clickers in those settings?  And why do you think that clickers aren’t more widely used in language instruction?

(I should also note that Ellen Johnson’s post appeared on a group blog from a group of instructors at Georgetown exploring the use of clickers this spring.  Take a look at previous blog posts for more interesting discussion of teaching with clickers.  This “community of practice” is another example of the value of fostering discussions about teaching and learning across the disciplines.)

Image: “Pink AC Bienvenidos” by Flickr user lopolis / Creative Commons licensed

Maybe this is obvious to others, but I hadn’t thought of this particular use of numeric-response clicker questions, shared with me by a humanities professor recently: In a class that deals with history, ask students to identify the year in which a particular event happened using a numeric-response clicker question.

This question type is typically used in math and science classes to have students respond with their answers to open-ended computational questions, but it can just as easily be used in a humanities class to have students respond with dates (e.g. 1776, 2010).  Sure, one could ask students to respond to a multiple-choice date question, but the free-response format might surface some wrong answers you wouldn’t predict.

This kind of question isn’t limited to events, of course.  You could also ask students to identify the year a piece of literature was written or an artwork was created.  This type of question need not be a factual recall question, either.  You could present to students a piece of art, for instance, they haven’t likely seen before and ask them to analyze the artwork and estimate when it was created.

Some classroom response systems allow you to set a range for the correct answer to a numeric-response question.  With that feature, you could give students a little wiggle room in their answers (“To within 5 years, in what year did X occur?”) or have them respond to the nearest decade.

(By the way, I’ve just signed up for the twitterfeed service, so a tweet about this post should automatically appear in my Twitter stream in the next hour.  Fingers crossed!)

Clyde Herreid of the University of Buffalo’s National Center for Case Study Teaching in Science has received a $500,00 grant from the National Science Foundation to study the use of clicker questions in case studies used in biology classes, with a particular focus on students’ emotional engagement with science.  (The University of Buffalo seems to be a happening place for clickers.  I mentioned their School of Dentistry’s use of classroom response systems on Monday.)  I haven’t blogged about the use of clickers in case studies yet, but I mentioned the practice in my book, citing Herreid’s paper on the topic (Herreid, 2006) as well as Peggy Brickman’s clicker-enhanced case study on DNA fingerprinting.

I’m glad to see an investigation of this pedagogy receive such a sizable grant, particularly given its emphasis on the affective domain.  However, the press release describes students using clickers like a game show buzzer, which is a bit misleading, now that I think about it.  On Jeopardy, for instance, only the fastest responder is allowed to answer, but when using clickers all students have a chance to weigh in.

Meanwhile, the Faculty Technology Center at Louisiana State University recently hosted a presentation on teaching with clickers by biological sciences professor Steve Pomarico.  Pomarico noted that before using clickers in his 250-student course, attendance would vary from 60% to 30% on any given day.  Now that he uses clickers and awards participation points for students responding to his clicker questions, attendance is never below 65%.  He notes that merely using clickers to take attendance is a poor choice, however.  Asking questions that let students test their understanding and provide instructors with useful feedback on student learning is a better use of the technology.

• Herreid, C. F. (2006). “Clicker” cases: Introducing case study teaching into large classrooms. Journal of College Science Teaching, 36(2), 43-4.

Reference: Mayer, R. E., Stull, A., DeLeeuw, K., Almeroth, K., Bimber, B., Chun, D., Bulger, M., Campbell, J., Knight, A., & Zhang, H. (2009). Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes. Contemporary Educational Psychology, 34(1), 51-57.

Summary: In this article, Richard Mayer and his collaborators, nine in all, describe the results of an experiment comparing the use of clickers to non-clicker alternatives.  A large enrollment educational psychology course, taken mostly by junior and senior psychology majors, was taught one year in a “traditional” method, without the use of in-class questioning or clickers.  The next year, the same course (with very similar students) was taught using in-class questioning facilitated by clickers.  In the third year, in-class questions were used, but instead of having students respond using clickers, students wrote their responses down on paper quizzes, passed those papers in to the instructor, then indicated their responses to the questions with a show of hands.

Differences among the three courses were kept to a minimum.  The same instructor taught all three courses, and the lecture materials were repeated, as well, with the exception of the additional questions added to the clicker and no-clicker groups.  Reading assignments and exam questions were identical, as well.  Having the students respond to questions in writing in the no-clicker class meant that their initial responses to a question were largely made independently of their peers, just as in the clicker class.  (The answers they signified during the shows of hands were, on the other hand, not necessarily independent.)

There were some differences, however.  The in-class questions in the clicker and no-clicker groups were graded (1 point for answering incorrectly, 2 points for answering correctly), which meant grade incentives were a possible motivator in those two groups.  There was no parallel grade incentive in the “control” group.  Also, in the no-clicker class, the paper quizzes were typically administered at the end of a class session for logistic reasons (distributing and collecting the quizzes took time), whereas in the clicker class, questions were asked at various points during class.

The authors’ findings were certainly interesting.  When they compared midterm and exam performance across the three courses, they found that the clicker class performed significantly better on the exams, averaging 75.1 points out of a possible 90.  The no-clicker class averaged 72.3, and the control group averaged 72.2.  (The difference here was statistically significant with p=.003.)  So the clicker class ended up with an average grade in the course 1/3 of a letter grade higher than the other two classes, a B instead of a B-.  And the paper quizzes plus hand-raising had “no discernible difference on student learning outcomes.”

Even more interesting was the following.  The clicker class performed almost identically to the other two classes on exam questions that were similar to questions asked (via clickers or paper quizzes) in class.  However, on exam questions that were dissimilar to in-class questions, the clicker class performed significantly better (50.2 vs. 47.9 and 48.2, p=.002).

The authors conclude from these data that the logistical difficulty of implementing the paper quizzes (distributing the quizzes, collecting the quizzes, and so on) interfered with any benefit gained from questioning students in this manner.  They also note that doing the questioning at the end of a class session might reduce the impact of the questioning on the students’ learning.  The use of clickers made questioning students “seamless” for the instructor and allowed the instructor to test and provide feedback to students closer in time to the initial learning experience.

The authors also note that some of the components of active learning–”(a) paying more attention to the lecture in anticipation of having to answer questions, (b) mentally organizing and integrating learned knowledge in order to answer questions, and (c) developing metacognitive skills for gauging how well they understood the lecture material”–might serve to explain why the clicker class outperformed the other two classes on exam questions dissimilar to in-class questions.

Comments: These results are fairly persuasive.  The authors did a good job of controlling for potentially confounding variables, and the use of three groups–clickers, no clickers, and control–meant that they could isolate the effect of the clickers from the effect of having students respond to questions during class.  Their conclusion–that clickers make questioning easier for both instructors and students and so allow questioning to have more impact–makes sense to me.

Another possible explanation for the higher learning gains in the clicker class is that the students in the clicker class were able to see the display of results of the clicker questions, whereas the students in the no-clicker class had to rely on a show of hands to see where their peers stood on a question.  Since it’s been shown that the hand-raising method leads to inaccurate representations of student understanding (see, for instance, Stowell and Nelson, 2007), it could be that the more accurate reporting of student responses to questions allowed by the classroom response system led to students taking the process more seriously in one way or another.

It’s also worth noting that after questions were asked and answered by students in both the clicker and no-clicker class, not too much happened.  The instructor would state the correct answer, have a student volunteer share reasons for the correct answer, then share his own reasons for the correct answer.  There wasn’t much in the way of agile teaching (doing something different in class in response to the results of a clicker question) or peer instruction (having students discuss questions with each other prior to answering).  There wasn’t much discussion of incorrect answers, apparently.  All of these processes have potential pedagogical benefits.  Had they been employed, the different in learning outcomes between the clicker class and the other two classes might have been even greater.

I should also point out that the article doesn’t clearly state the instructor’s experience teaching with clickers, although it seems a safe bet that the instructor was new to using clickers.  Instructor experience is another important variable, as is the nature and difficulty of the questions used.  A few sample questions were included in the article, but it would have been helpful to know how difficult the students found these questions.  Did most students answer them correctly?  Did a lot of students answer them incorrectly?

There were a lot of mentions on Twitter the other day of the recent Campus Technology article “Fostering Classroom Interaction, Minus the Clickers.” The article describes a Web application called LectureTools developed by Perry Samson, a professor of atmospheric, ocean, and space sciences at the University of Michigan.  In a nutshell, LectureTools is a suite of tools designed to support active learning, including clicker-like survey and quiz tools, PowerPoint sharing and annotation tools, a backchannel (student-to-instructor) tool, and a podcast tool.  These tools are useful outside of class, but they are even more useful during class, provided one’s students all have laptops and WiFi connections.

It’s clear that Samson has put a lot of time and effort in developing this suite of tools and that he’s had success in using it to enhance his classroom dynamic.  The functions accomplished by these tools can be accomplished in other ways, of course, but LectureTools integrates these functions, which likely has some value for students and instructors.

Here’s what caught my attention in the Campus Technology article:

“Where on this weather map do you expect it’s going to rain today?” Dr. Perry Samson asks the 200 students in his introductory class on extreme weather. Almost instantly, dots begin to appear on the displayed map, as students indicate their answers through their wireless laptops. In moments, a clear pattern emerges on the classroom display as Samson continues the lecture.

That’s a kind of free-response question that I haven’t seen before!  I really like the idea of sending an image to students’ laptop screens (or smart phone screens, for that matter), letting them mark it up somehow, and then displaying an aggregation of students’ markups on the big screen for the whole class to see.  That would seem to set the stage for a productive class discussion, one informed by patterns in the student responses.

Samson calls this kind of question an “image-based question.” I might call it a “placemark question,” after the little virtual push-pins one can use to mark locations in Google Earth.  This kind of question would be particularly useful, I think, in earth, oceanic, and atmospheric science courses, although any course in which students analyze images or graphics might find this question type useful.

LectureTools is another example of a classroom response system in which the response devices aren’t clickers.  These more general classroom response systems have a lot of potential, in part because of the greater flexibility in the nature of student responses.  See my last post for a different (and hypothetical) classroom response system with potential use in literature courses.

One more thing: This probably goes without saying, but I’m not opposed to the use of clicker-based classroom response systems!  Multiple-choice questions can be incredibly useful, sometimes in ways that surprise those who consider them limiting.  Also, many clicker systems are very simple and easy to use, which is a plus for many instructors.  I don’t know how easy or hard LectureTools is to use, but I know some instructors prefer simple, single-use systems to complex, integrated systems.  You’ll need to consider what kinds of technologies work best for you.