Reference: Freeman, M., Blayney, P., & Ginns, P. (2006). Anonymity and in class learning: The case for electronic response systems. Australasian Journal of Educational Technology, 22(4), 568-580.

Summary: In an effort to investigate the importance to students of the anonymity afforded by classroom response systems, Freeman, Blayney, and Ginns surveyed the 139 students in an introductory management accounting course at the end of the term. Notably, the majority of students in this course were female (73%), 20-22 years old (93%), with non-English speaking backgrounds (82%).

Each three-hour class session in this course featured a number of “formative, mainly rules based, multiple choice questions.”  Students responded to these questions via clickers or hand-raising in alternating class sessions and were encouraged, but not required, to discuss their answers with their peers before responding.  The instructor would announce the correct answer to each question immediately after the distribution of responses was shared with the class and then practice “agile teaching” by using the results of the question to guide subsequent lecture and discussion.  This alternating system of response methods ensured that students were given opportunities to experience a response method that allowed students to remain anonymous (clickers) and one that did not (hand-raising).

Sixty-eight percent of students agreed or strongly agreed with the statement “I preferred answering in-class questions when my answers were anonymous to the instructor.”  Almost as many students (62%) preferred answering questions when their answers were anonymous to their peers.  About the same number of students (63%) agreed or strongly agreed that “anonymity was more important with in-class questions when [they] were uncertain about the answer.”

Additionally, students were asked to rank in order of preference four potential response methods-clickers, hand-raising, volunteering to respond when they knew the answer, and “cold-calling.”  Average student rankings of these four methods indicated a preference for anonymity, as the methods were ranked (on average) in the same order as I just listed them.

The authors conclude from these survey data that students value the anonymity (peer-to-peer anonymity and student-to-instructor anonymity) enabled by a classroom response system.  Since other response methods, such as hand-raising and response cards, are less anonymous, this argues for the use of clickers.

The authors make a couple of additional points about the potential interplay of culture and use of clickers.  One is that peer instruction was not as highly valued by these students as other aspects of learning with clickers, according to survey results.  It’s possible that the cultural diversity of the students (as measured by proxy by the percentage of students with non-English-speaking backgrounds) made peer instruction less useful for these students than for more homogeneous students surveyed in other studies who found peer instruction more useful.

The other is that demographic variables, including native language, had no significant impact on the student responses to survey questions, particularly the questions about the importance of anonymity.  “Using first language as a rough proxy for culture, and in particular openness to criticism, these results might be seen to contradict Banks (2003) who suggested that cultural background could impact preference for ERS usage.”  The authors suggest further study of the role of culture in students’ valuing of anonymity.

Comments: For a relatively simple survey-based study, this article raises some interesting questions.  For instance, given that the instructor of this course practiced agile teaching, altering his instruction based on the distribution of student responses to in-class questions, I would think that response methods that generated more honest responses from students would lead to more useful agile teaching.  That is to say, if students were less likely to indicate confusion about a question when the hand-raising method was used (because their answers would not be anonymous to their peers), then the instructor might overestimate his students’ comprehension of the topic at hand when relying on the response distribution and subsequently spend less time on the topic than might actually be warranted.  If true, then students who prefer anonymous response methods and who have instructors who practice agile teaching might benefit more from an anonymous response method than students whose instructors do no practice agile teaching.  Since I have yet to read a study that compared use of clicker questions with and without agile teaching, it’s unclear at this point how important this issue might be.

The questions raised about the interface of culture, participation, criticism, and anonymity are also very interesting ones.  They are ones that haven’t been on my radar until recently, when I heard a presentation by Parvanak “Pary” Fassihi, who described her use of clickers to engage students in English-as-a-Second-Language (ESL) classes at Boston University and the Boston campus of the Showa Women’s University (of Japan).  She shared that some of her students come from cultures where it is seen as impolite to disagree with others publicly.  These students are often very hesitant to engage in the small-group and classwide discussion of clicker questions that Pary tries to generate in her courses.  (I’ll note here that I interviewed Pary for my book.  She had a lot of interesting things to say about the role of clickers in her language instruction courses, but this observation about cultural views on public disagreement didn’t come up in our interview.)

Pary’s observation is consistent with the finding by Freeman, Blayney, and Ginns that their students, many of whom came from diverse cultures given that few of them spoke English as a first language, did not value peer instruction as highly as might be expected.  I imagine it is also consistent with Banks’ suggestion, mentioned in the article, that “cultural background could impact preference” for clicker usage.  I haven’t read the Banks article, but I hope to do so soon and review it here on the blog.

One criticism I would have of the article at hand is that comparing the clicker and hand-raising response methods doesn’t quite isolate the effect of anonymity since clickers offer a number of advantages over hand-raising that might be at play here.  Given the nature of the survey questions used, this isn’t a significant issue, since student perception of anonymity was the focus, not a control-group-study comparison of two difference response methods.  However, it is possible that the students’ preference for clickers over hand-raising over hearing from student volunteers could be an indication not of their preference for anonymity but of their preference for response methods that encourage more complete and/or more independent participation and/or greater accountability for participation.  Thus, I would argue that those data in particular don’t necessarily imply that students value anonymity when responding to in-class questions.

(Cold-calling also leads to more complete and independent participation and greater accountability for participation since students have to stay on their toes in case they are called upon.  However, I’m comfortable arguing that the stress generated by this method would outweigh any perceived benefits regarding participation or accountability.)

A better comparison might be between having students respond via clickers with and without a display of individual participant responses.  (Many classroom response systems have the ability to display on screen not only a histogram showing the distribution of responses but also a list of individual students and their particular responses.)  That would better isolate the anonymity factor from other advantages of clickers over alternate response methods.  It’s not a perfect solution, in part because students in a large class aren’t likely to know each other’s names and so might not be bothered by having their names displayed next to their responses and in part because this is a feature of classroom response systems that is infrequently used by instructors so its use in this context might seem too artificial.  Also, this plan would not isolate peer-to-peer anonymity from student-to-instructor anonymity.  However, I think it would yield more relevant data than comparing clicker use with hand-raising.

Reference: Mayer, R. E., Stull, A., DeLeeuw, K., Almeroth, K., Bimber, B., Chun, D., Bulger, M., Campbell, J., Knight, A., & Zhang, H. (2009). Clickers in college classrooms: Fostering learning with questioning methods in large lecture classes. Contemporary Educational Psychology, 34(1), 51-57.

Summary: In this article, Richard Mayer and his collaborators, nine in all, describe the results of an experiment comparing the use of clickers to non-clicker alternatives.  A large enrollment educational psychology course, taken mostly by junior and senior psychology majors, was taught one year in a “traditional” method, without the use of in-class questioning or clickers.  The next year, the same course (with very similar students) was taught using in-class questioning facilitated by clickers.  In the third year, in-class questions were used, but instead of having students respond using clickers, students wrote their responses down on paper quizzes, passed those papers in to the instructor, then indicated their responses to the questions with a show of hands.

Differences among the three courses were kept to a minimum.  The same instructor taught all three courses, and the lecture materials were repeated, as well, with the exception of the additional questions added to the clicker and no-clicker groups.  Reading assignments and exam questions were identical, as well.  Having the students respond to questions in writing in the no-clicker class meant that their initial responses to a question were largely made independently of their peers, just as in the clicker class.  (The answers they signified during the shows of hands were, on the other hand, not necessarily independent.)

There were some differences, however.  The in-class questions in the clicker and no-clicker groups were graded (1 point for answering incorrectly, 2 points for answering correctly), which meant grade incentives were a possible motivator in those two groups.  There was no parallel grade incentive in the “control” group.  Also, in the no-clicker class, the paper quizzes were typically administered at the end of a class session for logistic reasons (distributing and collecting the quizzes took time), whereas in the clicker class, questions were asked at various points during class.

The authors’ findings were certainly interesting.  When they compared midterm and exam performance across the three courses, they found that the clicker class performed significantly better on the exams, averaging 75.1 points out of a possible 90.  The no-clicker class averaged 72.3, and the control group averaged 72.2.  (The difference here was statistically significant with p=.003.)  So the clicker class ended up with an average grade in the course 1/3 of a letter grade higher than the other two classes, a B instead of a B-.  And the paper quizzes plus hand-raising had “no discernible difference on student learning outcomes.”

Even more interesting was the following.  The clicker class performed almost identically to the other two classes on exam questions that were similar to questions asked (via clickers or paper quizzes) in class.  However, on exam questions that were dissimilar to in-class questions, the clicker class performed significantly better (50.2 vs. 47.9 and 48.2, p=.002).

The authors conclude from these data that the logistical difficulty of implementing the paper quizzes (distributing the quizzes, collecting the quizzes, and so on) interfered with any benefit gained from questioning students in this manner.  They also note that doing the questioning at the end of a class session might reduce the impact of the questioning on the students’ learning.  The use of clickers made questioning students “seamless” for the instructor and allowed the instructor to test and provide feedback to students closer in time to the initial learning experience.

The authors also note that some of the components of active learning–”(a) paying more attention to the lecture in anticipation of having to answer questions, (b) mentally organizing and integrating learned knowledge in order to answer questions, and (c) developing metacognitive skills for gauging how well they understood the lecture material”–might serve to explain why the clicker class outperformed the other two classes on exam questions dissimilar to in-class questions.

Comments: These results are fairly persuasive.  The authors did a good job of controlling for potentially confounding variables, and the use of three groups–clickers, no clickers, and control–meant that they could isolate the effect of the clickers from the effect of having students respond to questions during class.  Their conclusion–that clickers make questioning easier for both instructors and students and so allow questioning to have more impact–makes sense to me.

Another possible explanation for the higher learning gains in the clicker class is that the students in the clicker class were able to see the display of results of the clicker questions, whereas the students in the no-clicker class had to rely on a show of hands to see where their peers stood on a question.  Since it’s been shown that the hand-raising method leads to inaccurate representations of student understanding (see, for instance, Stowell and Nelson, 2007), it could be that the more accurate reporting of student responses to questions allowed by the classroom response system led to students taking the process more seriously in one way or another.

It’s also worth noting that after questions were asked and answered by students in both the clicker and no-clicker class, not too much happened.  The instructor would state the correct answer, have a student volunteer share reasons for the correct answer, then share his own reasons for the correct answer.  There wasn’t much in the way of agile teaching (doing something different in class in response to the results of a clicker question) or peer instruction (having students discuss questions with each other prior to answering).  There wasn’t much discussion of incorrect answers, apparently.  All of these processes have potential pedagogical benefits.  Had they been employed, the different in learning outcomes between the clicker class and the other two classes might have been even greater.

I should also point out that the article doesn’t clearly state the instructor’s experience teaching with clickers, although it seems a safe bet that the instructor was new to using clickers.  Instructor experience is another important variable, as is the nature and difficulty of the questions used.  A few sample questions were included in the article, but it would have been helpful to know how difficult the students found these questions.  Did most students answer them correctly?  Did a lot of students answer them incorrectly?

This afternoon’s “All Tech Considered” segment on NPR featured a story about teaching with clickers.  If you follow that link and click the “Listen Now” button, you can hear the entire piece, including a few comments by NPR technology correspondent Omar Gallaga.  Omar shares a few more resources on the “All Tech Considered” blog, including a link to my recent interview with Inside Higher Ed.

I thought I would comment on a couple of points raised in the piece.  I liked this quote from Conor McLennan of Cleveland State University:

“I thought people would be less likely to speak up because now they can respond anonymously, they don’t have to open their mouth,” McLennan says. “But it turns out, it’s the other way around.

“They know everybody is answering the questions, everyone is in the same boat, so they are more likely to speak up and interact.”

This is one of the reasons clickers can be great tools to encourage discussion.

Omar Gallaga made the point that instructors using clickers should be careful that students aren’t giving them the false impression of understanding just because they clicked the correct answer.  I agree with that–following a clicker question with some class discussion is usually a good idea to help verify the results of a clicker question.

However, I would argue that other methods of having students respond during class (hand-raising, flashcards) are even more likely to give instructors the false impression that their students are following along since these methods can make it possible for students to see how their peers respond before responding themselves.  Furthermore, just asking one’s students “Any questions?” every now and then can be misleading, too, since students are often hesitant to admit in front of their peers that they have questions.  Clickers are a great tool for overcoming these difficulties.

NPR host Melissa Block made the point in the piece that multiple-choice questions might not be the best types of questions for helping students develop critical thinking skills.  As Peter Pappas pointed out in the comments on the NPR site, it’s quite possible to ask multiple-choice questions that target critical thinking skills.  I’ve mentioned a few possibilities here on the blog, and I have a whole section in my book on critical thinking clicker questions.

These points notwithstanding, I’m glad to see clickers getting some more press coverage.  By the way, Turning Technologies, make of a popular classroom response system, was mentioned in this story.  They were featured in another NPR story the other week on economic revitalization in Youngstown, Ohio, their home base.

Reference: Lasry, N. (2008). Clickers or flashcards: Is there really a difference? The Physics Teacher 46(4), 242-244.

Summary: Lasry reports the results of a study contrasting the use of clickers and flashcards in facilitating peer instruction in an introductory physics course.  Two sections of the course were taught in the same semester by Lasry.  In one section, students responded to multiple-choice, conceptual understanding questions using clickers; in the other they responded using flashcards.  In both sections, student responses to the questions were used to determine what followed the question–further explanations of the topic at hand by the instructor if most students missed the question, moving on to the next topic if most students answered correctly, or peer instruction otherwise.

Lasry administered the Force Concept Inventory to both sets of students at the start and end of the semester as an assessment of the students’ conceptual understanding.  The normalized gain, (post%-pre%/100-pre%), for the clickers section was 0.486, and for the flashcard section it was 0.520, not a statistically significant difference in this case.

Lasry’s conclusion is that “using peer instruction with clickers does not provide any significant learning advantage over low-tech flashcards.”  He notes that clickers might provide other advantages, such as enabling instructors to analyze student response data for the purpose of improving in-class questions over time and interesting other instructors in experimenting with peer instruction.

Comments: Lasry’s data are certainly interesting and provide some evidence that peer instruction works as well with flashcards as with clickers.  However, he describes the “contributions of clickers” as being “more on the teaching side than on the learning side of the educational equation.”  I find this separation of teaching and learning a little artificial.  The effects on student learning that any instructional technology has depend on how the technology is implemented.  There are a couple of ways of implementing clickers that have the potential to positively impact student learning that don’t appear to be addressed in this study.  These factors might explain the lack of difference in learning gains between the two sections.

For example, since clickers allow an instructor to track individual student responses, they can be used to hold students more accountable for their responses than they would be using flashcards, which has the potential to increase student motivation to participate and engage with questions asked during class.  It’s unlikely that student responses in the clicker section in this study were factored into student grades since tracking individual student responses in the flashcard section would have been impractical and Lasry apparently tried to keep as many aspects of each section constant as he could.  If that’s the case, then students in each section would have been similarly motivated to participate, which might explain the lack of difference in learning gains.  Had student responses to clicker questions been included in student grades in the clicker section, students might have performed better on end-of-semester assessments.

One of the points that Tim Stelzer made in his clicker conference keynote last November was that student participation tended to decrease over time when flashcards were used at the University of Illinois.  I would be interested in finding out if there was any difference in participation in the two class sections in Lasry’s study.  If there was not, then there might have been other factors, such as instructor experience or instructor-student rapport, that kept participation high in the flashcard section and offering another explanation why the clicker section didn’t exhibit greater learning gains.

Another implementation choice that has a potential effect on student learning is “agile teaching,” that is, using response data from clicker or flashcard questions during class to make teaching decisions.  In Lasry’s study, response data were used to determine when to engage students in peer instruction.  Such choices are likely most effective when based on accurate assessments of student learning.  As Stowell and Nelson (2007) showed, the flashcard method can lead to instructors overestimating their students’ comprehension since the method makes it possible for students to see other students’ responses as they select their own responses.  Clickers tend to provide more accurate feedback on student learning since they promote independent answering by students.  It’s possible that in the Lasry study, the flashcard method provided accurate enough assessments for the teaching choices that were made.  Other kinds of agile teaching choices might have benefited from the more accurate data provided by clickers.  The impact of clickers on agile teaching is an issue that hasn’t been studied well to date to my knowledge.

Finally, another way in which clickers might provide benefits over flashcard methods is that clickers make it easy for students to see the distribution of responses to a question.  Flashcards provide this distribution (in rough form) only to the instructor.  Seeing the distribution of responses has a potentially motivating effect on students, particularly when students find out that most of their peers answered a question incorrectly.  It’s unclear from the article the extent to which clicker or flashcard questions were used to generate “times for telling” in this fashion.  It’s possible that in classes where these kinds of questions are asked more regularly, clickers have a bigger impact on student learning because of the easy display of results to the class.

Reference: Bunce, D. M., VandenPlas, J., & Havanki, K. (2006). Comparing the effectiveness on student achievement of a student response system versus online WebCT quizzes. Journal of Chemical Education, 83(3), 488-493.

Summary: In the context of a 41-student chemistry course for nursing students, after certain topics were introduced during lecture, students were asked to respond to questions about those topics delivered via a classroom response system (CRS).  (In this case, the system used software running on wireless-enabled laptops that were loaned to students for this purpose.)  After class, students were asked to complete online quizzes on class topics prior to the next class.  The impact of these activities on student learning was assessed through performance on instructor-written hourly exams and a standardized final exam provided by the American Chemical Society.

Since some questions on the hourly and final exams featured topics covered in the CRS questions, some in the out-of-class quizzes, some in both, and some in neither, the authors were able to assess the impact on student learning of the two review mechanisms.  Students did better on hourly exam questions tied to the online quizzes than other questions, indicating that the online quizzes did the best job of helping students do well on these exams.  Results from the final exam indicate that neither the in-class CRS questions nor the out-of-class online quizzes helped students do better on the final, however.

The authors report a couple of problems that undercut these results.  One was that as students responded to the in-class CRS questions, they were able to see the bar chart showing the results as they came in.  This meant that, as one student put it, “People who don’t know the answer simply wait for the graph [to enter their responses] and no real learning occurs.”

The other was that the online quiz questions were made available to students after the quizzes for study purposes, and, according to a survey of students, students took advantage of this resource.  The in-class CRS questions, however, were not made available to students for review.

Students reported on a survey that the most useful features of the CRS were that it helped students reinforce what they learned in class and that giving them the opportunity to talk about class material with their peers helped them learn the material.

Comments: This article is a useful example of how difficult it can be to design a study about classroom response systems that provides meaningful results.  I’m glad the authors published this account, even though their results aren’t strong.  I think others designing similar studies are likely to learn from some of the design mistakes made in this study, in part because the authors did a great job of exploring the constraints of their study design in the conclusion of the article.

One of the primary advantages of having students respond via clickers or similar classroom response systems is that students are able to respond to a question independently.  Each student is asked to respond before he or she finds out what his or her peers think.  This can increase the level of engagement of the students with the question and any subsequent discussion.  The “bandwagon effect” seen in this study, in which students wait to find out what their peers think and then pick the most popular response, is why asking for a show of hands can be unproductive.  Since the classroom response system in this study was used in a such a way as to eliminate this primary advantage, it’s difficult to draw meaningful conclusions from the study.

What is clear from this study is that the availability of clicker questions for students to review after class is a potentially important variable, one that should be included in future research on the effects of classroom response systems.  This study presents some evidence that this may be a key ingredient in the impact of clicker questions on student learning.  I hope that future studies take a closer look at this variable.

Studies like this one that compare classroom response systems with alternatives (like online quizzes or asking for a show of hands) can get complicated very quickly.  I thought I might list just a few of the variables at play in this study to illustrate how difficult it can be to isolate the effects of a CRS.

• Independent Answers – In this case, students didn’t have to answer in-class questions independently; they could see their peers answers before responding.  This was probably not the case for the online quizzes; students likely had to respond on their own before seeing their peers’ responses.
• Peer Instruction - Students in this study were required to pair up and provide consensus answers to the in-class questions.  Students worked on their own when responding to the online quiz questions.  This seems like a significant difference between the two methods.
• Class Results - Students were shown the class results to the in-class questions.  It’s not clear if they were shown the class results to the online quiz questions, either online or in subsequent classes.  The impact of seeing these class results is something worth exploring in CRS research since these results aren’t typically immediately available when other response mechanisms are used.
• Immediacy of Feedback – With the in-class questions, students found out immediately whether they answered the questions correctly.  It’s not clear from the article if students had to wait to receive feedback on their online quizzes.
• Agile Teaching - In-class CRS questions that were missed by significant numbers of students were reviewed in some fashion during class, which meant that the “lecture” was responsive to student learning needs evidenced by the CRS.  It’s not clear if online quiz questions influenced class time in a similar fashion in this study.  The ability to practice “agile teaching” is a primary advantage of a CRS, so this is an important variable to consider.
• Question Type – Not much is said in this article about the nature of questions asked in class via CRS or out of class via online quizzes.  The two sets of questions were judged to be of similar difficulty levels by a panel of four chemical educators, however.  The students appeared to answer the online quiz questions correctly about 88% of the time, so the questions weren’t that difficult.  The format, difficulty, and learning goals associated with questions are potentially important variables.
• Availability for Review – The online quiz questions were made available to students for test review; the CRS questions were not.
• Grading Scheme – The article doesn’t state how either the in-class questions or the online quizzes were factored into students’ grades.  Given the role grades play in student motivation, this is a variable worth noting.

These are some of the variables I look for when I read about studies exploring the impact of classroom response systems.  I hope this partial list will be of use to researchers reading my blog.

This week’s Chronicle of Higher Education includes an essay by Michael Bugeja, director of the Greenlee School of Journalism and Communication at Iowa State University, titled “Classroom Clickers and the Cost of Technology.”  (You’ll need a subscription to the Chronicle to use that link, unfortunately.)  In his essay, Bugeja expands on a few points he made about clickers in a prior essay.  I thought I would respond to a few of his points here.

I agree with some of Bugeja’s takeaways from his institution’s experiences with clicker vendors.  He argues that students should be involved in decisions about instructional technology, that chief information officers should be consulted by departments making such decisions, that faculty adopting technologies should be aware of not-so-obvious costs of using these technologies, and that administrators should be prudent when conducting cost-benefit analyses of new instructional technologies.

Those are all very sensible points.  However, I see some problems in the ways Bugeja uses clickers as an example in support of these points.  The fundamental weakness of the essay is that Bugeja seems to be doing a cost-benefit analysis on clickers without paying much attention to the benefits portion of that analysis.  As well-referenced as the cost portion of his analysis is, he fails to consider any of the research looking into the impact of teaching with clickers on student learning.

For instance, he quotes Ira David Socol of Michigan State University as saying, “The idea of wasting money on a device no more sophisticated pedagogically than raising your hand drives me nuts…”  However, there’s strong evidence (Stowell and Nelson, 2007) that when the hand-raising method is used, fewer students participate and students are more hesitant to answer questions honestly than when a classroom response system is used.  Those are significant differences and to ignore them is to fail to accurately describe key benefits of using clickers.

Bugeja also writes that had students at his institution been asked to weigh in on the cost-benefit question regarding clickers, “they probably would have said no because of excessive student fees.”  I can’t speak for students at Iowa State, but a number of published studies of student perceptions of clickers, including Trees and Jackson (2007), MacGeorge et al. (2007), and Kaleta and Joosten (2007), indicate that students respond positively to clickers, particularly when clickers are used in ways that engage them in class and provide them with feedback on their learning.  It should be noted that in the studies I just listed, students were required to purchase their own clickers.  Thus, there is evidence that students see the benefits of clickers outweighing the costs.

A second weakness of Bugeja’s argument is that he discusses the cost side of the cost-benefit analysis by focusing on the cost to install and maintain infrared-based classroom response systems.  IR systems are indeed costly to install and maintain and a bit of a pain for faculty and students to use.  However, arguing that classroom response systems aren’t worth the cost because infrared-based systems are costly is a bit like arguing that automobiles aren’t worth purchasing because steam-powered cars are a pain to use.  Very few colleges and universities are still using infrared-based clicker systems.  The radio frequency systems now in common use eliminate almost all of the installation, maintanence, and usage problems of the infrared systems.

As Bugeja points out, at Iowa State relatively few faculty members used clickers when the infrared system was the only one available.  When the easier-to-use and more-reliable radio frequency system was made available, “users then multiplied throughout the university.”  Bugeja makes good points about the costs involved in supporting early versions of clicker systems, but given how usage increased when more mature technologies were made available, I think a stronger takeaway is that institutions should be cautious when implementing new technologies.  Waiting for “version 2″ can help institutions avoid costs.  Universities now in the process of rolling out clickers widely can take advantage of the more mature radio frequency technologies and thus avoid all the hassle of the older systems.

There’s more I could say about this essay, but I’ll stop here for now.  I encourage you to respond to Michael Bugeja’s essay as well as my thoughts in the comments section below.

In my last post, I noted that students at Washburn University in Topeka, Kansas, used clickers to answer polling questions before and after they watched the recent presidential debate.  I heard from Barb DeSanto, chair of the Mass Media Department at Washburn, who organized the clicker portion of the debate viewing.  She let me know that they’re holding viewings for all the presidential and vice-presidential debates this year and that the clickers question asked and the results of those questions for each debate are available on the Mass Media Department wiki.

Barb DeSanto noted that by asking students pre-viewing clicker questions and letting them know that they would be answering post-view questions, as well, it’s likely that the students were a little more engaged during the debate.  This is a useful feature of clickers in other settings, too.  When students know that something will be asked of them, they tend to be a little more focused during any activities leading up to that “deliverable.”

Dr. DeSanto found that the reactions of Washburn students to the debates aligned well with national polling results.  This strikes me as useful, since I’ve talked with other instructors who have found that when student responses to in-class polling questions align well with national data, the students often take that national data more seriously.

Finally, I’ll point out that student responses to pre- and post-debate questions could have been collected without using clickers, perhaps through paper surveys.  One important advantage of clickers over surveys is that clickers provide immediate results.  This allows instructors to use those results to shape subsequent discussion.  I can imagine asking a few post-debate polling questions, then asking students to volunteer reasons why their peers might respond in the ways they did to the polling question.

If you have any thoughts on using clickers during this presidential election season, feel free to leave a comment below!

Reference: Jenkins, A. (2007). Technique and technology: Electronic voting systems in an English literature lecture. Pedagogy, 7(3), 526-533.

Summary: In this article, Alice Jenkins of Glasgow University in Scotland describes her use of clickers to teach an undergraduate poetry class with 110 students. Her primary use of clickers was for formative assessment immediately following a portion of a lecture on a particular topic, leading into “agile” teaching and class-wide discussion. Types of questions included the following:

• Application Questions – In the article, Jenkins focuses on using clickers to teach students metrical analysis of poems. She provides an example of this type of question, asking students to predict certain properties of the next line of a given poem, as well as an analysis of student responses to her example question.
• Critical Thinking Questions – She mentions asking her students “to assess certain formal qualities of poems” using a Likert scale and to choose an adjective that best describes a poem’s diction.

Jenkins also makes an interesting point about the use of the “hand-raising” method of answering questions in class. She asserts that this method works better for questions with two possible answers, since students can be asked to raise their hands for one answer and keep their hands down for the other answer. This allows students to answer a little more independently than the usual method of asking for a show of hands for each answer choice sequentially. However, she also mentions that non-participating students confuse the results here, since their lack of raised hands would be interpreted as “votes” for one of the two answer choices.

Jenkins is also fond of the “I don’t know” option (which isn’t possible with binary “hands-up” questions) since it discourages students from voting randomly (useful presumably because it encourages students to ask themselves, “How confident am I in this answer?”) and provides Jenkins with a sense of the difficulty level of a given question.

Jenkins was observed by a colleague during the classes in which she used clickers. The colleague reported that Jenkins had “asked for and received oral responses from the students 28 times.” Jenkins said she was “astonished” to hear this, presumably because this high level of interaction was unusual for this large course.

Jenkins also surveyed her students about the use of clickers. One interesting result was that 60% of her students said they “worked out the answers to all the questions” when clickers were used, versus only 10% without clickers. The primary uses of clickers students identified as beneficial were (a) helping students assess their own understanding, (b) allowing for anonymous responses, (c) helping the instructor assess student learning, and (d) increasing participation.

Commentary: This is the first published article I’ve found describing the use of clickers in a humanities class, so I was pretty excited to discover it. (Stuart, Brown, and Draper (2004), also from Glasgow University, describe the use of clickers in a philosophical logic course, but that kind of course is fairly unusual in the humanities.) I think there’s a lot of potential for the use of clickers in the humanities, and the interesting application and critical thinking questions Jenkins describes in this article are great examples of that potential. I hope that this article encourages others in the humanities to consider using clickers.

One of the reasons I think that instructors in the humanities have difficulty seeing value in the use of clickers is that their experience with multiple-choice questions is based on the use of such questions on exams, where they are usually factual questions. Asking these kinds of questions in class with clickers isn’t usually particularly exciting or useful, so I can understand why humanities instructors might not see value in clickers.

However, one can use clickers to ask “one-best-answer” questions that encourage critical thinking. For these questions, students are asked to choose from several answers, more than one of which has some merit. The point of these kinds of questions isn’t to find out if students can identify the “right” answer, since there are no “right” answers. Instead, the point is to engage students in a question (by asking all students to think about the question independently and commit to an answer) to lead into a rich class-wide discussion of the material. These kinds of “one-best-answer” questions don’t work well on exams (unless one requests that students defend their choices, I guess), but they can work very well in class.

Finally, I think it’s really interesting that 60% of Jenkins’ students said they thoughtfully considered questions asked via clickers versus 10% who said they would do so for questions not asked via clickers. I’m reminded of a student of Elizabeth Barkley‘s captured in a video Elizabeth shared at a conference I attended. In commenting on the Think-Pair-Share collaborative learning technique, the student said something like, “With Think-Pair-Share, I know I’m going to have to pair up and share my thoughts on a question, so I think about the question. If I know I’m not going to ‘pair’ or ‘share,’ then why should I ‘think’?” Jenkins’ survey results support this assertion!

A couple of weeks ago, Inside Higher Ed posted an article titled “Keeping Clickers in the Classroom” based on a presentation made by Kathy Keairns of the University of Denver at a recent Blackboard conference.  The author of the article, Andy Guess, seems to conclude that this presentation is evidence that clickers aren’t just a passing fad in instructional technology.  I don’t think a single conference presentation is sufficient evidence to support that conclusion, but I do agree with the conclusion.  I believe that the use of classroom response systems is only going to grow in higher education, a conclusion drawn from my conversations with dozens of faculty members (and those who support faculty members) across the country.  The particulars of the technology involved is likely to change (from dedicated clicker devices to smart phones, perhaps), but classroom response system pedagogies (polling all students in a class, using the results to inform classroom decisions, etc.) are too effective to fade away.

In the comments following the IHE article, a question was raised about the merits of clickers and similar high-tech devices over low-tech alternatives.  I weighed in on that discussion there, and I thought I might reproduce my comments here.

Bradley Beck wrote, “It seems the clicker can have it’s place, but if the red and green cards, or a quick request from students to write down what they learned that day will do the trick instead, then why waste time and money on clickers?” I certainly see pedagogical value in the use of flash cards or response cards and in having student submit “one-minute” papers at the end of class. I would, however, like to point out a couple of advantages clickers (or some other electronic system, like the Poll Everywhere text-messaging system mentioned above) have over these methods.

One limitation of the response card method is that it doesn’t allow instructors to hold students accountable for their answers. This can decrease participation and engagement. Also, the response card method doesn’t allow the students to see the distribution of responses, which can be an important use of classroom response systems. (Imagine asking a question about student opinion where students are surprised to learn about the diversity of opinions held by their peers.) Finally, there’s some evidence that the response card method provides less accurate information about student learning and student perspectives than clickers since students are able to see the responses of their immediate neighbors and switch their answers to more popular answers, even if those answers are not honest ones.

As for the idea of having students respond in writing to a prompt at the end of class, this method doesn’t allow an instructor to act upon information gained about student perspectives during class. Classroom response systems (including response cards) allow instructors to practice what is sometimes called “agile teaching,” responding in the moment to what they learn about their students’ learning. This can help instructors tailor their instructor to the immediate learning needs of their student, and it’s one of the great advantages of using a classroom response system.

Reference: Stowell, J. R., & Nelson, J. M. (2007). Benefits of electronic audience response systems on student participation, learning, and emotion. Teaching of Psychology, 34(4), 253-258.

Summary: In this short article, the authors present the results of a study comparing three different in-class response methods: clickers, response cards, and hand-raising.  About 140 introductory psychology students, mostly first-year undergraduates, were assigned to four different groups.  The first group experienced a “standard lecture” with no formal response mechanisms.  The other three groups experienced lectures enhanced with formal response mechanisms (clickers, response cards, hand-raising).  The authors analyzed the students’ participation and accuracy rates on these in-class questions, as well as their performance on a post-lecture quiz and their responses to the Academic Emotions Questionnaire (AEQ) as a way of investigating impact on the students’ affective responses.

Key findings are as follows:

• The response card and clicker methods increased student response rates over the hand-raising method (97%, 100%, and 76%, respectively).
• All three groups involving formal response methods reported significantly less boredom during lecture than the “standard” lecture group.
• The clicker group appeared to answer in-class questions more honestly than the response card and hand-raising groups.  This was the authors’ conclusion after noting that the percent of questions answered correctly using clickers more closely mirrored the percent of questions answered correctly on the post-lecture quiz.  (There was a 22% drop in accuracy from during-lecture to post-lecture for clickers versus a 38% drop for hand-raising and 40% drop for response cards.)

Comments: The first two findings aren’t particularly surprising, but are encouraging.  The third finding is the most interesting one.  Anecdotal evidence from instructors has certainly indicated that the hand-raising method leads to the “bandwagon” effect, where students change their responses after seeing how their peers respond, making it difficult to use that method to accurately assess student understanding.  However, I hadn’t heard until this article that the bandwagon effect was present in the response card method, a method that has a fairly long history in the teaching of psychology

It’s worth noting that in this study, the lecturers did not practice any kind of agile teaching; they just stated the correct answer to a question after the students voted.  Although there was no significant difference on the performance of the four groups on the post-lecture quiz in this study, it’s quite possible that there would be an improvement in the clicker group were agile teaching to be implemented.  By using response data to guide classroom lecture and discussions in ways that focus on student difficulties, I believe that students are likely to learn more.  If that’s the case and if clickers provide more accurate data on student difficulties, then that would be a strong argument for using clickers to improve student learning, not just increase participation and reduce boredom.

One variable not discussed in the article is the type of question asked with the various response methods and on the post-lecture quiz.  It’s possible that the effects seen here would be different with different types of questions: factual questions, conceptual questions, application questions, etc.

Finally, I like the use of the Academic Emotions Questionnaire here to explore the affective aspects of clickers.  I’m glad to know about this instrument.