Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Preservice Elementary Teachers’ Perceptions of Clicker Use in their College Mathematics Course,” Travis K. Miller, Millersville University of Pennsylvania [Slides]

In my last post, I mentioned that Janet White first used clickers in her courses for pre-service teachers at Millersville University of Pennsylvania. Another speaker in the contributed paper session back in January was her colleague, Travis Miller, who shared results of a student survey he conducted in the pre-service teacher course he taught. Travis used clickers for only six lessons during that course in each of the four sections he was teaching. His clicker questions weren’t graded, and he followed the “classic” peer instruction model each time, having students vote individually, then discuss the question in small groups, then vote again.

Travis’ students overwhelmingly (96%) liked using clickers in the course. Travis mentioned that there are very few things he does as a teacher that are as uniformly popular with his students! Almost as many students (89%) believed that the clicker activities helped them learn the material in those six lessons. Travis drilled down on this, asking students to say why the clickers were useful. The number one answer (59% of students) was that the clicker questions provided students with an opportunity to discuss and think about course content. The number two answer (23%) was that the clickers provided a sense of accountability and involvement.

Travis didn’t stop there, either. He asked his students which topics they understood better because of the clicker activities. Of the six topics that Travis addressed using clickers, sets and Venn diagrams was cited by 52% of the students as the one that most benefited from clickers. Numeration / base arithmetic was a distant second with 15%, and deductive reasoning came in third with 13%. When sharing these data, Travis floated a very interesting hypothesis. He wondered if the fact that the number one topic (sets and Venn diagrams) was a visual one led to the students selecting it as most benefited by clicker questions. I’m a big fan of visual thinking, so this comment caught my attention. Is there something special about peer instruction with clickers and visual thinking?  I’d appreciate your thoughts in the comments.

Travis’ other interesting hypothesis was that his more competitive students liked the competitive aspects of clickers (being the first to answer, answering correctly more frequently than other students, and so on), while the non-competitive students didn’t mind those aspects since they were essentially opt-in. That is, the students who didn’t want to compete could still participate fully with the peer instruction and voting process without feeling any pressure to treat it like a game. Graham, Tripp, Seawright, & Joeckel (2007) found that most students who are hesitant to participate in class liked clickers as well as those who were fine with participating, but I don’t think I’ve seen any research that compared competitive students with non-competitive students. That would make for an interesting research question.

Travis also taught some sections of his pre-service teacher course without using clickers, and he surveyed students in these sections about the potential advantages and drawbacks of using clickers. What concerns did they have about using clickers? They worried about the cost of the devices, that clickers weren’t necessary in small classes, that clicker activities take up too much class time, and that the technology might not be reliable. I found it interesting that these are among the common concerns of faculty members not already using clickers, too!

Image: “Happy Pi Day!” by Flickr user Mykl Roventine / Creative Commons licensed

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Using Personal Response Systems (Clickers) in Liberal Arts Mathematics Courses to Support a Lecture Format,” Janet A. White, Millersville University of Pennsylvania [Slides]

Just like Jean McGivney-Burelle and Kimberly Burch, Janet White shared her experiences teaching with clickers in a “liberal arts” mathematics course taken by non-majors. Unlike Jean and Kimberly, who teach relatively small sections of this kind of course, Janet teaches in a large lecture hall with 75 students per section. Janet had used clickers in courses for pre-service math teachers in the past and found them useful, so when it was her turn to teach this larger course, she decided to use them again. A classroom response system was hardly the only technology Janet used in this course: She also had students complete online homework and quizzes and she annotated her PowerPoint lecture slides using an Interwrite Mobi.

Janet used clickers on a daily basis in her course, usually either to assess students’ prior knowledge or to assess their understanding of a topic taught during lecture.  Her questions came from a bank of multiple-choice questions provided by her textbook publisher.  She counted the clicker questions as part of her students’ participation grades, but in a low-stakes manner.  Given her use of the questions as well as the source of the questions, many were on the lower levels of Bloom’s taxonomy, aimed at recall and application of procedural knowledge.  She shared an example of a prior knowledge question that asked students to find the measure of an angle that complements a 36 degree angle.  A slightly harder question aimed at assessment of something taught during the course asked students to identify the cut edge in a given graph (or to assert that the graph had no cut edge).

Student survey results indicated that 85% of Janet’s students who used clickers regularly liked using them, and 71% said that using clickers helped them learn the material.  Students who used clickers regularly during the course ended up with higher grades in the course than students who didn’t, but, of course, that can’t necessarily be attributed to the use of the clickers.  (And since clicker questions were factored in the course grade, students who participated more frequently in clicker questions would almost certainly have higher grades in the course anyway.)

Student comments about the clickers were generally positive.  My favorite one was, “I liked getting the wrong answer anonymously.”  Other comments addressed the usual points that students like about clickers: They liked the interactivity, they liked discussing questions with classmates, they liked seeing where they stood relative to their peers, and they liked the feedback on their own learning the clicker questions provided.  The only significant negative aspect for the students was the cost, about $50 in Janet’s case.

Janet found that having students discuss clicker questions in small groups led to very engaged students, even in the large auditorium environment.  In the future, she plans to write more of her own questions, instead of relying on ones from the textbook’s question bank.  She hopes to write more difficult questions that will generate even more engaged discussion during class.  She’s also hoping to find ways to reduce the technology cost to the students, either by selecting a different vendor or facilitating the resale of clickers after each semester to students taking the course the next semester.

Also, Janet mentioned that the earth science faculty at Millersville are big users of clickers.  Earth science instructors looking for advice on using clickers might want to investigate!

Image: “Recursive Daisy” by Flickr user gadl / Creative Commons licensed

More from my round-up of articles on clickers in the health professions.  This time, another article that doesn’t add much to the literature, but raises an interesting idea.  Again, your comments are invited…

Reference: Williams, B., & Boyle M. (2008). The use of interactive wireless keypads for interprofessional learning experiences by undergraduate emergency health students. International Journal of Education and Development Using ICT, 4(1).

Notes: This article has features results from a survey of students using clickers in a “foundations of health” course taken by emergency health students as well as students majoring in “nursing, physiotherapy, occupational therapy, midwifery, health science, and social work.”  The authors refer to this as interprofessional education, “learning that represents a way of fostering collaborative and seamless, integrated patient-care education.”  I first heard about this approach from my POD Network colleague Marilla Svinicki, who is involved in interprofessional education at the Clinical Education Center affiliated with the University of Texas-Austin.  The CEC is an impressive initiative.

I was interested to hear how clickers would play out in this setting, one featuring students with a diverse set of backgrounds and career goals.  However, the course is a first-year course, so the students weren’t likely to have differentiated themselves yet.  Moreover, there’s no attention paid to these different majors in the survey results that are reported here.  (The survey results are very positive, however, and are in keeping with other surveys I’ve mentioned here on the blog before.)

Have you used clickers in a course that included groups of students from different majors?  I can imagine forming heterogeneous student groups, then giving each group a single clicker as part of small-group activities during class.  How would you teach a course like this?

Update: Just a couple of days after posting this, I learned that Vanderbilt University has an interprofessional program something like the one at UT-Austin.  The Vanderbilt program involves medical and nursing students from Vanderbilt and social work students from Tennessee State University.  The students work together (with mentors) in clinical settings half a day per week and participate in classroom-based learning (using reflective exercises and case study activities) half a day a week, as well.  The rest of the week they participate in their respective programs as normal.  Given the complex nature of health care today, this seems like an incredibly sensible approach to health professions education.

Image: “Colourful Army” by Flickr user maistora / Creative Commons licensed

In spite of including experiences from not one, but two language instructors in my book, I still haven’t found any studies exploring the use of clickers in language classrooms for my bibliography.  And, if you check out the column to the right of this post, you’ll see the various disciplines I’ve covered here, and language instruction is not well represented.  (This very post will be only the second in that category.)  Since I’m pretty sure clickers have incredible potential in language instruction, you can imagine how glad I was to see a recent blog post about clickers in a Spanish class at Georgetown University!

The post is a report from Ellen Johnson, a PhD student in applied linguistics, who teaches and coordinates Spanish courses at Georgetown.  After hearing about clickers at a workshop hosted by Georgetown’s Center for New Designs in Learning and Scholarship (CNDLS), Johnson experimented with clickers in her language classroom.  Not only did she experiment with clickers, she collected some data useful in helping her judge their effectiveness.  Here’s what she did:

In a nutshell, 58 students enrolled in Beginning Spanish courses participated in the study on ser and estar. They were introduced to their uses in context, practiced answering questions using clickers with around 20 slides while viewing their performance in relation to their peers, and then completed posttests and reflection questionnaires.

Johnson also had colleagues observe the classes, and the feedback from both students and instructors about the use of clickers was very positive.  The students were particularly enthusiastic about getting immediate feedback on their learning, and Johnson’s fellow instructors thought the clickers had potential for helping them target their feedback to their students “in a more coherent manner.”

Students raised a couple of concerns in their feedback, however.  They thought that the clicker questions made it difficult for them to take notes during class.  Making clicker questions available to students after class is something I’ve mentioned here before, and there’s a little evidence that doing so is, in fact, very important since it allows students to review clicker questions later.  Knowing that clicker questions will be available online after class also frees students from having to take as many notes during class, which is likely to help them spend more time actually thinking during class.

Johnson’s students also noted that clicker questions don’t allow students to practice their speaking skills in a language class.  That’s a good point, but given the experiences of the language instructors I interviewed for my book, it would seem that clicker questions work very well for listening, reading, and writing skill development.

The main concerns raised by language instructors in Johnson’s study were logistical ones.  They worried that the technology would be difficult to start using or might fail during class.  I don’t know what system they use at Georgetown, but I know there are easy-to-use and reliable systems out there.  Also, Georgetown doesn’t seem to have a full-scale clicker implementation, one where students could be expected to purchase clickers at their bookstore, as is the case at many US colleges and universities.  That creates a logistical barrier, as well, since clickers would have to be distributed and collected each class session.

Thanks to Ellen Johnson for sharing her experiences with clickers.  I would be interested to hear more uses of clickers in language courses.  What kinds of questions and activities work well with clickers in those settings?  And why do you think that clickers aren’t more widely used in language instruction?

(I should also note that Ellen Johnson’s post appeared on a group blog from a group of instructors at Georgetown exploring the use of clickers this spring.  Take a look at previous blog posts for more interesting discussion of teaching with clickers.  This “community of practice” is another example of the value of fostering discussions about teaching and learning across the disciplines.)

Image: “Pink AC Bienvenidos” by Flickr user lopolis / Creative Commons licensed

Today, as you may have heard, is Ada Lovelace Day, a day in which bloggers are encouraged to write about women in technology they admire.  The day is named in honor of Ada Lovelace, widely regarded as the world’s first computer programmer.  Lovelace worked with Charles Babbage, mathematician, cryptographer, and organ grinder hater, who designed–but did not build–the world’s first computer, the analytical engine.  (As a result of Babbage’s lack of follow-through, Lovelace’s computer programs were, sadly, not actually implemented in her lifetime.)

I would like to take this opportunity to recognize someone I admire in the world of educational technology: Angel Hoekstra, who recently completed a PhD in sociology at the University of Colorado-Boulder.  For her dissertation research, Angel studied student perspectives on learning with classroom response systems in large chemistry courses.  Her first paper on this work came out in 2008:

• Hoekstra, A. (2008). Vibrant student voices: Exploring effects of the use of clickers in large college courses. Learning, Media, & Technology, 33(4), 329-341.

From the abstract:

This study investigates social, educational, and emotional effects of the use of SRSs–clickers–at the University of Colorado at Boulder.  Methods include participant observation, survey data from over 2000 students enrolled in three semesters of General Chemistry, and in-depth interviews exploring the nature of student experiences with clickers.

Angel’s use of not one, but three interesting methods of surfacing student uses of and perspectives on clickers–surveys, classroom observations, and in-depth interviews–give her findings a great deal of weight.  I’m a great fan of this kind of qualitative research.  I find it’s often more useful than the kind of quasi-control group, quantitative research often conducted around teaching with clickers–more useful in the sense that it provides a richer understanding of how students learn both with and without clickers.  Dr. Hoekstra’s work is a significant contribution to the literature on classroom responses in particular and educational technology in particular.  (More of my thoughts on this paper are available in an earlier blog post.)

Beyond her dissertation work, Angel also uses clickers in the sociology courses she teaches.  Her recent paper (with Stefanie Mollborn) on the use of clickers in sociology education is the first and, so far, only item in the “sociology” section of my clickers bibliography.

• Mollborn, S., & Hoekstra, A. (2010). “A meeting of minds”: Using clickers for critical thinking and discussion in large sociology classes. Teaching Sociology, 38(1), 18-27.

Angel will be presenting her dissertation research at the upcoming clickers conference at the University of Louisville, June 4, 2010.  Here’s her abstract:

Drawing from five years of research into the effects of clicker use in higher education, this presentation will make use of quantitative and qualitative data to illustrate the practical experiences of students and teachers who use clickers.  The presentation will compare best practices for clicker use across disciplines, with a focus on effective pedagogy in natural and social science courses.  Data from four disciplines (chemistry, astronomy, sociology, journalism) will be used to illustrate the potential of clicker technology for fostering increased engagement, conceptual understanding, cooperation, and solidarity, while engaging intermediate and experienced clicker users in general and discipline-specific strategies for clicker use.

If you’re heading to the Louisville conference, I encourage you to hear from Angel about her work!  And if you’re not going to Louisville, her talk is another great reason to go.

I’m glad that Angel is bringing a sociologist’s perspective to the growing body of research exploring the effects of teaching with classroom response systems, and I look forward to reading and hearing about her future work in this area.

For more Ada Lovelace Day posts, see the list of posts (over 2000 and still going–March 24th has a few more hours in it depending on your time zone!) at FindingAda.com.  Here are a few posts from blogs I follow:

• ProfHacker – “Today Is Ada Lovelace Day“
• Partially Ordered Thoughts (Mitch Keller) – “Women in Science & Technology: Helen E. Grenga“
• Old is the New New (Rob MacDougall) – “Lovelace and Somerville“
• Great Sufficiency (Michelle M. Hudson) – “Ada Lovelace Day 2010 – Mary Lou Jepsen“
• Future Tense (podcast by Jon Gordon) – “Bloggers Pause to Honor Ada Lovelace“
  
• 2D Goggles (Sydney Padua) – “Ada Lovelace Day!“
  

I received an inquiry from a colleague recently asking my thoughts on the use of a “Are you here?” clicker question asked at the beginning of class to take attendance.   Since this isn’t an uncommon question I hear, I thought I would weigh in on it here on the blog.

In general, I would advise against the use of an attendance slide for the following primary reason:  Surveys of students regarding their opinions of clicker use consistently indicate that students have negative reactions to being “monitored” by clickers.  Students prefer to use clickers in ways that more clearly and explicitly benefit their learning.

For instance, in a 2007 study by Graham, Tripp, Seawright, and Joeckel, the primary reasons for negative student feelings about clickers were technical problems, the cost of the devices, the use of clickers for grading, and (most relevant to this discussion) the use of clickers to mandate attendance.

Similarly, in another 2007 student opinion study, Trees and Jackson (citation here) wrote, “Students were apathetic or negative toward clickers when the technology did not change the classroom experience in a self-evident manner.  Students need to be able to accomplish different things through the use of clicker questions than could be accomplished by simply listening to a lecture of the same material.”  Since an attendance question doesn’t add anything to the students’ learning experience in class, students are likely to be “apathetic or negative” toward such a use, particularly if the students are the ones purchasing the clickers.

Since students are more positive about the use of clickers when they are used to clearly enhance their learning, I would suggest that instead of using a stand-alone clicker question for attendance, it’s usually better to infer attendance from another clicker question, one that is designed to help students engage more deeply with the material.  For instance, if you have students engage in peer instruction around a clicker question, the students will have a valuable learning experience, and you’ll still be able to determine from the response system reports which students were present that day in class.

Another option that many instructors implement is to start class with a “warm-up” question, typically an “easy” question about the reading or the previous class or an off-topic, sometimes funny question (perhaps about some recent event in pop culture or campus news).  Some instructors will have this question displayed on-screen as students arrive as a reminder to them to take their clickers out and get ready for future clicker questions during class.

For example, here’s a warm-up question I used last fall the week after the MTV Video Music Awards:

What do you think of Kanye West’s treatment of Taylor Swift at the MTV Video Music Awards?

1. He should be sent to a place where people aren’t.
2. He’s a jerk.
3. That was pretty funny.
4. He had a point–Beyonce’s video was way better.

This question had nothing to do with linear algebra, but it did provide a fun way to start the class session.  (If you’re interested, the top answer was “He’s a jerk” with 43% of the vote.  Runner-up with “He had a point” with 29%.)  Warm-up questions allow instructors to take attendance while also providing something else of value to the students–a friendly reminder to gear up for class and perhaps an icebreaker moment at the start of class.

I’ll also point out that if you include clicker questions as part of students’ participation grades in your course, even if you’re just grading on effort and not accuracy of their answers, you are effectively mandating attendance.  However, it seems that students see participation grades much more favorably than they do attendance policies.

Do you take attendance with clickers?  If so, how?

Image: “Angry Face” by Flickr user teapics

A math colleague of mine, who blogs under the name Doc Turtle, recently blogged about his use of a calculus worksheet that helps his students “guide themselves through the algebraically intense process of partial fractions.”  Doc Turtle reports that his students look forward to this kind of work, and he’s planning to develop more activities along these lines.

I’ve heard from several instructors who have students engage in this kind of active, self-directed learning in class (through worksheets, clicker questions, and so on) that some students complain that the professor isn’t doing any work.  I suspect that these are the students who expect to come to class, take a lot of notes, and figure the material out while working through their homework.  They can sometimes push back when their instructor isn’t presenting course content in the way they expect.

Of course, instructors who design and implement activities like Doc Turtle’s worksheet activity aren’t avoiding the hard work of teaching.  Instead, they’re being intentional about what they want their students to learn and they’re planning and facilitating experiences designed to help their students learn those things.

As Ian Beatty wrote over on his blog, “It’s not really creating [clicker] questions that’s tough.  The hard part is figuring out what I want my students to learn from the class, and casting that in terms of what I want my students to be able to do.”  Once he’s done that, he says it’s relatively easy for him to write effective clicker questions.  “Just formulate a question asking them to do that (in a particular context), and then much of the class activity is me helping them struggle through the process as they learn how.”

What struck me about Doc Turtle’s post was how excited his students are to engage in this kind of active learning.  As I mentioned above, not all students see this kind of learning as valuable.  Did Doc Turtle just get lucky with a batch of exceptional students?  I suspect not.  I’m guessing that he’s been teaching his students to learn this way since the first day of classes so that by this point in the semester, his students are perfectly willing to see this kind of activity as valuable.  I think that’s an important takeaway: If we’re asking our students to learn in a “new” way, then we need to help them learn how to learn in that way.

Do you find that your students push back when you ask them to engage in active learning in class?  How do you help them see the value in this kind of learning over time?

Reference: DeBourgh, G. A. (2008). Use of classroom “clickers” to promote acquisition of advanced reasoning skills. Nurse Education in Practice, 8(2), 76-87.

Summary: Gregory DeBourgh provides a useful introduction to using clickers in nursing education, focusing on pedagogical strategies that use clickers to promote critical thinking.  His exploration of critical thinking in the context of nursing education is particularly interesting.  Here’s a sample:

“Reasoning is about using intellectual power to draw conclusions, form judgments, and make inferences based on evidence, education, and experience… The practical significance of acquiring skill in advanced reasoning is to move to the level of predictive clinical reasoning which enables one to anticipate both ideal and likely outcomes given a set of data.”

DeBourgh argues that using classroom response systems to engage students in high-level questions is an effective strategy for developing their critical thinking skills.  He supports this assertion by drawing on the literature on the roles of feedback and questioning in learning and by sharing concrete examples of clicker uses in nursing education.

Included are three sample questions, including a “one-best-answer” question that asks students to identify the likely cause of a particular symptom shown by a patient in a case study.  DeBourgh endorses the use of such questions since they better represent situations students are likely to encounter in clinical settings where they must deal with ambiguity.  He also suggests asking question sequences based around patient cases that “change the focus to add new variables,” noting that doing so also reduces the cognitive load students experience when familiarizing themselves with a new case.

DeBourgh makes a good argument for using clicker questions to model critical thinking skills for students:

“Anticipate likely incorrect responses and prepare ‘talking points’ for discussion as this facilitates ‘thinking on your feet’ and makes more visible to students how an expert uses heuristics, reasoning, and refined problem-solving skills to gain command of a clinical situation.”

Asking questions designed to provide an opportunity for the instructor to model critical thinking is one instance of many DeBourgh describes of crafting questions to meet particular teaching and learning objectives.  In doing so, DeBourgh draws on articles by Ian Beatty on good question design, transferring Ian’s advice to the context of nursing education.

DeBourgh also points out that clicker questions embedded in PowerPoint can be particularly useful in nursing, a field which frequently uses pictures, diagrams, sound clips, and video–media that can also be embedded in PowerPoint.  He also notes that nursing courses often involve discussion of nursing ethics and student opinions about ethical decisions, topics that lend themselves well to clicker questions.

The article also includes results from a study survey about clicker use.  Student responses to rating questions are summarized, and student responses to open-ended questions are presented, as well.

DeBourgh ends with a few challenges involved in teaching with clickers, two of which are particularly significant.  He notes that since instructors can track student performance in a class on a daily basis, expectations for students are raised, which is not popular with all students.  DeBourgh also speaks to the increased expectations for instructors:

“The greatest challenge is the new role for faculty to plan the curriculum and instruction around ‘deep comprehension’ rather than ‘covering content’ using a traditional lecture format.”

Comments: I read this article in advance of my presentation at the Health Professional Educational Research Symposium earlier in the month, and I was particularly impressed with Gregory DeBourgh’s eloquence in describing critical thinking in the context of nursing education and in describing ways that clicker pedagogies can foster those critical thinking skills.

As I’ve tried to capture above, DeBourgh describes a variety of ways of using clickers in nursing education, and he included one approach that was entirely new to me, one inspired by the 50-50 option in the television game show Who Wants to Be a Millionaire? DeBourgh suggests that before the correct answer to a clicker question is revealed to students, an instructor might ask the students which answer choices should be eliminated.  It’s a little unclear how DeBourgh implements this, but I can even imagine setting up a multiple-mark question with four answer choices, then asking students not to select the one correct answer but to select two incorrect answers.  This would offer a nice change of pace in question format and would help students focus on more than just the correct answer.  It’s often useful for students to consider why some answer choices are plausible on the surface but actually incorrect.

Hopefully it’s also above that DeBourgh puts an emphasis on teaching with case studies (multimedia case studies, at that) in his article.  I understand that case study methods are perhaps more common in nursing than they are in other disciplines, and I appreciated reading this article as a way to better understand why that was the case.  DeBourgh’s comments about using clickers for discussing ethics also helped me better understand the disciplinary context here.

If you’re a nursing educator, please share a thought or two about using clickers in your field in the comments section!

Update: Greg DeBourgh emailed me and clarified his 50-50 technique.  Here’s what he said:

I display the potential four-answers to a given question, then before the students “vote” with their clickers, I ask for a volunteer or select a student at random (my clicker system has this feature) and ask the student to eliminate 2 of the 4 potential answers and to explain why they are eliminating these two. This speaking out loud of their rationale for eliminating two of the potential answers that are not related to the question strengthens the students’ reasoning skills. They actually get quite good at it. If the student I called upon to answer hesitates or is reluctant to speak, I invite them to choose a “consultant” in the room to help them out. I hope this clarifies a bit for you.

I asked Greg what he does if the student eliminates the correct answer.  Here’s his response:

If the student eliminates one of the correct answers, it is still learning, and so I ask “does everyone agree with the 50/50 elimination?” If someone objects, I ask for their rationale. If no one objects, I just let the process go and during the “reveal and rationale” we talk about why each answer is incorrect or correct.

Thanks, Greg, for this clarification, and for this great use of clickers.

Reference: Freeman, M., Blayney, P., & Ginns, P. (2006). Anonymity and in class learning: The case for electronic response systems. Australasian Journal of Educational Technology, 22(4), 568-580.

Summary: In an effort to investigate the importance to students of the anonymity afforded by classroom response systems, Freeman, Blayney, and Ginns surveyed the 139 students in an introductory management accounting course at the end of the term. Notably, the majority of students in this course were female (73%), 20-22 years old (93%), with non-English speaking backgrounds (82%).

Each three-hour class session in this course featured a number of “formative, mainly rules based, multiple choice questions.”  Students responded to these questions via clickers or hand-raising in alternating class sessions and were encouraged, but not required, to discuss their answers with their peers before responding.  The instructor would announce the correct answer to each question immediately after the distribution of responses was shared with the class and then practice “agile teaching” by using the results of the question to guide subsequent lecture and discussion.  This alternating system of response methods ensured that students were given opportunities to experience a response method that allowed students to remain anonymous (clickers) and one that did not (hand-raising).

Sixty-eight percent of students agreed or strongly agreed with the statement “I preferred answering in-class questions when my answers were anonymous to the instructor.”  Almost as many students (62%) preferred answering questions when their answers were anonymous to their peers.  About the same number of students (63%) agreed or strongly agreed that “anonymity was more important with in-class questions when [they] were uncertain about the answer.”

Additionally, students were asked to rank in order of preference four potential response methods-clickers, hand-raising, volunteering to respond when they knew the answer, and “cold-calling.”  Average student rankings of these four methods indicated a preference for anonymity, as the methods were ranked (on average) in the same order as I just listed them.

The authors conclude from these survey data that students value the anonymity (peer-to-peer anonymity and student-to-instructor anonymity) enabled by a classroom response system.  Since other response methods, such as hand-raising and response cards, are less anonymous, this argues for the use of clickers.

The authors make a couple of additional points about the potential interplay of culture and use of clickers.  One is that peer instruction was not as highly valued by these students as other aspects of learning with clickers, according to survey results.  It’s possible that the cultural diversity of the students (as measured by proxy by the percentage of students with non-English-speaking backgrounds) made peer instruction less useful for these students than for more homogeneous students surveyed in other studies who found peer instruction more useful.

The other is that demographic variables, including native language, had no significant impact on the student responses to survey questions, particularly the questions about the importance of anonymity.  “Using first language as a rough proxy for culture, and in particular openness to criticism, these results might be seen to contradict Banks (2003) who suggested that cultural background could impact preference for ERS usage.”  The authors suggest further study of the role of culture in students’ valuing of anonymity.

Comments: For a relatively simple survey-based study, this article raises some interesting questions.  For instance, given that the instructor of this course practiced agile teaching, altering his instruction based on the distribution of student responses to in-class questions, I would think that response methods that generated more honest responses from students would lead to more useful agile teaching.  That is to say, if students were less likely to indicate confusion about a question when the hand-raising method was used (because their answers would not be anonymous to their peers), then the instructor might overestimate his students’ comprehension of the topic at hand when relying on the response distribution and subsequently spend less time on the topic than might actually be warranted.  If true, then students who prefer anonymous response methods and who have instructors who practice agile teaching might benefit more from an anonymous response method than students whose instructors do no practice agile teaching.  Since I have yet to read a study that compared use of clicker questions with and without agile teaching, it’s unclear at this point how important this issue might be.

The questions raised about the interface of culture, participation, criticism, and anonymity are also very interesting ones.  They are ones that haven’t been on my radar until recently, when I heard a presentation by Parvanak “Pary” Fassihi, who described her use of clickers to engage students in English-as-a-Second-Language (ESL) classes at Boston University and the Boston campus of the Showa Women’s University (of Japan).  She shared that some of her students come from cultures where it is seen as impolite to disagree with others publicly.  These students are often very hesitant to engage in the small-group and classwide discussion of clicker questions that Pary tries to generate in her courses.  (I’ll note here that I interviewed Pary for my book.  She had a lot of interesting things to say about the role of clickers in her language instruction courses, but this observation about cultural views on public disagreement didn’t come up in our interview.)

Pary’s observation is consistent with the finding by Freeman, Blayney, and Ginns that their students, many of whom came from diverse cultures given that few of them spoke English as a first language, did not value peer instruction as highly as might be expected.  I imagine it is also consistent with Banks’ suggestion, mentioned in the article, that “cultural background could impact preference” for clicker usage.  I haven’t read the Banks article, but I hope to do so soon and review it here on the blog.

One criticism I would have of the article at hand is that comparing the clicker and hand-raising response methods doesn’t quite isolate the effect of anonymity since clickers offer a number of advantages over hand-raising that might be at play here.  Given the nature of the survey questions used, this isn’t a significant issue, since student perception of anonymity was the focus, not a control-group-study comparison of two difference response methods.  However, it is possible that the students’ preference for clickers over hand-raising over hearing from student volunteers could be an indication not of their preference for anonymity but of their preference for response methods that encourage more complete and/or more independent participation and/or greater accountability for participation.  Thus, I would argue that those data in particular don’t necessarily imply that students value anonymity when responding to in-class questions.

(Cold-calling also leads to more complete and independent participation and greater accountability for participation since students have to stay on their toes in case they are called upon.  However, I’m comfortable arguing that the stress generated by this method would outweigh any perceived benefits regarding participation or accountability.)

A better comparison might be between having students respond via clickers with and without a display of individual participant responses.  (Many classroom response systems have the ability to display on screen not only a histogram showing the distribution of responses but also a list of individual students and their particular responses.)  That would better isolate the anonymity factor from other advantages of clickers over alternate response methods.  It’s not a perfect solution, in part because students in a large class aren’t likely to know each other’s names and so might not be bothered by having their names displayed next to their responses and in part because this is a feature of classroom response systems that is infrequently used by instructors so its use in this context might seem too artificial.  Also, this plan would not isolate peer-to-peer anonymity from student-to-instructor anonymity.  However, I think it would yield more relevant data than comparing clicker use with hand-raising.

I couple of weeks ago on this blog, I shared a tweet by Colin Morris, a student at Kent State University in Ohio.  His comment (via Twitter) was, “44% OF MY U.S. HISTORY CLASS THINKS WATERBOARDING IS A SURFING TERM. I take back everything I’ve said about these ‘clickers’ being useless.”  After I shared this tweet on my blog, a few interesting things happened.

1. Colin Morris, the student who tweeted this (on the left above), found out about my blog post via Twitter and commented on my blog post, indicating that he saw pedagogical value in clickers but objected to the cost of his clicker, particularly as a senior who won’t use it in future courses.
2. Then Twitter user @iclickercrs, apparently affiliated with the i>clicker company, tweeted about my blog post.  Another Twitter user, @JonathanRose, a professor at Queen’s University in Ontario, Canada (on the right above), saw this tweet and decided to see how many of his (Canadian) students knew what waterboarding is.  He contacted Colin Morris, who directed Jonathan to the Kent State University instructor who posed this clicker question during class, John Jameson.  Jonathan Rose then used the same question in his introductory political science course.
3. Jonathan then posted the results of both questions–the student responses from Kent State and the ones from his Queen’s University students.  Here are the results [PDF].  As you can see, only 28% of his students thought that waterboarding is a surfing term.  Also, more of his students than the Kent State students viewed waterboarding more as torture than an interrogation technique.
4. To bring things full circle, Jonathan Rose tweeted about these results and @iclickercrs re-tweeted Jonathan’s tweet.  I saw this re-tweet, then tracked down Jonathan.  He let me know about items 2 and 3 above, filling in the gaps in my knowledge of this whole social media process.

Watching this all unfold has been very interesting, not only for the interesting uses and reactions to clicker questions, but for the way that Twitter has facilitated connections that might not have happened otherwise.

One Last Update: Colin Morris blogged about this, too, noting the importance of keeping in mind potential audiences when using social media.