There’s a lively discussion happening on the POD Network listserv this week on teaching large classes. The discussion detoured into a discussion of teaching with clickers. In responding to one of these posts, Louis Schmier wrote:

“Well, Ron, clickers might get feedback and active and collaborative involvement, but learning? Technology is a tool, not a panecea. The basic problem with large class as Ron defines it, is that it violates the basic Aristotelian tenet of KNOWING those in your audience and tailoring yourself so that those in the audience get it, understand it, and retain it.”

This comment struck me as interesting, so I responded to it on the listserv. I’m including my response here on the blog (with a couple of extra links for clarity), in case those not on the listserv find it helpful.

Anton [Tolman] has responded very eloquently to Louis’ concerns about classroom response systems, but I can’t resist weighing in myself. First, there’s plenty of evidence that “active and collaborative involvement” often leads to student learning, so if clickers are indeed fostering more student engagement during class, that sets the stage for more student learning.

And as for the idea of “KNOWING those in your audience and tailoring yourself so that those in the audience get it, understand it, and retain it,” once you get past 15-20 students, it becomes very difficult to do those two things—assessing your students’ learning during class and practicing “agile teaching” by responding to what you find out about their learning on the fly–without a tool like a classroom response system. In fact, these are two teaching tasks that clickers are incredibly well-suited to support.

Imagine you have a single student in your office asking for help in your course. It’s relatively easy to “diagnose” that student and get a sense of what the student understands and doesn’t understand, then to tailor some one-on-one instruction to help the student resolve his or her misunderstandings. If you have 2-3 students in an office hour setting, you can probably do the same thing, although you’re already juggling 2-3 different “private universes” at this point, which can be challenging.

When you move to the “small” class setting, say, 8-10 students, you now have 8-10 “private universes” to try to uncover and respond to. Sure, there could be some similarities among your students in terms of their prior experiences, misunderstandings, and perspectives on course material, but you’ve still got 8-10 different students to build your learning environment in response to. Given 50 or 75 minutes and plenty of discussion, you’ve got a good shot at this, however.

Now move to a bigger class, say 15-20 students. At this point, it’s tough to have enough “air time” for all the students during class. This makes the juggling of “private universes” very challenging. Small group discussions can help (outsourcing some of this work to the students themselves), as can pre-class assignments. But during class, you’ve got quite a task if you want to be responsive to all your students’ various learning needs.

(Here you have my answer to Jeanette [McDonald]‘s question. When is large large? I would say 15-20 students. At that point the dynamics shift in very significant ways.)

Now imagine more students—30, 50, 100, or 500. The challenge of responding to that many unique “private universes” is truly daunting! You have to start making some assumptions about commonalities among those private universes. Clickers are wonderful tools for getting a sense of the validity of those assumptions! You pose a multiple-choice question where the answers are crafted to tie into what you suppose are common understandings (correct or not) and perceptions about the topic at hand, you have the students think about (and maybe discuss in small groups) the question, then you poll them and find out which of the understandings and perceptions are *really* the most common.

The resulting bar chart tells you how to spend the next 5-20 minutes of class time: responding to the student views of the topic that are most common. This “agile teaching” allows you to make the best use of limited class time by responding to as many “private universes” as you can in the time available.

Some caveats: You could miss a very common student understanding or perspective completely when you write your clicker question! The more experience you have with the topic and with students learning about the topic, the less likely this is to happen, but it’s still something to watch out for. That’s why it’s helpful to have some kind of classwide discussion about the question, giving students whose views aren’t represented in the bar chart a chance to share.

You also won’t get to the “long tail” of student views this way. What about the two students in a class of 100 who voted for option D? Will you have time to address that minority view? Maybe not during class, but perhaps after class in some fashion.

I’m also assuming here that you’re teaching a large class! The debate over whether or not classes should have 100 students is secondary to my point here. My point is that *given* a large class, a classroom response system is an excellent tool for understanding one’s students (in the aggregate) and tailoring one’s instruction to one’s students.

Lots more on these ideas (with examples from real classrooms!) in the “agile teaching” category on my blog.

The discussion on the listerv continued productively from here. It’s worth checking out.

Image: “O is for Occipital Lobe” by Flickr user illuminaut / Creative Commons licensed

Over on the Vanderbilt University Center for Teaching blog, my Vanderbilt colleague Isabel Gauthier, professor of psychology, has shared her experiences asking her students to write their own clicker questions. I met with Isabel a few years ago and briefly discussed ways to use clickers in her courses, and she’s really taken the technology (and pedagogy) and run with it. She’s got a great handle on how to have students write their own clicker questions, and I’ve been wanting to share her experiences here on the blog for a while. Here’s her article, in her own words:

It is difficult to write meaningful and discriminative multiple-choice questions that students find clear and fair. Years ago, I met with CFT assistant director Derek Bruff, who gave me useful pointers to perfect this skill. But a side effect of this interaction transformed entirely the way I teach: I learned so much by working on writing better questions, surely my students could learn too! Derek said something like, “You know, some teachers ask their students to generate questions…” This idea took me on a path to use this strategy, cautiously at first, and then more boldly, as the central pedagogical and evaluative strategy in some of my courses, including Brain Damage and Cognition and Principles of Experimental Design.

I teach these courses three days a week. On two of these days each week, I lecture on course material. These lectures are informed by questions about the readings posted online by students and issues that emerge from a hands-on, semester long project I assign my students. On the third day, we use clickers to go through student-generated multiple choice questions.

Each week each student is responsible for turning in a single question on the weeks’ readings. Students use a PowerPoint template to submit their questions which facilitates use of the question in my clicker software, TurningPoint. In the notes area of the slide, each student includes their name, the correct answer, the page(s) that inspired the question, and, optionally, a justification for the correct answer. Before class, I concatenate all the questions in a single file and read them, grading each on a scale of 1 to 5. The grade goes in the notes area, and, in a textbox on the slide, I write comments about the question. This allows me to print the slides as a PDF with student names removed so that all questions and comments can be distributed to students. I then reorder the slides to choose the right mix of questions I want to use in class with the clickers.

This provides me in a single step with my preparation for the next class, an idea of what I need to focus on during my lecture days, an evaluation of each student, and a mechanism for providing students with feedback on their learning. This weekly feedback allows students to realize how difficult it is to write a good question, one that raises an important issue clearly and is appropriately challenging for their peers. Students eventually learn to key in on critical concepts and relationships in the readings and sometimes even go beyond the readings in interesting ways. They take a more active part in their own and their peers’ learning, and their questions keep me focused on what is most challenging for these students at each point in the course.

Each week students answer the best of these questions in class using clickers, accumulating points for their answers using a generous but motivating grading scheme. If there’s controversy over the correct answer to a question, the class can decide to eliminate a question or to accept multiple answers as correct, provoking interesting discussions. As needed, I can lecture for a few minutes, but issues are generally clarified in class discussion. Questions are used anonymously in class, but students want their question to be picked and use wit and humor to this effect, making the experience more enjoyable for everyone.

This method completely replaces any exams I used to give: They are no longer needed since my students now share the responsibility to evaluate their own learning throughout the semester.

Isabel and her use of clickers were featured on Nashville’s NewsChannel 5 last year. Here’s the video clip:

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Using Personal Response Systems (Clickers) in Liberal Arts Mathematics Courses to Support a Lecture Format,” Janet A. White, Millersville University of Pennsylvania [Slides]

Just like Jean McGivney-Burelle and Kimberly Burch, Janet White shared her experiences teaching with clickers in a “liberal arts” mathematics course taken by non-majors. Unlike Jean and Kimberly, who teach relatively small sections of this kind of course, Janet teaches in a large lecture hall with 75 students per section. Janet had used clickers in courses for pre-service math teachers in the past and found them useful, so when it was her turn to teach this larger course, she decided to use them again. A classroom response system was hardly the only technology Janet used in this course: She also had students complete online homework and quizzes and she annotated her PowerPoint lecture slides using an Interwrite Mobi.

Janet used clickers on a daily basis in her course, usually either to assess students’ prior knowledge or to assess their understanding of a topic taught during lecture.  Her questions came from a bank of multiple-choice questions provided by her textbook publisher.  She counted the clicker questions as part of her students’ participation grades, but in a low-stakes manner.  Given her use of the questions as well as the source of the questions, many were on the lower levels of Bloom’s taxonomy, aimed at recall and application of procedural knowledge.  She shared an example of a prior knowledge question that asked students to find the measure of an angle that complements a 36 degree angle.  A slightly harder question aimed at assessment of something taught during the course asked students to identify the cut edge in a given graph (or to assert that the graph had no cut edge).

Student survey results indicated that 85% of Janet’s students who used clickers regularly liked using them, and 71% said that using clickers helped them learn the material.  Students who used clickers regularly during the course ended up with higher grades in the course than students who didn’t, but, of course, that can’t necessarily be attributed to the use of the clickers.  (And since clicker questions were factored in the course grade, students who participated more frequently in clicker questions would almost certainly have higher grades in the course anyway.)

Student comments about the clickers were generally positive.  My favorite one was, “I liked getting the wrong answer anonymously.”  Other comments addressed the usual points that students like about clickers: They liked the interactivity, they liked discussing questions with classmates, they liked seeing where they stood relative to their peers, and they liked the feedback on their own learning the clicker questions provided.  The only significant negative aspect for the students was the cost, about $50 in Janet’s case.

Janet found that having students discuss clicker questions in small groups led to very engaged students, even in the large auditorium environment.  In the future, she plans to write more of her own questions, instead of relying on ones from the textbook’s question bank.  She hopes to write more difficult questions that will generate even more engaged discussion during class.  She’s also hoping to find ways to reduce the technology cost to the students, either by selecting a different vendor or facilitating the resale of clickers after each semester to students taking the course the next semester.

Also, Janet mentioned that the earth science faculty at Millersville are big users of clickers.  Earth science instructors looking for advice on using clickers might want to investigate!

Image: “Recursive Daisy” by Flickr user gadl / Creative Commons licensed

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Clickers in the Classroom,” Kimberly J. Burch, Indiana University of Pennsylvania [Slides]

Kimberly teaches a “Math 101″ survey course called “Foundations of Mathematics.”  Topics covered include set theory, graph theory, and counting methods (among others), and Kimberly shared several interesting clicker questions on each of these topics.  For example, here’s one of her questions from the unit on graph theory:

How many vertices are there in a tree with 19 edges?

1. 19
2. 18
3. 20
4. Not enough information given

Kimberly practices the “classic” peer instruction technique of having students vote individually first, then discuss the questions in small groups, then vote again.  She finds that students often converge to the correct answer on the second vote.

In the example above, her students were split between 18 and 20 on the first vote, but after the peer discussion time, most students went with the correct answer, 20.  I found this interesting because the “Not enough information given” seemed to be the obvious wrong answer to this question.  A graph with 19 edges might have any number of vertices, but a tree with 19 edges can only have 20 vertices.  Students who don’t realize that trees are graphs with very specific properties might be tempted to go for the “Not enough information given” option.

I suspect that Kimberly used this question after the students learned the relationship between the number of edges and number of vertices in a tree and that this question was meant to assess whether students remembered that relationship.  Some students likely remembered that one of these numbers was one more than the other but weren’t sure which one was higher.  That would account for the split vote between 18 and 20.  Had this question been asked as an exploratory question and not a review question, I’m betting the split would have been between 20 and “Not enough information given.”

Kimberly also mentioned that she uses her clicker system’s priority ranking questions to have her students decide what topics should be emphasized during exam review sessions.  Kimberly gives her students a list of 8-10 exam topics, and the students indicate the top three or four toughest topics in order.  Kimberly said that this helps her make good use of limited exam review time by focusing on the topics the students find the most difficult.

Kimberly also shared some data from a quasi-control group experiment she conducted.  She taught two sections of this survey course and alternated which topics she covered with clickers in the two sections.  For example, section A might cover topic 1 with clickers while section B covered topic 1 without.  Then for topic 2, section B used clickers and section A didn’t.  She then compared test scores for the two sections by topic.  For some topics, students using clickers performed better on exams but for other topics, the students not using clickers performed better.  And for other topics, there was no difference.  The data was generally favorable to using clickers, but the “quasi” part of this quasi-control group experiment made it difficult to draw firm conclusions.

Image: “Point Marian Bridge” by Flickr user timmenzies / Creative Commons licensed

In spite of including experiences from not one, but two language instructors in my book, I still haven’t found any studies exploring the use of clickers in language classrooms for my bibliography.  And, if you check out the column to the right of this post, you’ll see the various disciplines I’ve covered here, and language instruction is not well represented.  (This very post will be only the second in that category.)  Since I’m pretty sure clickers have incredible potential in language instruction, you can imagine how glad I was to see a recent blog post about clickers in a Spanish class at Georgetown University!

The post is a report from Ellen Johnson, a PhD student in applied linguistics, who teaches and coordinates Spanish courses at Georgetown.  After hearing about clickers at a workshop hosted by Georgetown’s Center for New Designs in Learning and Scholarship (CNDLS), Johnson experimented with clickers in her language classroom.  Not only did she experiment with clickers, she collected some data useful in helping her judge their effectiveness.  Here’s what she did:

In a nutshell, 58 students enrolled in Beginning Spanish courses participated in the study on ser and estar. They were introduced to their uses in context, practiced answering questions using clickers with around 20 slides while viewing their performance in relation to their peers, and then completed posttests and reflection questionnaires.

Johnson also had colleagues observe the classes, and the feedback from both students and instructors about the use of clickers was very positive.  The students were particularly enthusiastic about getting immediate feedback on their learning, and Johnson’s fellow instructors thought the clickers had potential for helping them target their feedback to their students “in a more coherent manner.”

Students raised a couple of concerns in their feedback, however.  They thought that the clicker questions made it difficult for them to take notes during class.  Making clicker questions available to students after class is something I’ve mentioned here before, and there’s a little evidence that doing so is, in fact, very important since it allows students to review clicker questions later.  Knowing that clicker questions will be available online after class also frees students from having to take as many notes during class, which is likely to help them spend more time actually thinking during class.

Johnson’s students also noted that clicker questions don’t allow students to practice their speaking skills in a language class.  That’s a good point, but given the experiences of the language instructors I interviewed for my book, it would seem that clicker questions work very well for listening, reading, and writing skill development.

The main concerns raised by language instructors in Johnson’s study were logistical ones.  They worried that the technology would be difficult to start using or might fail during class.  I don’t know what system they use at Georgetown, but I know there are easy-to-use and reliable systems out there.  Also, Georgetown doesn’t seem to have a full-scale clicker implementation, one where students could be expected to purchase clickers at their bookstore, as is the case at many US colleges and universities.  That creates a logistical barrier, as well, since clickers would have to be distributed and collected each class session.

Thanks to Ellen Johnson for sharing her experiences with clickers.  I would be interested to hear more uses of clickers in language courses.  What kinds of questions and activities work well with clickers in those settings?  And why do you think that clickers aren’t more widely used in language instruction?

(I should also note that Ellen Johnson’s post appeared on a group blog from a group of instructors at Georgetown exploring the use of clickers this spring.  Take a look at previous blog posts for more interesting discussion of teaching with clickers.  This “community of practice” is another example of the value of fostering discussions about teaching and learning across the disciplines.)

Image: “Pink AC Bienvenidos” by Flickr user lopolis / Creative Commons licensed

Earlier this month, David Clemens, who teaches literature at Monterey Peninsula College, authored a post titled “The Data-Driven Classroom” on the National Association of Scholars (NAS) blog.  In his satirical post, Clemens he refers to clickers as “the future of edubiz:”

Clickers generate tsunamis of admin’s holy grail—data! With data I can prepare reports and quantify learning to show that I’m accountable.

His post goes on in this tone, satirizing those who use technology for technology’s sake.  (He includes a few shots at the use of PowerPoint and hyperlinked texts.)  I objected to his characterization of clickers as primarily used to generate assessment data for administrators, so I left a couple of comments on the post, arguing that clickers are primarily used to engage students during class, not to generate data for administrators.

David Clemens then wrote a follow-up post, “Reply to Bruff,” on the NAS blog.  In this post, he asserts the following:

Students urgently need unmediated classrooms and Exemplary-level teachers [those who model critical reasoning for students] who through Socratic dialogue and shared inquiry develop independent, informed, and original minds.

He argues that clickers (and PowerPoint and student learning objectives) don’t help teachers meet this goal.  Since I believe they do and that the useful effects of clickers are frequently underestimated in the humanities, I posted a lengthy comment elaborating on my earlier comments on the NAS blog.  You can read Dr. Clemens’ posts by following the links above.  Below you’ll find my response.

I was quoted this morning in “At Universities, Is Better Learning a Click Away?“, an Associated Press story on the future of classroom response systems by AP reporter Eric Gorski.  The story features Michael Dubson, who teaches physics with clickers at the University of Colorado-Boulder.  CU-Boulder, and its physics education research group in particular, has been very active in the world of clickers (including contributing to these great videos), and I was glad to hear Michael Dubson’s perspectives on the technology in the AP piece.

CU-Boulder is an i>clicker campus, and Dubson makes the case in the AP story that a simple, dedicated clicker device is preferable in most instances to more flexible systems based on smart phone apps.  Indeed, i>clicker devices have only six buttons–an on/off button and buttons labeled A, B, C, D, and E.  This is a very simple system, but, as inventor Tim Stelzer argued at the Louisville clicker conference back in 2008, multiple-choice questions with five answer choices work very well for the kinds of formative assessment and peer instruction many instructors use clickers to implement.

Gorski places me on the other side of a somewhat-artificial divide:

Derek Bruff, assistant director of Vanderbilt University’s Center for Teaching, said simple clickers are great at multiple choice questions. But he’s more excited about using smart phones, which allow students to ask questions of instructors, hold back-channel discussions with each other and respond in their own words.

Regular readers of this blog know that I’m definitely excited by the possibilities of using smart phones as “super-clickers” or to facilitate backchannel discussion in the classroom.  It’s true that I’m more excited by smart-phone systems than I am by simple clickers like i>clicker, but that’s largely because I’ve been involved in teaching with clicker with several years and I’m eager to leverage that experience to consider new kinds of technology-facilitated classroom dynamics.  (For one thoughtful perspective on those potential dynamics, consider Sean Seepersad’s recent post on moving away from clickers.  I hope to blog about Sean’s post soon!)

I’ve spent plenty of time thinking about the pedagogy of multiple-choice questions (while writing my book, blogging about clickers here, and giving talks on the subject around the country), and I think the multiple-choice format is often underrated.  I even have an article coming out (soon, I hope!) titled, “Multiple-Choice Questions You Wouldn’t Put on a Test: Promoting Deep Learning with Clickers.”  So I definitely get where Michael Dubson is coming from: Five-answer multiple-choice clicker questions are incredibly useful in all kinds of courses.

All this to say that one of the principles I attempted to uphold when writing my book was that everyone’s teaching context is different–different students, different disciplines, different institutions, different teaching styles and experiences.  I’m interested in helping instructors think more intentionally about their teaching choices, exploring the pros and cons of choices both traditional and innovative.  So while I may be more excited myself about smart phone systems, I always encourage instructors to select technologies and teaching practices that make the most sense in their particular teaching contexts.

I’m glad for clickers to receive the attention of the Associated Press.  The story has been all over Twitter today, and I hope it makes its way into print and online newspapers across the country.  And I’m glad that I could help Eric Gorski out as he was researching this story.  Eric also contributed to a short video piece to accompany his article, and he blogged about the story on the AP’s Facebook page.

Thoughts on the AP story?

Image: “The Nabla System (Forgotten Seed)” by Flickr user Syntopia

Classroom assessment techniques (CATs) are simple, non-graded, usually anonymous, in-class activities designed to give you and your students useful feedback on the teaching-learning process as it is happening.  The standard reference on CATs is Classroom Assessment Techniques: A Handbook for College Teachers by Thomas A. Angelo and K. Patricia Cross (Jossey-Bass, 1993). This book includes 50 CATs, each described in detail with examples from a variety of disciplines.  You’ve probably heard of a few of these, such as the minute paper, muddiest point exercise, and background knowledge probe.

CATs provide what is known as formative assessment, something I’ve frequently blogged about.  This is assessment of student learning intended to inform future teaching.  Formative assessment is often contrasted with summative assessment, which is performed in order to evaluate student performance.  Summative assessment comes at the end of a learning experience; formative assessment happens as the students are learning.  Feedback from formative assessment can provide instructors with useful insight into what students are understanding, what they are not understanding, and how they might target their teaching to their particular students.

At the recent EDUCAUSE Learning Initiative conference, Charlotte Briggs (University of Illinois-Chicago) and Deborah Keyek-Franssen (University of Colorado-Boulder) presented the results of a very useful study.  They combed through all 50 CATs in the Angelo and Cross book and determined that 23 of them could be used with clickers.  I’ve long thought of classroom response systems as a sort of “technoCAT,” a technology-enhanced classroom assessment technique, since they provide such useful formative assessment of student learning.  Charlotte and I connected via Twitter some time ago, and she had let me know that this analysis of the Angelo and Cross book was in the works.  I was very excited to see her work presented at the ELI meeting!

Charlotte and Deborah’s PowerPoint slides are available, as is their handout listing all 23 CATs that can be performed with clickers.  In their slides, they provide the following example of a CAT that can be used “as is” with clickers.

Background Knowledge Probe: Which of the following is NOT a characteristic of Romanticism?

1. Attention to “the natural”
2. Valued “folk” literature, such as fairy tales
3. Had a strong geographical center in Düsseldorf
4. Referred to “the blue flower” as a central symbol for longing
5. Valued medieval literature and art.

You can imagine asking this kind of clicker question at the start of a unit on Romanticism–or a unit that referenced Romanticism but didn’t focus on it.  If knowledge of Romanticism is important for participating in the discussion that followed, then this question will let instructors know how much time they need to spend reviewing Romanticism at the start of the unit.

The background knowledge probe CAT is one that I referenced in my book since it’s such a common use of clickers.  Where Charlotte and Deborah’s work gets more interesting is in their analysis of the other 49 CATs in the Angelo and Cross book!  For instance, they identify 12 other CATs that can be used “as is” with clickers, including such CATs as approximate analogies, problem recognition tasks, self-confidence surveys, and goal ranking and matching.  They also identify 10 CATs that can be modified to work with clickers.

For example, Angelo and Cross describe the “one sentence summary” CAT, in which students are asked to write a one-sentence summary of a reading or lecture using the WDWWWWHW format: Who Does What to Whom When Where How and Why.  Charlotte and Deborah note that students aren’t able to construct and submit these sentences using clickers.  However, they can be given a potential one-sentence summary and asked to identify its flaws.  The example they share in their slides is this one:

One-Sentence Summary: Find the errors in WDWWWWHW: A grand jury is a panel of judges (who) that decides if someone should be charged with a crime (does what to whom) when the offense might be a felony carrying prison time (when) if federal courts and most state courts (where) by listening to arguments by attorneys from both sides (how) so common sense and community perspectives are part of the criminal justice system (why).

1. Who and Why
2. When and Where
3. How and Why
4. Who and How
5. Does What to Whom and How

This clicker serves much the same purpose as a “traditional” one-sentence summary, in part because it’s a “multiple mark” style of question, asking students to identify not one, but two things wrong with the given summary.  If your clicker system allows actual multiple-mark questions, allowing students to select as many incorrect elements as they wish, the question becomes even more complex–and thus closer in usage to the one-sentence summary described by Angelo and Cross.

Charlotte and Deborah make a few very good points about modifying CATs to work with clickers.  They note that doing so “tends to down-grade the complexity” of the CAT itself.  With the one-sentence summary, for instance, you lose the ability to see what surprising things students might say in their constructed sentences.  However, Charlotte and Deborah point out that class discussion of the clicker question can restore that complexity.  As they write, “Instructors often get the most out of clickers when they are used to prompt discussion,” which is a point I always make when I talk about teaching with clickers.

Here’s one more great example along those lines.  Instead of asking students to write down the “muddiest point” of a lecture at the end of class, Charlotte and Deborah suggest in their handout the following:

List potential topics on slide and include an “other” option. Ask students to indicate the topic with which they had the most difficulty. If a significant proportion of the class selects “other”, probe the class to identify other “muddy” issues.

For other ideas on adapting CATs for use with clickers, take a look at their handout.  The Angelo and Cross CATs book is well-known in some educational circles (not so much in others, unfortunately), and Charlotte and Deborah’s work serves as a nice introduction to teaching with clickers for those familiar with the book.  Conversely, those already teaching with clickers are likely to find a few new ideas for using them as they explore the CATs framework.

Have you seen the new product (in beta) from Microsoft called Mouse Mischief?  I heard about it on Twitter a few weeks ago.  It’s an add-on to PowerPoint aimed squarely at the K12 market, but it’s of potential interest to those in higher education looking for alternatives to clickers.

To use Mouse Mischief as a classroom response system, a teacher embeds a special multiple-choice question slide in her PowerPoint presentation.  Each student in the classroom needs a mouse connected (wirelessly, no doubt) to the teacher’s computer.  When the question slide is shared with the class, each student sees a unique mouse cursor on the big screen.  They manipulate these cursors with their mice, using them to click on their answers to the question at hand.  The program then provides a bar chart showing the distribution of student responses.

I hear a lot about clicker alternatives that involve student cell phones or smart phones or laptops, but this is the first tool I’ve seen that uses mice as student input devices.  That’s a clever idea, but it has a very significant flaw.  Since the students can see their peers’ mouse cursors on the screen as they answer, students can’t answer independently!  This means that you’ll see the same lemming effect you see with hand-raising.  Once students start to figure out which cursors belong to the “smart kids,” they’ll just wait for those kids to answer and copy them.

Given this very significant flaw, I can’t really see how this tool would be useful.  It’s better than a show of hands, I guess, since it records student responses, allowing teachers to hold students accountable for their class participation.  That’s a good thing.  But the lack of independent responses is a real deal-breaker in my view.  What are your thoughts on Microsoft Mouse Mischief?

Reference: James, Mark C., Barbieri, F., & Garcia, P. (2008). What are they talking about? Lessons learned from a study of peer instruction.  Astronomy Education Review, 7(1).

Summary: This study is a follow-up to James (2006), a study comparing the effect on student discourse during clicker questions of grading incentive.  In the earlier study, student conversations were audio-recorded in two different astronomy course, each taught by different instructors and each involving different grading schemes for clicker questions.  The main finding of the earlier study was that a low-stakes grading scheme (one in which incorrect answers counted as much as correct answers) encouraged richer student-to-student discussions prior to voting on clicker questions than a high-stakes grading scheme (one in which incorrect answers counted only a third as much as correct answers).

A key drawback to James’ earlier study was that the two courses being compared were different in significant ways other than the grading scheme used for clicker questions.  In particular, they had different topics and different instructors.  That drawback has been mostly eliminated in the current study by James, Barbieri, and Garcia.  In this study, the same instructor taught the same course, an introduction to astronomy course with about 180 students, in two consecutive semesters.  In both semesters, clicker questions contributed 12.5% of the students’ overall course grades.  In the first semester, incorrect answers counted one-third as much as correct answers, but in the second semester, incorrect answers counted 90% as much as correct answers.

The instructor used a version of the standard peer instruction technique.  Students were not asked to vote on clicker questions independently, but were instead asked to discuss the questions in pairs prior to voting.  Random samples of students in each semester were audio-recorded during these pair discussions throughout the two semesters.  The audio-recordings were analyzed in two different ways to measure “discourse bias,” “the difference between the fractional contributions to a conversation between partners.”  For instance, if one partner contributed 70% of the time and the other contributed 30% of the time, then the pair’s discourse bias would be 40%.

First, each idea shared by the students during the discussions was coded according to ten categories, including categories such as restating question elements, stating answer preferences, and providing justifications.  (One side finding was that there was no correlation between “type” of clicker question and the nature of the ideas shared by students during the discussion.)  Second, the total number of words produced by each student during the discussions was counted.  Both techniques provided measured of each student’s contribution to the discussions.

The results strongly indicated that the low-stakes grading scheme encouraged more balanced participation by students during pair discussions.  For example, when using the first measure of discourse bias (counting ideas), the average bias for the high-stakes class was 33.2%, whereas the average bias for the low-stakes class was 19.5%.  That is to say that each pair of students engaged in a conversation more dominated by one of the students in the high-stakes class.  The second measure of discourse bias (counting words) provided similar results-an average bias of 39.8% in the high-stakes class and 26.6% in the low-stakes class.

The authors also note that the low-stakes grading scheme promoted more independent student responses to clicker questions following pair discussions.  In the high-stakes class, only 7.6% of the time did two partners submit different answers to clicker questions, whereas in the low-stakes class, this occurred 17.1% of the time.  The authors conclude from this that in the high-stakes class, students’ concern for earning points motivated them to submit their partners’ answers to clicker questions even when they didn’t really believe those answers.

Comments: This study improves on James’ earlier study and provides persuasive evidence that low-stakes grading schemes for clicker questions promotes more meaningful student participation in small group discussions prior to voting.  True, this wasn’t a double-blind, randomized control group experiment (in which students were randomly assigned to the two grading schemes and the instructor didn’t know which grading scheme would be used with each group of students), but such experiments are practically impossible to implement in educational settings.  Short of that “gold standard,” this is a very well-designed and persuasive study, in part because many of the possibly confounding variables in the earlier study were eliminated and in part because of the use of direct, qualitative measures of student participation.

Willoughby and Gustafson (2009) conducted a similar study in physics courses, audio-recording student discussions in some sections and not in others.  They found that students in the sections that were not audio-recorded “block-voted” more when high-stakes grading schemes were used for clicker questions and less when low-stakes grading schemes were used.  In the sections where audio-recorders were used, there was no statistically significant difference in block-voting rates.  They concluded that the presence of the audio-recorders might have influenced student voting behaviors (an example of the Hawthorne effect).  If true, it’s possible that the difference James, Barbieri, and Garcia found in block-voting behavior might have been even greater had audio-recorders not been used-all the more reason to use low-stakes grading schemes when using clickers for formative assessment.

I would have liked to have seen a little additional information provided in the article about the use of clicker questions in these courses.  Were students asked to vote on their own before discussion the clicker questions in pairs?  (I don’t think they were, but this isn’t stated in the article.)  Also, what instructions were given to students prior to the peer discussion times?  I’ve seen some evidence (Lucas, 2009) and heard some advice (from Doug Duncan) that the instructions given to students prior to peer instruction can affect the quality of the discussions.  And while it was found here that the type of clicker question did not correlate with the kinds of ideas shared during peer instruction, it would have been informative to know what kinds of clicker questions were used.

What’s not directly address in this article, however, is the assumption that more student participation during small-group discussion of clicker questions leads to greater student learning.  This was an issue I raised in my comments on James’ earlier study, which included data on student performance on final exams.  In response to that study I asked if greater class participation led to greater student learning or if students who knew the material better simply dominated class discussions.  While it’s possible that the latter is true, evidence from non-clicker studies strongly suggests that more active participation in class discussions leads to greater student learning.  I wish this assumption (that participation leads to student learning) had been stated as such in the article.

The takeaway here is that low-stakes grading schemes for clicker questions leads to greater student participation and clicker questions results that more accurately reflect students’ actual understanding (or lack of understanding).  These results have important implications for instructors using clickers to motivate student participation and inform agile teaching choices.