If you’re heading to the Turning Technologies User Conference at Harvard University in October or the Lilly Conference at Miami University in November, you’ll see me there! I’m delivering keynote talks at both conferences. Abstracts can be found below.

Connecting with Participatory Culture: Clickers and Deep Learning

Turning Technologies User Conference
Harvard University
October 10, 2010

Today’s students vote for their favorite contestants on American Idol, “like” a friend’s wall post on Facebook, comment on news and events on Twitter, and engage in robust online discussions about World of Warcraft. We live in a participatory culture, one in which voting, commenting, creating, and sharing are the norm and people prefer being contributors to being consumers. Teaching with clickers is one way to tap into this culture, engaging students in ways that motivate them to participate during class in meaningful ways. In this talk, Derek Bruff will explore ways that using clickers connects with our students’ participatory culture and how those connections can be leveraged to promote deep learning.

Clickers and Backchannel: Engaging Students with Classroom Response Systems

Lilly Conference on College Teaching
Miami University of Ohio
November 18, 2010

Classroom response systems offer instructors useful options for encouraging student participation and engagement during class. For instance, “clickers” enable instructors to rapidly collect and analyze student responses to multiple-choice questions. Backchannel conversation tools can help more students share questions and ideas with instructors during class and provide new venues for in-class discussion among students. In this session, we’ll explore the kinds of activities and questions that make the most of these systems, including ways to foster small-group and class-wide discussion, turn quizzes into learning experiences for students, practice more “agile” teaching, and make class time more enjoyable.

Image: “Suitcase” by Flickr user EssjayNZ, Creative Commons licensed

I received an email last week from Bill Goffe who teaches economics at SUNY-Oswego (and contributed to this great guide to teaching economics with clickers) with a neat tip for practicing agile teaching. He noted that he’s heard Harvard physics professor Eric Mazur talk about bringing to class a folder full of transparencies, each with a different clicker question. Mazur asks his students the clicker question at the top of the stack and, depending on how well the students do, either moves on to the next question or skips a few to move to a question on a different topic. This is a great way to practice agile teaching by basing the selection of a clicker question on the results of a previous one.

Bill wrote that it can be challenging to take a similar approach if one’s clicker questions are embedded in PowerPoint slides. Breaking out of presentation mode, wandering through one’s slides in “Normal” or “Slide Sorter” mode to find one’s next question, then switching back to presentation mode to display the question–that seems like an awkward process, particularly if it’s visible to the students. However, Bill found a better way:

Yesterday I came across [the Inside Higher Ed article] “The Advantages of a 2,500 Slide PowerPoint Deck,” and it had the solution: number your slides and bring a printed version of the slides in outline view. Look at the latter, find the desired slide, type that number, and then PP takes you to that slide, still in display mode. It might seem like a small point, but it would make class much smoother.

I had no idea that you could just type a slide number while in presentation mode in PowerPoint and instantly go to that slide. (Try it, it really works!) That’s a handy trick for going nonlinear in PowerPoint. It might require you to have a printout of your slides handy so you can determine the number of your next slide, but as the comments on that article point out, if you use the same slide deck over and over, you’ll probably start to memorize some of those numbers. (The presenter described in the article uses the same 2,500 slide PowerPoint deck for all of his presentations. He just skips around nonlinearly in response to questions from the audience!)

This trick is probably better than the one that came to mind when I first read Bill’s email, which is to switch from the “slides” view in PowerPoint to the “outline” view. The outline view gives a more compact view of your slide deck, and, depending on how you’ve formatted your clicker questions, can show you your clicker questions particularly well. I think the type-a-slide-number trick is even slicker.

Another way to go nonlinear is to use Prezi. In Prezi, you can organize all your content (text, images, clicker questions, whatever) visually on a great big canvas. This means that finding a particular bit of content is pretty easy, assuming you’ve placed it on the canvas in a sensible location. And while you can set up a “path” in Prezi to follow somewhat linearly, you can always go “off path” and zoom around to other content at will. I’ve used Prezi for the visuals in a few presentations that also included clicker questions, such as this talk at the University of Louisville and this one at Central Michigan University. In both cases, I simply embedded my clicker questions in the Prezi (in a particularly clever way in the Louisville talk) and ran my clicker software on top of Prezi. Worked like a charm.

(See how I turned the entire Prezi into one big clicker question? The letters A, B, C, and D weren’t visible until near the end of the presentation when I zoomed out and posed my final clicker question.)

Of course, in those talks, I was mostly moving linearly through the Prezi. I can see, however, setting up a Prezi where your clicker questions are organized visually in groups and subgroups and using that to go nonlinear during a class session. It’s not quite as slick as typing a number and instantly moving to a different slide, but I would guess that some of us would be faster at navigating visually to a new question than remembering or looking up a question number.

Finally, one of the comments on the Inside Higher Ed article that Bill sent me links to a blog post describing a “Choose Your Own Adventure” session on information literacy designed and facilitated by librarians at the University of Dubuque. PowerPoint hyperlinks (yet another way to move nonlinearly through a slide deck) were used with clicker questions to have the students determine the progression through the slide deck as they grappled with information literacy tasks like finding and evaluating the quality of sources.

I’ve been eager to find more examples of this kind of classroom response system use since I first read about it in the David Banks book on response systems. That edited volume includes a chapter (Hinde & Hunt, 2006) on the use of a “Choose Your Own Adventure” style question tree in an economics course. For more on the Dubuque library use of CYOA / question trees, see this follow-up blog post and the PowerPoint deck itself.

Thanks for sharing, Bill!

Image: “Choose Your Own Adventure 1” by Flickr user Jason Permenter, Creative Commons licensed

Earlier this week, I gave a virtual presentation at the Muskegon Community College Math and Technology Workshop organized by Maria Andersen. The participants were all math instructors spending the week at MCC learning from Maria and others about various uses for educational technology in math instruction.

I’ve blogged often about teaching math with clickers here, but I don’t think I’ve shared slides from any of my presentations on this topic. Since Maria asked me to put my slides on Slideshare for the workshop participants, I thought I would share them here.

Back in May 2010, I led a webinar on teaching with clickers as part of the CIRTLcast series for the Center for the Integration of Research, Teaching, and Learning (CIRTL), an NSF-sponsored network of six universities interested in preparing future science, engineering, and mathematics faculty. The full webinar was 60 minutes, and you can access the audio recording and my slides in the CIRTLcast archive. However, CIRTL has done a great job taking some excerpts from the session and packaging them as a 10-minute YouTube video, complete with a transcript!

In the video, you’ll hear me talk about using clickers to generate small-group and classwide discussion, create “times for telling,” encourage metacognition, facilitate peer assessment, structure class time, turn quizzes into learning experiences, and make class more fun. Clickers can be used very effectively to engage students in the learning process during class, and this short video is a nice introduction to these uses of clickers.

Thanks to CIRTL for giving me the opportunity to present this webinar and for putting together this great video!

There’s a lively discussion happening on the POD Network listserv this week on teaching large classes. The discussion detoured into a discussion of teaching with clickers. In responding to one of these posts, Louis Schmier wrote:

“Well, Ron, clickers might get feedback and active and collaborative involvement, but learning? Technology is a tool, not a panecea. The basic problem with large class as Ron defines it, is that it violates the basic Aristotelian tenet of KNOWING those in your audience and tailoring yourself so that those in the audience get it, understand it, and retain it.”

This comment struck me as interesting, so I responded to it on the listserv. I’m including my response here on the blog (with a couple of extra links for clarity), in case those not on the listserv find it helpful.

Anton [Tolman] has responded very eloquently to Louis’ concerns about classroom response systems, but I can’t resist weighing in myself. First, there’s plenty of evidence that “active and collaborative involvement” often leads to student learning, so if clickers are indeed fostering more student engagement during class, that sets the stage for more student learning.

And as for the idea of “KNOWING those in your audience and tailoring yourself so that those in the audience get it, understand it, and retain it,” once you get past 15-20 students, it becomes very difficult to do those two things—assessing your students’ learning during class and practicing “agile teaching” by responding to what you find out about their learning on the fly–without a tool like a classroom response system. In fact, these are two teaching tasks that clickers are incredibly well-suited to support.

Imagine you have a single student in your office asking for help in your course. It’s relatively easy to “diagnose” that student and get a sense of what the student understands and doesn’t understand, then to tailor some one-on-one instruction to help the student resolve his or her misunderstandings. If you have 2-3 students in an office hour setting, you can probably do the same thing, although you’re already juggling 2-3 different “private universes” at this point, which can be challenging.

When you move to the “small” class setting, say, 8-10 students, you now have 8-10 “private universes” to try to uncover and respond to. Sure, there could be some similarities among your students in terms of their prior experiences, misunderstandings, and perspectives on course material, but you’ve still got 8-10 different students to build your learning environment in response to. Given 50 or 75 minutes and plenty of discussion, you’ve got a good shot at this, however.

Now move to a bigger class, say 15-20 students. At this point, it’s tough to have enough “air time” for all the students during class. This makes the juggling of “private universes” very challenging. Small group discussions can help (outsourcing some of this work to the students themselves), as can pre-class assignments. But during class, you’ve got quite a task if you want to be responsive to all your students’ various learning needs.

(Here you have my answer to Jeanette [McDonald]‘s question. When is large large? I would say 15-20 students. At that point the dynamics shift in very significant ways.)

Now imagine more students—30, 50, 100, or 500. The challenge of responding to that many unique “private universes” is truly daunting! You have to start making some assumptions about commonalities among those private universes. Clickers are wonderful tools for getting a sense of the validity of those assumptions! You pose a multiple-choice question where the answers are crafted to tie into what you suppose are common understandings (correct or not) and perceptions about the topic at hand, you have the students think about (and maybe discuss in small groups) the question, then you poll them and find out which of the understandings and perceptions are *really* the most common.

The resulting bar chart tells you how to spend the next 5-20 minutes of class time: responding to the student views of the topic that are most common. This “agile teaching” allows you to make the best use of limited class time by responding to as many “private universes” as you can in the time available.

Some caveats: You could miss a very common student understanding or perspective completely when you write your clicker question! The more experience you have with the topic and with students learning about the topic, the less likely this is to happen, but it’s still something to watch out for. That’s why it’s helpful to have some kind of classwide discussion about the question, giving students whose views aren’t represented in the bar chart a chance to share.

You also won’t get to the “long tail” of student views this way. What about the two students in a class of 100 who voted for option D? Will you have time to address that minority view? Maybe not during class, but perhaps after class in some fashion.

I’m also assuming here that you’re teaching a large class! The debate over whether or not classes should have 100 students is secondary to my point here. My point is that *given* a large class, a classroom response system is an excellent tool for understanding one’s students (in the aggregate) and tailoring one’s instruction to one’s students.

Lots more on these ideas (with examples from real classrooms!) in the “agile teaching” category on my blog.

The discussion on the listerv continued productively from here. It’s worth checking out.

Image: “O is for Occipital Lobe” by Flickr user illuminaut / Creative Commons licensed

Jeffrey R. Young's Chronicle article, "Reaching the Last Technology Holdouts at the Front of the Classroom," has apparently struck a nerve among professors, particularly those who are critical of educational technology. As I write this, the article has 59 comments on the Chronicle site, which is far more than most articles receive. Even the graph accompanying the article has received 13 comments!

Since clickers are mentioned in the article and in many of the comments, I thought I would weigh in here on the blog...

First, it's worth noting that Chris Dede, the Harvard University learning technologies professor interviewed for the article, doesn't make the argument that professors who don't use technology are shirking their duties. Several of those who left comments seem to think so, however. For example, here's comment #33 from Emily in NY:

"Dede does nothing in this article but set up a false dichotomy between professors committed to outdated, boring and irrelevant teaching methods and those eagerly embracing the modern technologies that contemporary students crave."

Here's the closest Dede comes to that argument, in the National Educational Technology Plan he helped draft for the US Department of Education in March:

"The challenge for our education system is to leverage the learning sciences and modern technology to create engaging, relevant, and personalized learning experiences for all learners that mirror students' daily lives and the reality of their futures."

Dede's arguments in the Chronicle argument are focused on motivating professors to tap into the latest research on learning and continue to improve their teaching practices over time. From the report he drafted, it's clear he thinks that technology can help with that, but he doesn't seem to be making the argument that professors who don't use technology are irresponsible, just those who stick with the same teaching methods you'd find in a classroom circa 1900. Sure, technology can be a big part of change, but many of the teaching innovations mentioned in the article (such as David Pace's work on enhancing history teaching) don't involve any technology.

Speaking of false dichotomies, however, here's one from comment #18 by user "tee_bee":

"What matters is that students learn--and a skilled teacher with a blackboard is still going to do a far better job than a bozo with some clickers and powerpoint slides."

True, a skilled teacher is going to do a better job than a bozo any day, regardless of technology. But comparing a skilled teacher to a bozo isn't really important here. Might technology (including clickers) help a skilled teacher be even more effective? Yes, that happens. And might technology help a relatively novice teacher become more effective? Yes, that happens, too. Those are the kinds of changes in teaching that are worth thinking about and encouraging, and I think that's a point that Chris Dede would agree with.

How might teaching with clickers help a good teacher be even more effective? Several comments on the Chronicle article were skeptical of clickers' potential for doing this. For example, here's what user "ikant" said in comment #21:

"I'm young, tech-savvy, and pretty unconvinced by this article. I can't speak for all fields, of course, but I'm pretty skeptical that good class discussions and quality writing in the humanities are particularly improved by clickers etc... the heart of what I do is in trying to educe questions, critical thought and excitement about books which students might previously have thought were utterly irrelevant to them, and (my evaluations indicate that) I do this very well with no particular technological bells and whistles in the classroom. Am I missing something?"

I'm glad that this instructor is capable of leading effective class discussions, foster critical thinking, and increase student motivation in the classroom. Let me clear: Doing so is entirely possible without clickers! However, not all instructors are as skilled as "ikant" appears to be and even for instructors like "ikant," it's possible that clickers would enhance an already productive classroom environment. Some examples from past blog posts:

• Case Study: Using clickers to promote critical thinking in text-based, political science courses
• Case Study: Using clickers to teach ethics in a history class
• My Thoughts: Using clickers to foster creativity and collaboration
• My Thoughts: Using clickers to teach critical thinking in the humanities

Here's a similar comment (#26 on the Chronicle site) from user "csgirl":

"The reason I don't use blogs and clickers is that they simply are not appropriate to the material I teach. Clickers in particular are useless to me - I care about the strategies my students are using to solve problems, not whether they can click the right answer in a quiz."

This is a common misconception about clickers, that they're just good for quizzing students basic conceptual understanding and recall. Here's another formulation of it, from user "chewy18" in comment #53:

"They might work well for understanding basic concepts or in preparation for recognition/recall examinations where the test question is a line long and the answer a word or two in length. What about those of us who teach upper division courses where we struggle with students who have not, until they reach senior status, even been exposed to the analytical reasoning process. Suddenly they discover that life is, after all, not a multiple choice test and developing an argument that could go either way, is a requirement. How does that appeal to the clicker technology?"

Sure, clickers work well for assessing basic conceptual understanding and factual recall, but they're useful for teaching at the higher levels of Bloom's Taxonomy, too. Here are some more examples from past blog posts that demonstrate this:

• Case Study: Using clickers to teach difficult material in upper-division physics
• Case Study: Using clickers to teach students about plagiarism in library instructional settings
• Case Study: Using clicker questions without single correct answers in mathematics
• Case Study: Using clickers to teach nursing students to make evidence-based decisions

And for "csgirl," here's a great collection of resources on using clickers and peer instruction in computer science from Daniel Zingaro.

Finally, you can imagine how this comment (#37) from user "fizmath" made me feel:

"The teacher/physician analogy is lousy. We have real data to show that new medical tech benefits patients. You can't say the same about blogs, videoconferencing and those stupid clickers."

(This is a response to Chris Dede's analogy that teachers who don't update their teaching methods over time are akin to physicians who don't update their medical practices over time.)

Want some research? Try these studies, all of which are well designed and support the claim that clickers used in appropriate ways enhance student learning:

• Stowell & Nelson (2007) - Clickers provided instructors with more accurate assessment of student learning during class than other response methods, including a show of hands.
• Yourstone, Kraye, & Albaum (2008) - The use of clickers for end-of-class quizzes improved student exam scores by four points over the use of pencil-and-paper quizzes discussed the next day in class, likely because of the immediate feedback clickers provided to students on their learning.
• Hoesktra (2008) - Clickers helped students be more attentive during class (since they know clicker questions could be asked at any time) and participate in more meaningful ways (both before votes are submitted and after results are displayed).
• Smith et al. (2009) - Students actually learned from each other when discussing clicker questions in pairs prior to voting. They don't "simply choose the answer most strongly supported by neighbors they perceive to be knowledgeable."
• Mayer et al. (2009) - Clickers made it easier for instructors to ask their students questions during class and for students to respond to those questions, leading to improved student learning through better class discussions.

My summary for those skeptical of using clickers in the classroom: Read the literature, find out how those in your discipline are using clickers effectively, and see (preferably by experimentation) if those methods might help you to enhance your teaching, regardless of how effective you are currently as a teacher. If a classroom response system doesn't help you do your job better, then don't use one. They're not for everyone. However, don't write clickers off without first investigating their potential. They're far more useful and versatile that you might think at first.

Image: "Innovation" by Flickr user thinkpublic, Creative Commons licensed

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Using Prediction and Classroom Voting via Clickers to Address Students’ Overreliance on the Representativeness Heuristic,” Tami Dashley, University of Texas-El Paso [Slides]

Tami Dashley is a graduate student in math education and a student of Kien Lim, one of the organizers of the contributed paper session. She shared some of her thesis research, an investigation into the connection between classroom voting with clickers and certain misconceptions students have about probability. Her work focuses on the representativeness heuristic, which she defines as “determining the likelihood for events based on how well an outcome represents some aspect of its parent population.”

Tami gave the following example: Suppose you toss a coin six times, getting a sequence of heads (H) and tails (T). Which of the following is more likely to occur: TTHHTH or HTTHHH? Someone using the representativeness heuristic would say that TTHHTH is more likely to occur since it includes an equal amount of heads and tails, just like the coin does. The other option includes more heads than tails, so it would not seem as likely to someone using the representativeness heuristic. Actually, both of those outcomes are equally likely (each occurring with probability 1/64), so the representative heuristic is a misleading one in this example.

The issue is that the representativeness heuristic is useful in some cases, but not useful in all cases. The misconception that many students have is that it’s always useful.

How to help students stop over-relying on the representativeness heuristic? Tami has been investigating the use of prediction questions, ones that ask students to predict an outcome or probability without actually computing anything. For example, students might be asked to determine which of several outcomes is most likely to occur. Since students need not be as precise when responding to prediction questions, they have some cognitive processing power freed up to focus on concepts. Clicker questions are a natural match here, since they allow students to commit to their predictions and compare their predictions to those of their peers. Then discussion of the incorrect answer choices provides an opportunity to deal with misconceptions.

Tami conducted her research in a high school setting, using three groups of students. Her “control” group received a lesson exploring the representativeness heuristic that didn’t ask the students to predict any probabilities. A second group was asked several prediction questions but didn’t use clickers to respond to the questions. The third group used clickers to respond to prediction questions during the lesson. Tami used pre- and post-tests to determined the efficacy of these three different lessons.

Tami found that her “control” group did pretty well on the post-test compared to the two experimental groups. However, most of their success came from what she called a “learned response.” In this case, many of the students picked up on the fact that “all of the above outcomes are equally likely” is often the correct answer to questions exploring the representativeness heuristic. (These are what students might call trick questions!) When Tami looked at performance on questions where “all of the above outcomes are equally likely” was, in fact, not the correct answer, the prediction-with-voting group performed better than the control and prediction-only groups.

I was very impressed with Tami’s research design and the subtlety with which she explored student misconceptions in this teaching context. I don’t believe that Tami has published this work yet, but I look forward to reading it when she does.

Image: “Heads and Tails” by Flickr user canonsnapper, Creative Commons licensed

The summer meeting of the American Association of Physics Teachers (AAPT) wraps up in Portland, Oregon, today. There were several talks on teaching physics with clickers at the meeting, including one by Ian Beatty of the University of North Carolina at Greensboro physics education research group. Ian was the subject of my first podcast interview, and he’s been doing great work helping science instructors at the K12 and post-secondary levels teach effectively with clickers.

In Ian’s presentation, he identified and addressed several common concerns instructors express about teaching with clickers. For each concern, Ian identifies a belief about teaching and/or learning that likely underlies the concern, as well as an alternate belief that can be adopted to address the concern productively. Ian also includes some practical strategies and example clicker questions for each of these alternate beliefs.

For example, when many instructors hear about teaching with clickers, they’re concerned with having sufficient class time to cover what they need to cover in their courses given the time required by having students discuss and respond to clicker questions. Ian notes that this concern is likely a result of the following belief: “I must explicitly address in class everything students will be held accountable for.” Ian then presents an alternate perspective on this idea: “I can use class time to focus on core ideas and big-picture understanding, and charge students with filling in the details outside class.” This alternate perspective is, perhaps, non-intuitive to many instructors, but it’s a reasonable and useful perspective to have. Adopting this perspective leads to a shift from what Ian calls an understanding of class as a place to present content to an understanding of class as a place to help students digest content. Ian then shares five tips and techniques for implementing this shift in the classroom.

Ian addresses other concerns in a similar manner, including concerns about having enough time to write good clicker questions, concerns about poor student participation during class, and concerns about changing one’s teaching style. His visuals, which use the online presentation tool Prezi, are included below and are well worth checking out.

The Chronicle of Higher Education recently reported (briefly) on a new survey from CDW, a “leading provider of technology products and services for business, government, and education,” indicating some differences in how faculty and IT staff view the role of technology in higher education. Here’s what caught my eye from the Chronicle story:

“The most popular tools cited by professors were e-textbooks and online documents, with faculty members reporting far less enthusiasm for other electronic tools. Under a quarter of faculty members surveyed use wikis or blogs in their teaching…”

While I’m an active user of blogs in my courses and I see a lot of value in wikis for student collaboration, the bit about e-textbooks and online documents doesn’t interest me that much. Those two technologies are more about content delivery than interaction. Sure, they have their uses, but they’re not as likely to lead to active learning experiences as more interactive tools.

What bugs me about this Chronicle story is that there’s no mention of classroom response systems. I mentioned this on Twitter, and a couple of people there poked fun at my tweet about this omission. Sure, I’m going to notice whether or not clickers are included in a survey like this. I did write a book about teaching with clickers, after all. However, it’s not that clickers weren’t addressed in the survey. In fact, they were listed right along with many other educational technologies as response options in the survey itself, and the publicly available report from CDW notes that 34% of institutions support the use of clickers by faculty. (More on that statistic below.)

I think what bothers me is that clickers rarely seem to rate a mention in stories like the Chronicle‘s on educational technology. Sure, e-book readers are all the rage these days and there are plenty of people in academia talking about the potential of e-textbooks. Blogs and wikis get a lot of attention, too, which is great since they are useful tools for fostering out-of-class interactions among students. But what about technologies that enhance the in-class experience for students? Yeah, I know I’m biased, but those are the technologies that I see as having the greatest potential to have a positive impact on higher ed. Why? Because what happens during class still looks a lot like it did 20, 50, or even 100 years ago. There’s great potential for growth there, and in-class, interactive technologies like classroom response systems can be a big part of that.

Back to the survey: Only 34% of IT professionals surveyed indicated that they support faculty use of clickers? That seems low to me, given that it seems that every campus I hear about has at least a couple of faculty members teaching with clickers. Perhaps at many of those places, that’s all there is: a couple of faculty members using clickers without any formal IT support. That would explain the 34% statistic.

This bothers me, too, particularly when, according to the CDW survey, 59% of IT professionals consider lecture capture technologies “essential” to the 21st century classroom. That’s just more content delivery. It doesn’t do much to increase student engagement and interaction. Yes, it’s true that students who know they can watch a lecture after class might take fewer notes and have more mental bandwith for paying attention and engaging during class. And lecture capture tools that allow students to collaboratively mark-up and share lectures after class have a lot of potential for outside-of-class interaction and learning. But why not put some more support behind a technology like clickers that’s designed to support formative assessment and student engagement during class?

Image: “Overhead Projectors at US Grant High School in Oklahoma City” by Flickr user Wesley Fryer / Creative Commons licensed. I’ve been wanting to use this image for a while now. What we consider an essential classroom technology one year can be a recycling challenge the next!

Over on the Vanderbilt University Center for Teaching blog, my Vanderbilt colleague Isabel Gauthier, professor of psychology, has shared her experiences asking her students to write their own clicker questions. I met with Isabel a few years ago and briefly discussed ways to use clickers in her courses, and she’s really taken the technology (and pedagogy) and run with it. She’s got a great handle on how to have students write their own clicker questions, and I’ve been wanting to share her experiences here on the blog for a while. Here’s her article, in her own words:

It is difficult to write meaningful and discriminative multiple-choice questions that students find clear and fair. Years ago, I met with CFT assistant director Derek Bruff, who gave me useful pointers to perfect this skill. But a side effect of this interaction transformed entirely the way I teach: I learned so much by working on writing better questions, surely my students could learn too! Derek said something like, “You know, some teachers ask their students to generate questions…” This idea took me on a path to use this strategy, cautiously at first, and then more boldly, as the central pedagogical and evaluative strategy in some of my courses, including Brain Damage and Cognition and Principles of Experimental Design.

I teach these courses three days a week. On two of these days each week, I lecture on course material. These lectures are informed by questions about the readings posted online by students and issues that emerge from a hands-on, semester long project I assign my students. On the third day, we use clickers to go through student-generated multiple choice questions.

Each week each student is responsible for turning in a single question on the weeks’ readings. Students use a PowerPoint template to submit their questions which facilitates use of the question in my clicker software, TurningPoint. In the notes area of the slide, each student includes their name, the correct answer, the page(s) that inspired the question, and, optionally, a justification for the correct answer. Before class, I concatenate all the questions in a single file and read them, grading each on a scale of 1 to 5. The grade goes in the notes area, and, in a textbox on the slide, I write comments about the question. This allows me to print the slides as a PDF with student names removed so that all questions and comments can be distributed to students. I then reorder the slides to choose the right mix of questions I want to use in class with the clickers.

This provides me in a single step with my preparation for the next class, an idea of what I need to focus on during my lecture days, an evaluation of each student, and a mechanism for providing students with feedback on their learning. This weekly feedback allows students to realize how difficult it is to write a good question, one that raises an important issue clearly and is appropriately challenging for their peers. Students eventually learn to key in on critical concepts and relationships in the readings and sometimes even go beyond the readings in interesting ways. They take a more active part in their own and their peers’ learning, and their questions keep me focused on what is most challenging for these students at each point in the course.

Each week students answer the best of these questions in class using clickers, accumulating points for their answers using a generous but motivating grading scheme. If there’s controversy over the correct answer to a question, the class can decide to eliminate a question or to accept multiple answers as correct, provoking interesting discussions. As needed, I can lecture for a few minutes, but issues are generally clarified in class discussion. Questions are used anonymously in class, but students want their question to be picked and use wit and humor to this effect, making the experience more enjoyable for everyone.

This method completely replaces any exams I used to give: They are no longer needed since my students now share the responsibility to evaluate their own learning throughout the semester.

Isabel and her use of clickers were featured on Nashville’s NewsChannel 5 last year. Here’s the video clip: