Resources for engaging and assessing students with clickers
As I mentioned earlier this month, I was invited to deliver a keynote at the 3rd ESTICT (Engaging Students Through In-Class Technologies) conference hosted by the University of Bath on November 17th. ESTICT is a somewhat informal group of educational technologists, and they didn’t have the funds to fly me to Bath, England, to deliver the keynote in person. Instead, I agreed to do a virtual keynote. While I’ve done webinars before, this was my first time keynoting from afar, and I thought I’d reflect a little on the experience here.
One challenge was that the keynote was scheduled for 9:30 am local time or 3:30 in the morning here in Nashville. Since I worried about being coherent at 3 in the morning, I opted to record something to be played for the conference participants that morning while I was fast asleep in my bed in Nashville. Using the screen capture and webcam capabilities of Camtasia Studio, I recorded myself presenting a set of PowerPoint slides. This was the first time I had used Camtasia, but I found it remarkably easy to use for this purpose. Since I ran long during my recording, I even got to try out the editing tools in Camtasia to take out a few minutes here and there. It worked like a charm.
Since I worried that 45 minutes of watching my webcam footage might be a little dull for an opening keynote, I suggested that we use Google Moderator to add some interaction. Instead of recording one long video, I recorded two shorter ones. The first video was about 22 minutes and introduced participants to the use of clickers to encourage deep learning. At the end of that video, I instructed participants to work in small groups to generate questions about teaching with clickers. Each group needed to have a “scribe” who could use Google Moderator to submit those questions online and vote on their colleagues’ questions, as well. (Given the edtech focus of this conference, I felt confident that there would be enough people in the room with laptops and iPads and smart phones to pull this off, and I was right!) I promised to respond to the most popular questions later in the day (1:30 pm Bath / 7:30 am Nashville) during a live, virtual Q&A session.
In the second video (this one about 14 minutes long) I broadened my use of the term “classroom response system” to include various backchannel tools. I also made some connections between the use of classroom response systems and some of the research on student motivation I’ve been exploring recently. After sending the ESTICT crew (Nitin Parmar and Sian Lindsey) links to the videos, I was done with my prep work.
Knowing that (a) I had to get up by 6 to prepare for the live Q&A session and (b) there were conference participants watching my keynote videos at 3:30 in the morning made for a tough night’s sleep! I actually tossed and turned a lot and, at one point, dreamed that the Google Moderator session was devoid of questions when I woke up! However, that wasn’t the case. As soon as I got up, I checked the Twitter stream for the conference (which used the hashtag #estict) and found that the conference participants were generally pleased with my virtual and pre-recorded keynote! The Google Moderator activity seemed to have gone over particularly well.
I replied to a few tweets and quickly checked the results of the Google Moderator activity, then headed into work for a good Internet connection for the live Q&A. We used Adobe Connect for this, and it went well. We had webcams working at either end, and I was able to pull up the Google Moderator results within Adobe Connect and respond to the most popular questions. I gather that the folks in Bath could hear me just fine, but the audio didn’t work quite as well going the other way, mainly because they had me on speakers in the lecture hall there. Other than that, however, the live Q&A seemed to go well.
A few reflections on the experience…
Thanks so much to Nitin and Sian and the other conference organizers for inviting me to speak at ESTICT and for handling so much of the tech support before and during the conference. They did a great job and made things very easy for me. And they’re doing great work in connecting educational technologists in the UK around in-class technologies! I encourage you to join their Ning group to stay up-to-date on their activities.
Image: “.8 Apple-esque,” Johan Larsson, Flickr (CC)
I’m attending my very first Lilly Conference this weekend, and I’m pretty excited to be here. I’ve heard so many good things about the Lilly Conferences, so I’m glad to get the chance to experience one firsthand.
I thought I’d post some resources from my keynote last night. First, here’s the Prezi I used during my presentation. As is often the case when I use Prezi for a talk, I had a half dozen people come up to me after the talk and ask about it. I continue to find it an incredibly useful tool for showing the relationships between ideas and convey the “big picture” of what I’m discussing.
And here are some links to further resources about concepts, techniques, and tools I mentioned in my talk:
My latest article is now available in the POD Network Essays in Teaching Excellence series! The article is titled “Multiple-Choice Questions You Wouldn’t Put on a Test: Promoting Deep Learning Using Clickers,” [PDF] which is by far my best article title yet.
In the article, I describe four types of clicker questions that are useful for encouraging students to engage in meaningful ways with course material: one-best-answer questions, student perspective questions, misconception questions, and peer assessment questions. None of these would make much sense as multiple-choice exam questions, but as clicker questions designed to motivate and frame discussions, they work very well.
I wrote the article with clicker skeptics in mind, particularly those who believe that multiple-choice questions are of limited value. Many instructors see them as useful for assessing factual recall but not useful for much more than that. This belief is usually based on an instructor’s experience with multiple-choice test questions. However, multiple-choice clicker questions are used in the classroom, not on exams. As a result, they can play very different roles in student learning.
So if you have a colleague who hasn’t taken the plunge and tried teaching with clickers yet, I hope this is an article you can share!
Image: “classroom” by Flickr user velkr0, Creative Commons licensed.
I meant to blog about this sooner, but I’ll console myself with the tweets I sent out about it earlier. Stephanie Chasteen (@sciencegeekgirl on Twitter) is facilitating a webinar on teaching with clickers Tuesday, September 28th. I’ve linked to Stephanie’s work here on the blog several times in the past on such topics as using clickers in upper-division physics, prediction questions in lecture demos, advice on writing clicker questions, and clickers sessions at the recent AAPT meeting. I imagine her webinar will be top notch! Here’s the description:
In this interactive webinar, we’ll explore tips and ideas for incorporating clickers into your particular class. Clickers offer a powerful way to increasing student engagement and improve learning. At the University of Colorado, we have transformed our classrooms by using clickers to promote peer instruction. We’ll show research results on the most effective use of clickers, and discuss common challenges. In particular, we’ll focus on the attributes of “great” clicker questions, discuss example questions, and share ideas on facilitating effective wrap-up discussions once all the votes are in.
The webinar starts at 11 a.m. Mountain time / 12 p.m. Central. Register here.
Earlier this week, I gave a virtual presentation at the Muskegon Community College Math and Technology Workshop organized by Maria Andersen. The participants were all math instructors spending the week at MCC learning from Maria and others about various uses for educational technology in math instruction.
I’ve blogged often about teaching math with clickers here, but I don’t think I’ve shared slides from any of my presentations on this topic. Since Maria asked me to put my slides on Slideshare for the workshop participants, I thought I would share them here.
Back in May 2010, I led a webinar on teaching with clickers as part of the CIRTLcast series for the Center for the Integration of Research, Teaching, and Learning (CIRTL), an NSF-sponsored network of six universities interested in preparing future science, engineering, and mathematics faculty. The full webinar was 60 minutes, and you can access the audio recording and my slides in the CIRTLcast archive. However, CIRTL has done a great job taking some excerpts from the session and packaging them as a 10-minute YouTube video, complete with a transcript!
In the video, you’ll hear me talk about using clickers to generate small-group and classwide discussion, create “times for telling,” encourage metacognition, facilitate peer assessment, structure class time, turn quizzes into learning experiences, and make class more fun. Clickers can be used very effectively to engage students in the learning process during class, and this short video is a nice introduction to these uses of clickers.
Thanks to CIRTL for giving me the opportunity to present this webinar and for putting together this great video!
More from my round-up of articles on clickers in the health professions. A short, but interesting post today. Your comments are invited…
Reference: Zurmehly, J., & Leadingham, C. (2008). Exploring student response systems in nursing education. CIN: Computers, Informatics, Nursing, 26(5), 265-270.
Notes: This short article is another introduction to teaching with clickers, although I found it a little too prescriptive for my tastes. There’s nothing here you won’t find in other articles with one very interesting exception:
To date, there has been no evidence of hacking or compromise to the SRS systems that were evaluated. As a safeguard against tampering, a computer printout of responses can be generated and saved, to be used as a record for future references and to check for any attempted manipulation of grades.
Wow! I’ve never heard this worry before. Have you had to worry about students hacking into their clicker grades?
Image: “Me on Computer” by Flickr user Brian Lane Winfield Moore / Creative Commons licensed
More from my round-up of articles on clickers in the health professions…
Reference: Kenwright, K. (2009). Clickers in the classroom. TechTrends, 53(1), 74-77.
Notes: This short paper from Kathy Kenwright (University of Tennessee Health Science Center) serves as a concise introduction to teaching with clickers, complete with a brief review of the literature. As with Cain and Robinson (2008), the lit review isn’t comprehensive, but Kenwright does a good job of discussing the major benefits of clickers in the context of reported studies. Most of her observations are not specific to any one discipline. For example, she notes that clickers facilitate formative assessment of student learning, as well as agile teaching. She mentions the importance of the display of results of a clicker question and the use of clickers to facilitate in-class quizzes on pre-class readings.
I have concerns about a couple of Kenwright’s recommendations, however. She notes that many students in the health professions must pass national board exams, and uses this to support her claim that one shouldn’t ask too many clicker questions during class.
Asking too many questions during the lecture leaves less time to convey important content. In a curriculum such as the Clinical Laboratory Science program, there is a defined body of knowledge that must be delivered to the students.
She’s speaking of a coverage model of education here, which is problematic, as I’ve mentioned here before. I would argue that since students will be required to excel at the multiple-choice questions seen on these national board exams, they should spend plenty of class time practicing these kinds of questions. Clicker questions based on these exam questions work well for that.
Kenwright also notes that asking clicker questions “on the fly” during class can take too much class time:
If they are added during class the class will be kept waiting while the instructor is typing in the question and answer choices… There is nothing wrong with reverting to an old-fashioned show of hands, or calling on a particular student for an answer.
Asking “on the fly” questions doesn’t require you to type questions into your clicker system–asking them verbally usually does the trick. Moreover, if there was nothing wrong with a show of hands, there wouldn’t be any reason to use clickers to begin with. Why are clickers better than a show of hands? Because students don’t answer questions independently when you go with a show of hands (Stowell and Nelson, 2007).
What’s your view on the coverage issue? Is a lot of active learning possible in health professions education?
Image: “Stethoscope” by Flickr user vitualis / Creative Commons licensed
Back in January I gave a keynote talk at the Health Professionals Education Research Symposium hosted by Nova Southeastern University. Part of my preparation for that talk included reading some of the articles from related disciplines in my clickers bibliography. Shortly after the conference, I blogged about one great article about using clickers to promote critical thinking in nursing (Debourgh, 2008), and I’ve been meaning to post some notes about the other articles I read. Let’s get started…
Reference: Cain, J., & Robinson, E. (2008). A primer on audience response systems: Current applications and future considerations. American Journal of Pharmaceutical Education, 72(4), 77.
Notes: The literature review is the highlight of this article. It’s not as comprehensive as other lit reviews, but it does a great job of describing a few studies of the use of clickers in the health professions with particularly positive results. For example, Slain et al. (2004) report that students in clicker sections of two pharmacy courses scored significantly higher on exams than students in non-clickers sections. Similar results were found by Schackow et al. (2004) in classes for family medicine residents and by Pradhan, Sparano, and Ananth (2005) in classes for obstetrics and gynecology residents. These references are listed in my bibliography. Hopefully, I’ll find some time to read and blog about them soon.
Cain and Robinson also include a useful exploration of some of the logistical aspects of teaching with clickers. Instead of making recommendations, they describe the various choices a department might make and their pros and cons. They note that any clickers initiative should make sense given an institutions teaching philosophy and technology plan.
For example, a pharmacy school with a mandatory laptop program may highly value an ARS that can utilize laptops as response devices, rather than basing the decision on other features.
They also recommend purchasing a set of clickers available to faculty and staff to check out for one-shot events, like continuing education programs and faculty meetings.
The section on recommendations for future research is a strong one. Cain and Robinson write, “Any effects from using an instructional medium do not come from the use of the media itself, but from the instructional methods employed.” That’s something I’ve argued here before. Cain and Robinson call for research that explores the effects of very particular instructional strategies involving clickers, including strategies useful for facilitating discussion about matters of ethics and morality. While ethical issues are present in every discipline, they are often particularly important in professional education.
Cain and Robinson make an interesting statement in their section on student considerations: “Finally, appropriate application of the ARS in the curriculum should be defined and encouraged.” I understand the interest in encouraging instructors to use clickers in appropriate ways. It’s the “defining” piece that makes me wonder if pharmacy education is a bit more top-down than the kinds of programs you find in, say, colleges of arts and science. I find that faculty members in undergraduate liberal arts departments tend to have a high degree of autonomy when it comes to their teaching decisions. They might not be comfortable having appropriate uses of clickers “defined” for them. Am I reading too much into this word choice? Does your department (whatever your discipline) set policy on educational technology use?
Image: “Rx, San Antonio, TX” by Flickr user Tadson / Creative Commons licensed
Although relatively few instructors in the humanities use clickers, if there’s one discipline in the humanities where clickers are starting to get some traction, it would be philosophy. I interviewed a couple of philosophy faculty members for my book (including Ron McClamrock of SUNY-Albany), and I’ve recently found a few online resources for using clickers in philosophy, listed below.
Why the particular interest in clickers among philosophy instructors? Perhaps it’s because some teach courses in logic, and these courses are often more like math courses (where clickers are more mainstream) than typical humanities courses. Perhaps it’s because some philosophy instructors teach relatively large classes–larger than is typical in English and language instruction, certainly–and clickers excel in large classes. However, I suspect the primary reason clickers have been adopted in philosophy is because philosophy instructors like to ask what I call “student perspective questions” in my book. These opinion and experience questions work beautifully in ethics courses, and I imagine they work well in other philosophy courses, as well.
On the Teaching Philosophy 101 site, John Immerwahr provides an introduction to teaching with clickers in philosophy courses. He suggests a few uses of clickers that are of particular use in teaching philosophy. For instance, he suggests asking students a few opinion questions at the beginning of a unit to surface their perspectives on the topic, helping them have a great stake in the discussion that follows. He also suggests asking the same questions before and after a topic is discussed as a way to show students that “serious discussion of issues actually matters to how people think (a point which they sometimes don’t get initially).”
Immerwahr also stresses a point about clickers that is sometimes subtle: They can be used to generate “meta-conversations,” as he calls them.
Interestingly, the wording of the questions themselves often creates prompts for discussion. Student like to discuss why the class voted as it did, and people will sometimes make interesting distinctions (e.g., a student might say “If the question has said ‘can’ make a difference instead of ‘will’ make a difference, I would have voted differently,” which can then lead into another interesting discussion).
In my talks on teaching with clickers, I’ll often mention that the results display itself can generate useful discussion. Asking students why the class voted as it did can often lead to productive discussions of assumptions students make about themselves and each other.
Immerwahr’s example also reminds me of another point I often make, that the wording on clicker questions need not be as precise as the wording on exam questions. One reason is that if the question isn’t worded exactly right, an instructor can still make it work during the discussion of the question. Another is that clicker questions can be modified and asked again based on student comments during discussion. In Immerwahr’s example, for instance, the instructor could easily change “will” to “can” in the question and re-poll the students.
For an expanded version of Immerwahr’s introduction to clickers, read his Teaching Philosophy article, “Engaging the ‘Thumb Generation’ with Clickers.” The article includes more discussion of the clicker uses mentioned above, as well as other uses, and features several sample questions.
And for even more resources on using clickers in philosophy instruction, visit the Peer Instruction in the Humanities project out of Monash University in Australia. This site features a step-by-step guide to PI, advice on designing a PI lecture, a description of a sample PI lecture, examples of various types of clicker questions appropriate for this teaching context, and even a question bank organized by topic! I’m very glad to know that there’s a humanities clicker question bank out there to complement existing question banks in the sciences.Image: “Portrait of Erasmus Desiderius“, Andreas Praefcke, Wikimedia Commons