Earlier this week, I gave a virtual presentation at the Muskegon Community College Math and Technology Workshop organized by Maria Andersen. The participants were all math instructors spending the week at MCC learning from Maria and others about various uses for educational technology in math instruction.

I’ve blogged often about teaching math with clickers here, but I don’t think I’ve shared slides from any of my presentations on this topic. Since Maria asked me to put my slides on Slideshare for the workshop participants, I thought I would share them here.

Back in May 2010, I led a webinar on teaching with clickers as part of the CIRTLcast series for the Center for the Integration of Research, Teaching, and Learning (CIRTL), an NSF-sponsored network of six universities interested in preparing future science, engineering, and mathematics faculty. The full webinar was 60 minutes, and you can access the audio recording and my slides in the CIRTLcast archive. However, CIRTL has done a great job taking some excerpts from the session and packaging them as a 10-minute YouTube video, complete with a transcript!

In the video, you’ll hear me talk about using clickers to generate small-group and classwide discussion, create “times for telling,” encourage metacognition, facilitate peer assessment, structure class time, turn quizzes into learning experiences, and make class more fun. Clickers can be used very effectively to engage students in the learning process during class, and this short video is a nice introduction to these uses of clickers.

Thanks to CIRTL for giving me the opportunity to present this webinar and for putting together this great video!

More from my round-up of articles on clickers in the health professions.  A short, but interesting post today.  Your comments are invited…

Reference: Zurmehly, J., & Leadingham, C. (2008). Exploring student response systems in nursing education. CIN: Computers, Informatics, Nursing, 26(5), 265-270.

Notes: This short article is another introduction to teaching with clickers, although I found it a little too prescriptive for my tastes.  There’s nothing here you won’t find in other articles with one very interesting exception:

To date, there has been no evidence of hacking or compromise to the SRS systems that were evaluated.  As a safeguard against tampering, a computer printout of responses can be generated and saved, to be used as a record for future references and to check for any attempted manipulation of grades.

Wow!  I’ve never heard this worry before.  Have you had to worry about students hacking into their clicker grades?

Image: “Me on Computer” by Flickr user Brian Lane Winfield Moore / Creative Commons licensed

More from my round-up of articles on clickers in the health professions…

Reference: Kenwright, K. (2009). Clickers in the classroom. TechTrends, 53(1), 74-77.

Notes: This short paper from Kathy Kenwright (University of Tennessee Health Science Center) serves as a concise introduction to teaching with clickers, complete with a brief review of the literature.  As with Cain and Robinson (2008), the lit review isn’t comprehensive, but Kenwright does a good job of discussing the major benefits of clickers in the context of reported studies.  Most of her observations are not specific to any one discipline.  For example, she notes that clickers facilitate formative assessment of student learning, as well as agile teaching.  She mentions the importance of the display of results of a clicker question and the use of clickers to facilitate in-class quizzes on pre-class readings.

I have concerns about a couple of Kenwright’s recommendations, however.  She notes that many students in the health professions must pass national board exams, and uses this to support her claim that one shouldn’t ask too many clicker questions during class.

Asking too many questions during the lecture leaves less time to convey important content.  In a curriculum such as the Clinical Laboratory Science program, there is a defined body of knowledge that must be delivered to the students.

She’s speaking of a coverage model of education here, which is problematic, as I’ve mentioned here before.  I would argue that since students will be required to excel at the multiple-choice questions seen on these national board exams, they should spend plenty of class time practicing these kinds of questions.  Clicker questions based on these exam questions work well for that.

Kenwright also notes that asking clicker questions “on the fly” during class can take too much class time:

If they are added during class the class will be kept waiting while the instructor is typing in the question and answer choices… There is nothing wrong with reverting to an old-fashioned show of hands, or calling on a particular student for an answer.

Asking “on the fly” questions doesn’t require you to type questions into your clicker system–asking them verbally usually does the trick.  Moreover, if there was nothing wrong with a show of hands, there wouldn’t be any reason to use clickers to begin with.  Why are clickers better than a show of hands?  Because students don’t answer questions independently when you go with a show of hands (Stowell and Nelson, 2007).

What’s your view on the coverage issue?  Is a lot of active learning possible in health professions education?

Image: “Stethoscope” by Flickr user vitualis / Creative Commons licensed

Back in January I gave a keynote talk at the Health Professionals Education Research Symposium hosted by Nova Southeastern University.  Part of my preparation for that talk included reading some of the articles from related disciplines in my clickers bibliography.  Shortly after the conference, I blogged about one great article about using clickers to promote critical thinking in nursing (Debourgh, 2008), and I’ve been meaning to post some notes about the other articles I read.  Let’s get started…

Reference: Cain, J., & Robinson, E. (2008). A primer on audience response systems: Current applications and future considerations. American Journal of Pharmaceutical Education, 72(4), 77.

Notes: The literature review is the highlight of this article.  It’s not as comprehensive as other lit reviews, but it does a great job of describing a few studies of the use of clickers in the health professions with particularly positive results.  For example, Slain et al. (2004) report that students in clicker sections of two pharmacy courses scored significantly higher on exams than students in non-clickers sections.  Similar results were found by Schackow et al. (2004) in classes for family medicine residents and by Pradhan, Sparano, and Ananth (2005) in classes for obstetrics and gynecology residents.  These references are listed in my bibliography.  Hopefully, I’ll find some time to read and blog about them soon.

Cain and Robinson also include a useful exploration of some of the logistical aspects of teaching with clickers.  Instead of making recommendations, they describe the various choices a department might make and their pros and cons.  They note that any clickers initiative should make sense given an institutions teaching philosophy and technology plan.

For example, a pharmacy school with a mandatory laptop program may highly value an ARS that can utilize laptops as response devices, rather than basing the decision on other features.

They also recommend purchasing a set of clickers available to faculty and staff to check out for one-shot events, like continuing education programs and faculty meetings.

The section on recommendations for future research is a strong one.  Cain and Robinson write, “Any effects from using an instructional medium do not come from the use of the media itself, but from the instructional methods employed.”  That’s something I’ve argued here before.  Cain and Robinson call for research that explores the effects of very particular instructional strategies involving clickers, including strategies useful for facilitating discussion about matters of ethics and morality.  While ethical issues are present in every discipline, they are often particularly important in professional education.

Cain and Robinson make an interesting statement in their section on student considerations: “Finally, appropriate application of the ARS in the curriculum should be defined and encouraged.”  I understand the interest in encouraging instructors to use clickers in appropriate ways.  It’s the “defining” piece that makes me wonder if pharmacy education is a bit more top-down than the kinds of programs you find in, say, colleges of arts and science.  I find that faculty members in undergraduate liberal arts departments tend to have a high degree of autonomy when it comes to their teaching decisions.  They might not be comfortable having appropriate uses of clickers “defined” for them.  Am I reading too much into this word choice?  Does your department (whatever your discipline) set policy on educational technology use?

Image: “Rx, San Antonio, TX” by Flickr user Tadson / Creative Commons licensed

Although relatively few instructors in the humanities use clickers, if there’s one discipline in the humanities where clickers are starting to get some traction, it would be philosophy. I interviewed a couple of philosophy faculty members for my book (including Ron McClamrock of SUNY-Albany), and I’ve recently found a few online resources for using clickers in philosophy, listed below.

Why the particular interest in clickers among philosophy instructors? Perhaps it’s because some teach courses in logic, and these courses are often more like math courses (where clickers are more mainstream) than typical humanities courses. Perhaps it’s because some philosophy instructors teach relatively large classes–larger than is typical in English and language instruction, certainly–and clickers excel in large classes. However, I suspect the primary reason clickers have been adopted in philosophy is because philosophy instructors like to ask what I call “student perspective questions” in my book. These opinion and experience questions work beautifully in ethics courses, and I imagine they work well in other philosophy courses, as well.

On the Teaching Philosophy 101 site, John Immerwahr provides an introduction to teaching with clickers in philosophy courses.  He suggests a few uses of clickers that are of particular use in teaching philosophy.  For instance, he suggests asking students a few opinion questions at the beginning of a unit to surface their perspectives on the topic, helping them have a great stake in the discussion that follows.  He also suggests asking the same questions before and after a topic is discussed as a way to show students that “serious discussion of issues actually matters to how people think (a point which they sometimes don’t get initially).”

Immerwahr also stresses a point about clickers that is sometimes subtle: They can be used to generate “meta-conversations,” as he calls them.

Interestingly, the wording of the questions themselves often creates prompts for discussion. Student like to discuss why the class voted as it did, and people will sometimes make interesting distinctions (e.g., a student might say “If the question has said ‘can’ make a difference instead of ‘will’ make a difference, I would have voted differently,” which can then lead into another interesting discussion).

In my talks on teaching with clickers, I’ll often mention that the results display itself can generate useful discussion.  Asking students why the class voted as it did can often lead to productive discussions of assumptions students make about themselves and each other.

Immerwahr’s example also reminds me of another point I often make, that the wording on clicker questions need not be as precise as the wording on exam questions.  One reason is that if the question isn’t worded exactly right, an instructor can still make it work during the discussion of the question.  Another is that clicker questions can be modified and asked again based on student comments during discussion.  In Immerwahr’s example, for instance, the instructor could easily change “will” to “can” in the question and re-poll the students.

For an expanded version of Immerwahr’s introduction to clickers, read his Teaching Philosophy article, “Engaging the ‘Thumb Generation’ with Clickers.”  The article includes more discussion of the clicker uses mentioned above, as well as other uses, and features several sample questions.

And for even more resources on using clickers in philosophy instruction, visit the Peer Instruction in the Humanities project out of Monash University in Australia.  This site features a step-by-step guide to PI, advice on designing a PI lecture, a description of a sample PI lecture, examples of various types of clicker questions appropriate for this teaching context, and even a question bank organized by topic!  I’m very glad to know that there’s a humanities clicker question bank out there to complement existing question banks in the sciences.

Image: “Portrait of Erasmus Desiderius“, Andreas Praefcke, Wikimedia Commons

Reference: DeBourgh, G. A. (2008). Use of classroom “clickers” to promote acquisition of advanced reasoning skills. Nurse Education in Practice, 8(2), 76-87.

Summary: Gregory DeBourgh provides a useful introduction to using clickers in nursing education, focusing on pedagogical strategies that use clickers to promote critical thinking.  His exploration of critical thinking in the context of nursing education is particularly interesting.  Here’s a sample:

“Reasoning is about using intellectual power to draw conclusions, form judgments, and make inferences based on evidence, education, and experience… The practical significance of acquiring skill in advanced reasoning is to move to the level of predictive clinical reasoning which enables one to anticipate both ideal and likely outcomes given a set of data.”

DeBourgh argues that using classroom response systems to engage students in high-level questions is an effective strategy for developing their critical thinking skills.  He supports this assertion by drawing on the literature on the roles of feedback and questioning in learning and by sharing concrete examples of clicker uses in nursing education.

Included are three sample questions, including a “one-best-answer” question that asks students to identify the likely cause of a particular symptom shown by a patient in a case study.  DeBourgh endorses the use of such questions since they better represent situations students are likely to encounter in clinical settings where they must deal with ambiguity.  He also suggests asking question sequences based around patient cases that “change the focus to add new variables,” noting that doing so also reduces the cognitive load students experience when familiarizing themselves with a new case.

DeBourgh makes a good argument for using clicker questions to model critical thinking skills for students:

“Anticipate likely incorrect responses and prepare ‘talking points’ for discussion as this facilitates ‘thinking on your feet’ and makes more visible to students how an expert uses heuristics, reasoning, and refined problem-solving skills to gain command of a clinical situation.”

Asking questions designed to provide an opportunity for the instructor to model critical thinking is one instance of many DeBourgh describes of crafting questions to meet particular teaching and learning objectives.  In doing so, DeBourgh draws on articles by Ian Beatty on good question design, transferring Ian’s advice to the context of nursing education.

DeBourgh also points out that clicker questions embedded in PowerPoint can be particularly useful in nursing, a field which frequently uses pictures, diagrams, sound clips, and video–media that can also be embedded in PowerPoint.  He also notes that nursing courses often involve discussion of nursing ethics and student opinions about ethical decisions, topics that lend themselves well to clicker questions.

The article also includes results from a study survey about clicker use.  Student responses to rating questions are summarized, and student responses to open-ended questions are presented, as well.

DeBourgh ends with a few challenges involved in teaching with clickers, two of which are particularly significant.  He notes that since instructors can track student performance in a class on a daily basis, expectations for students are raised, which is not popular with all students.  DeBourgh also speaks to the increased expectations for instructors:

“The greatest challenge is the new role for faculty to plan the curriculum and instruction around ‘deep comprehension’ rather than ‘covering content’ using a traditional lecture format.”

Comments: I read this article in advance of my presentation at the Health Professional Educational Research Symposium earlier in the month, and I was particularly impressed with Gregory DeBourgh’s eloquence in describing critical thinking in the context of nursing education and in describing ways that clicker pedagogies can foster those critical thinking skills.

As I’ve tried to capture above, DeBourgh describes a variety of ways of using clickers in nursing education, and he included one approach that was entirely new to me, one inspired by the 50-50 option in the television game show Who Wants to Be a Millionaire? DeBourgh suggests that before the correct answer to a clicker question is revealed to students, an instructor might ask the students which answer choices should be eliminated.  It’s a little unclear how DeBourgh implements this, but I can even imagine setting up a multiple-mark question with four answer choices, then asking students not to select the one correct answer but to select two incorrect answers.  This would offer a nice change of pace in question format and would help students focus on more than just the correct answer.  It’s often useful for students to consider why some answer choices are plausible on the surface but actually incorrect.

Hopefully it’s also above that DeBourgh puts an emphasis on teaching with case studies (multimedia case studies, at that) in his article.  I understand that case study methods are perhaps more common in nursing than they are in other disciplines, and I appreciated reading this article as a way to better understand why that was the case.  DeBourgh’s comments about using clickers for discussing ethics also helped me better understand the disciplinary context here.

If you’re a nursing educator, please share a thought or two about using clickers in your field in the comments section!

Update: Greg DeBourgh emailed me and clarified his 50-50 technique.  Here’s what he said:

I display the potential four-answers to a given question, then before the students “vote” with their clickers, I ask for a volunteer or select a student at random (my clicker system has this feature) and ask the student to eliminate 2 of the 4 potential answers and to explain why they are eliminating these two. This speaking out loud of their rationale for eliminating two of the potential answers that are not related to the question strengthens the students’ reasoning skills. They actually get quite good at it. If the student I called upon to answer hesitates or is reluctant to speak, I invite them to choose a “consultant” in the room to help them out. I hope this clarifies a bit for you.

I asked Greg what he does if the student eliminates the correct answer.  Here’s his response:

If the student eliminates one of the correct answers, it is still learning, and so I ask “does everyone agree with the 50/50 elimination?” If someone objects, I ask for their rationale. If no one objects, I just let the process go and during the “reveal and rationale” we talk about why each answer is incorrect or correct.

Thanks, Greg, for this clarification, and for this great use of clickers.

Last weekend I was honored to give the opening keynote at Nova Southeastern University’s Health Professions Education Research Symposium (HPERS) in Ft. Lauderdale, Florida.  (And, yes, Ft. Lauderdale is a great place to be in mid-January!)  Below you’ll find the slides from my presentation.  Many of my slides are designed to visually complement my verbal presentation, so they might not make too much sense on their own.  Still, I think they’ll give you a sense of what I talked about.  There are also some sample clicker questions in the slidedeck appropriate for the health professions.

And speaking of backchannel, I live-tweeting during several of the presentations later in the day at HPERS.  Here are the highlights of the conference for me, according to my Twitterstream.

Here’s another example of a slideshow that combines great visuals with a few short sentences and phrases to explain something.  The “something” in this slideshow is teaching with clickers, naturally.

This slideshow is by Sidney Eve Matrix of Queen’s University.  She does a great job of using visuals in her presentation.  My favorite image is probably the overflowing glass on slide 19.  It’s a beautiful image that works very well as a metaphor for Sidney’s point on that slide.

I’m always a little surprised when I run into college teachers who haven’t heard of Bloom’s Taxonomy.  For some strange reason, I’ve known about it since fourth grade.  (Mrs. Orchard also introduced me to Greek mythology, Spanish, and a variety of other interesting topics.)  However, I’ve found that many instructors aren’t familiar with the taxonomy, which is a shame since it provides such a useful framework for thinking about the questions we ask of our students.

The original 1956 taxonomy by Benjamin Bloom and his collaborators consisted of six educational objectives: knowledge, comprehension, application, analysis, synthesis, and evaluation.  There’s a lesser known 2001 revision of the taxonomy that I find a little more useful than the original (which is why I used it my book).  In the revision, the objectives are described using verbs: recall, understand, apply, analyze, evaluate, and create.

I recently found a couple of Web sites that use the taxonomy to provide guidance for writing clicker questions.  The Technology Enhanced Learning and Research (TELR) center at the Ohio State University has a great site on teaching with clickers, one that includes advice for designing clicker questions using the 1956 taxonomy.  Here’s their example of a “synthesis” question, one that asks students to “put parts together to form a new whole”:

If Homer wrote The Iliad today, Stanley Fish and Harold Bloom would argue, respectively, whether the work should be classified as:

1. Existential vs. romantic
2. Postmodern vs. classical
3. Modern vs. romantic
4. Postcolonial vs. modern
5. Preliterate vs. postliterate

This question requires students to know something about The Iliad as well as Stanley Fish and Harold Bloom’s approaches to critical theory.  I think this is a great example of a clicker question that calls for higher-level thinking, one that would work well in a literature course.

The e-Learning Center at Northern Arizona University has a similar set of advice for designing clicker questions.  They apparently used the TELR page as a model and thus also used the 1956 taxonomy.  Interestingly, their sample questions are all from the field of archaeology.  (Now that I think about it, Mrs. Orchard included archaeology in my fourth-grade curriculum, too.)  Here’s their example of a “synthesis” question:

We recently excavated a site in northeastern Arizona with a small 6 – 8 room roomblock, tusayan black-on-white pottery, and a small kiva. Nearby we found prehistoric rock alignments in association with large multi-use bifaces suggesting the area was used as an agricultural fields. What kind of site is this?

1. Pueblo II Anasazi
2. Pueblo II Mogollon
3. Basketmaker II

Answer:  A. Pueblo II Anasazi. The Anasazi were the prehistoric archaeological culture that lived small roomblocks, made tusayan black-on-white pottery, and lived in northeastern Arizona.

This looks like a great question, but I might dispute its categorization as a “synthesis” question.  Whereas the TELR example above clearly requires students to put ideas together, this question seems to require students to know the characteristics of three kinds of archaeological sites, which is more of a “comprehension” question as I see it.

Agreeing on how to categorize a question using Bloom’s Taxonomy isn’t always easy.  At a conference back in 2007, Shelley Smith from the University of Minnesota-Duluth led a session on writing clicker questions using Bloom’s Taxonomy as a framework.  She shared several example questions with those of attending the session, then asked us to categorize each question according to the taxonomy (using clickers, naturally).  It was very interesting to see how much discussion this activity generated, as many of us enthusiastically debated how to categorize the questions.

Nailing down categories isn’t the ultimate point of using Bloom’s Taxonomy when writing clicker questions, however.  The point is to take advantage of some framework (the 1956 taxonomy, the 2001, some other taxonomy) to help us think about the questions we ask our students and thus be more intentional about aligning our questions with our learning objectives.