I received an email last week from Bill Goffe who teaches economics at SUNY-Oswego (and contributed to this great guide to teaching economics with clickers) with a neat tip for practicing agile teaching. He noted that he’s heard Harvard physics professor Eric Mazur talk about bringing to class a folder full of transparencies, each with a different clicker question. Mazur asks his students the clicker question at the top of the stack and, depending on how well the students do, either moves on to the next question or skips a few to move to a question on a different topic. This is a great way to practice agile teaching by basing the selection of a clicker question on the results of a previous one.

Bill wrote that it can be challenging to take a similar approach if one’s clicker questions are embedded in PowerPoint slides. Breaking out of presentation mode, wandering through one’s slides in “Normal” or “Slide Sorter” mode to find one’s next question, then switching back to presentation mode to display the question–that seems like an awkward process, particularly if it’s visible to the students. However, Bill found a better way:

Yesterday I came across [the Inside Higher Ed article] “The Advantages of a 2,500 Slide PowerPoint Deck,” and it had the solution: number your slides and bring a printed version of the slides in outline view. Look at the latter, find the desired slide, type that number, and then PP takes you to that slide, still in display mode. It might seem like a small point, but it would make class much smoother.

I had no idea that you could just type a slide number while in presentation mode in PowerPoint and instantly go to that slide. (Try it, it really works!) That’s a handy trick for going nonlinear in PowerPoint. It might require you to have a printout of your slides handy so you can determine the number of your next slide, but as the comments on that article point out, if you use the same slide deck over and over, you’ll probably start to memorize some of those numbers. (The presenter described in the article uses the same 2,500 slide PowerPoint deck for all of his presentations. He just skips around nonlinearly in response to questions from the audience!)

This trick is probably better than the one that came to mind when I first read Bill’s email, which is to switch from the “slides” view in PowerPoint to the “outline” view. The outline view gives a more compact view of your slide deck, and, depending on how you’ve formatted your clicker questions, can show you your clicker questions particularly well. I think the type-a-slide-number trick is even slicker.

Another way to go nonlinear is to use Prezi. In Prezi, you can organize all your content (text, images, clicker questions, whatever) visually on a great big canvas. This means that finding a particular bit of content is pretty easy, assuming you’ve placed it on the canvas in a sensible location. And while you can set up a “path” in Prezi to follow somewhat linearly, you can always go “off path” and zoom around to other content at will. I’ve used Prezi for the visuals in a few presentations that also included clicker questions, such as this talk at the University of Louisville and this one at Central Michigan University. In both cases, I simply embedded my clicker questions in the Prezi (in a particularly clever way in the Louisville talk) and ran my clicker software on top of Prezi. Worked like a charm.

(See how I turned the entire Prezi into one big clicker question? The letters A, B, C, and D weren’t visible until near the end of the presentation when I zoomed out and posed my final clicker question.)

Of course, in those talks, I was mostly moving linearly through the Prezi. I can see, however, setting up a Prezi where your clicker questions are organized visually in groups and subgroups and using that to go nonlinear during a class session. It’s not quite as slick as typing a number and instantly moving to a different slide, but I would guess that some of us would be faster at navigating visually to a new question than remembering or looking up a question number.

Finally, one of the comments on the Inside Higher Ed article that Bill sent me links to a blog post describing a “Choose Your Own Adventure” session on information literacy designed and facilitated by librarians at the University of Dubuque. PowerPoint hyperlinks (yet another way to move nonlinearly through a slide deck) were used with clicker questions to have the students determine the progression through the slide deck as they grappled with information literacy tasks like finding and evaluating the quality of sources.

I’ve been eager to find more examples of this kind of classroom response system use since I first read about it in the David Banks book on response systems. That edited volume includes a chapter (Hinde & Hunt, 2006) on the use of a “Choose Your Own Adventure” style question tree in an economics course. For more on the Dubuque library use of CYOA / question trees, see this follow-up blog post and the PowerPoint deck itself.

Thanks for sharing, Bill!

Image: “Choose Your Own Adventure 1” by Flickr user Jason Permenter, Creative Commons licensed

There’s a lively discussion happening on the POD Network listserv this week on teaching large classes. The discussion detoured into a discussion of teaching with clickers. In responding to one of these posts, Louis Schmier wrote:

“Well, Ron, clickers might get feedback and active and collaborative involvement, but learning? Technology is a tool, not a panecea. The basic problem with large class as Ron defines it, is that it violates the basic Aristotelian tenet of KNOWING those in your audience and tailoring yourself so that those in the audience get it, understand it, and retain it.”

This comment struck me as interesting, so I responded to it on the listserv. I’m including my response here on the blog (with a couple of extra links for clarity), in case those not on the listserv find it helpful.

Anton [Tolman] has responded very eloquently to Louis’ concerns about classroom response systems, but I can’t resist weighing in myself. First, there’s plenty of evidence that “active and collaborative involvement” often leads to student learning, so if clickers are indeed fostering more student engagement during class, that sets the stage for more student learning.

And as for the idea of “KNOWING those in your audience and tailoring yourself so that those in the audience get it, understand it, and retain it,” once you get past 15-20 students, it becomes very difficult to do those two things—assessing your students’ learning during class and practicing “agile teaching” by responding to what you find out about their learning on the fly–without a tool like a classroom response system. In fact, these are two teaching tasks that clickers are incredibly well-suited to support.

Imagine you have a single student in your office asking for help in your course. It’s relatively easy to “diagnose” that student and get a sense of what the student understands and doesn’t understand, then to tailor some one-on-one instruction to help the student resolve his or her misunderstandings. If you have 2-3 students in an office hour setting, you can probably do the same thing, although you’re already juggling 2-3 different “private universes” at this point, which can be challenging.

When you move to the “small” class setting, say, 8-10 students, you now have 8-10 “private universes” to try to uncover and respond to. Sure, there could be some similarities among your students in terms of their prior experiences, misunderstandings, and perspectives on course material, but you’ve still got 8-10 different students to build your learning environment in response to. Given 50 or 75 minutes and plenty of discussion, you’ve got a good shot at this, however.

Now move to a bigger class, say 15-20 students. At this point, it’s tough to have enough “air time” for all the students during class. This makes the juggling of “private universes” very challenging. Small group discussions can help (outsourcing some of this work to the students themselves), as can pre-class assignments. But during class, you’ve got quite a task if you want to be responsive to all your students’ various learning needs.

(Here you have my answer to Jeanette [McDonald]‘s question. When is large large? I would say 15-20 students. At that point the dynamics shift in very significant ways.)

Now imagine more students—30, 50, 100, or 500. The challenge of responding to that many unique “private universes” is truly daunting! You have to start making some assumptions about commonalities among those private universes. Clickers are wonderful tools for getting a sense of the validity of those assumptions! You pose a multiple-choice question where the answers are crafted to tie into what you suppose are common understandings (correct or not) and perceptions about the topic at hand, you have the students think about (and maybe discuss in small groups) the question, then you poll them and find out which of the understandings and perceptions are *really* the most common.

The resulting bar chart tells you how to spend the next 5-20 minutes of class time: responding to the student views of the topic that are most common. This “agile teaching” allows you to make the best use of limited class time by responding to as many “private universes” as you can in the time available.

Some caveats: You could miss a very common student understanding or perspective completely when you write your clicker question! The more experience you have with the topic and with students learning about the topic, the less likely this is to happen, but it’s still something to watch out for. That’s why it’s helpful to have some kind of classwide discussion about the question, giving students whose views aren’t represented in the bar chart a chance to share.

You also won’t get to the “long tail” of student views this way. What about the two students in a class of 100 who voted for option D? Will you have time to address that minority view? Maybe not during class, but perhaps after class in some fashion.

I’m also assuming here that you’re teaching a large class! The debate over whether or not classes should have 100 students is secondary to my point here. My point is that *given* a large class, a classroom response system is an excellent tool for understanding one’s students (in the aggregate) and tailoring one’s instruction to one’s students.

Lots more on these ideas (with examples from real classrooms!) in the “agile teaching” category on my blog.

The discussion on the listerv continued productively from here. It’s worth checking out.

Image: “O is for Occipital Lobe” by Flickr user illuminaut / Creative Commons licensed

In spite of including experiences from not one, but two language instructors in my book, I still haven’t found any studies exploring the use of clickers in language classrooms for my bibliography.  And, if you check out the column to the right of this post, you’ll see the various disciplines I’ve covered here, and language instruction is not well represented.  (This very post will be only the second in that category.)  Since I’m pretty sure clickers have incredible potential in language instruction, you can imagine how glad I was to see a recent blog post about clickers in a Spanish class at Georgetown University!

The post is a report from Ellen Johnson, a PhD student in applied linguistics, who teaches and coordinates Spanish courses at Georgetown.  After hearing about clickers at a workshop hosted by Georgetown’s Center for New Designs in Learning and Scholarship (CNDLS), Johnson experimented with clickers in her language classroom.  Not only did she experiment with clickers, she collected some data useful in helping her judge their effectiveness.  Here’s what she did:

In a nutshell, 58 students enrolled in Beginning Spanish courses participated in the study on ser and estar. They were introduced to their uses in context, practiced answering questions using clickers with around 20 slides while viewing their performance in relation to their peers, and then completed posttests and reflection questionnaires.

Johnson also had colleagues observe the classes, and the feedback from both students and instructors about the use of clickers was very positive.  The students were particularly enthusiastic about getting immediate feedback on their learning, and Johnson’s fellow instructors thought the clickers had potential for helping them target their feedback to their students “in a more coherent manner.”

Students raised a couple of concerns in their feedback, however.  They thought that the clicker questions made it difficult for them to take notes during class.  Making clicker questions available to students after class is something I’ve mentioned here before, and there’s a little evidence that doing so is, in fact, very important since it allows students to review clicker questions later.  Knowing that clicker questions will be available online after class also frees students from having to take as many notes during class, which is likely to help them spend more time actually thinking during class.

Johnson’s students also noted that clicker questions don’t allow students to practice their speaking skills in a language class.  That’s a good point, but given the experiences of the language instructors I interviewed for my book, it would seem that clicker questions work very well for listening, reading, and writing skill development.

The main concerns raised by language instructors in Johnson’s study were logistical ones.  They worried that the technology would be difficult to start using or might fail during class.  I don’t know what system they use at Georgetown, but I know there are easy-to-use and reliable systems out there.  Also, Georgetown doesn’t seem to have a full-scale clicker implementation, one where students could be expected to purchase clickers at their bookstore, as is the case at many US colleges and universities.  That creates a logistical barrier, as well, since clickers would have to be distributed and collected each class session.

Thanks to Ellen Johnson for sharing her experiences with clickers.  I would be interested to hear more uses of clickers in language courses.  What kinds of questions and activities work well with clickers in those settings?  And why do you think that clickers aren’t more widely used in language instruction?

(I should also note that Ellen Johnson’s post appeared on a group blog from a group of instructors at Georgetown exploring the use of clickers this spring.  Take a look at previous blog posts for more interesting discussion of teaching with clickers.  This “community of practice” is another example of the value of fostering discussions about teaching and learning across the disciplines.)

Image: “Pink AC Bienvenidos” by Flickr user lopolis / Creative Commons licensed

I can’t remember how I stumbled upon this, but back in 2008 three University of California-Berkeley students, Sohyeong Kim, Nathan Gandomi, and Kate Smith, prototyped an interesting kinesthetic classroom response system they called Students on the Move.  Instead of clickers (or smart phones, etc.) students were given joysticks.  They were told to push their joysticks forward if they felt the lecture should pick up the pace and to pull their joysticks backward if they were confused and thought the instructor should slow down.  These feedback data were visualized on the instructor’s computer screen as a sequence of circles, one for each student.  As a student pushed forward on the joystick, that student’s circle would move upward and turn green.  As the student pulled back, the circle would move downward and turn red.  More details and some photos can be found in the students’ project report [PDF].

The idea of opening up a backchannel for students to give this kind of one-dimensional feedback (speed up / slow down) isn’t new.  Turning Technologies’ “moment to moment” feature does something similar, producing a moving line graph that shows the average student response on, say, a scale of 1 to 5 to a question.  The line graph updates in real time as long as the slide stays on the screen.  You might have seen something similar during the 2008 US presidential elections, when CNN would aggregate the reactions of likely voters during certain presidential debates using a similar system.

You can also generate similar data using i>clicker if you leave visible the results display when voting is open.  For example, in my book I profile Adam Rich, who teaches biology at SUNY-Brockport.  He’ll give students a clicker question using i>clicker and have students respond and change their responses throughout a class discussion of the question.  He’ll monitor the current voting results on the i>clicker base unit during this time and keep the discussion going until the results indicate the students have reached consensus on the correct answer.

While the idea behind Students on the Move isn’t new, these students’ implementation of the idea has some very creative elements.  Using a joystick as the response device creates a more kinesthetic experience for the student than they get with a clicker.  Moreover, the joystick lets you treat student feedback as a continuous variable, instead of the discrete one that’s produced by clicker-based systems.  That is, instead of having students respond with 1, 2, 3, 4, or 5, you can have students respond with any real number between 1 and 5 for a richer set of responses.  I don’t know of any other classroom response systems that provide this kind of continuous data.

The joystick has another useful feature–it returns to its neutral state when it’s not pushed forward or pulled back.  I’ve talked to a few researchers interested in this kind of backchannel tool, and one of their concerns was that students would forget to change their response back to neutral after indicating a more extreme response.  The joystick handles that problem nicely, resulting in fewer of what the Berkeley students call “false positives.”  (It doesn’t handle one of these researchers’ other concerns: the challenge of prompting students to respond as appropriate without interrupting the flow of the lesson.)

The other key innovation here is the visualization of the aggregate data.  Instead of using a line graph or a bar chart, Students on the Move produces this very interesting set of circles that float up and down and change color as students provide feedback.  I don’t know if it’s necessary to change both position and color as students respond, since both visual elements are conveying the same information, but I like the creativity in this idea.  I can imagine that watching most of the circles drop to the bottom of the screen during a tough part of a lecture is fairly dramatic.

The Berkeley students assert in their project report that “traditional” classroom response systems are designed for teachers, not students:

Although many studies show the positive effect of a CRS, some studies demonstrate that students feel stressed and frustrated by being constantly assessed. Our study and interviews indicate that this negativity is due to the fact that these classroom response systems are mainly designed for teachers. We argue that classroom response systems should be redesigned to include the needs of students, as well as the instructors, in order to best benefit both.

It’s true that some instructors use classroom response systems just to give quizzes and take attendance, but as I’ve argued here before, using clickers just to monitor students is problematic (for the reasons the Berkeley students note above) and instructors who do so miss out on some of a classroom response system’s key benefits–student engagement and “agile teaching.”  On the other hand, many instructors use clickers to engage students in meaningful explorations of course content and practice “agile teaching” by altering the course of a lesson based on student feedback.  In the hands of these instructors, I would argue that clickers are very useful to the students as well as to the instructor.

I think perhaps that the real difference between a “traditional” system and one like Students on the Move is that in a traditional classroom response system, the instructor is the one who opens voting.  With Students on the Move (and the other, similar systems mentioned above), the students can respond whenever they like.  The feedback generated is student-paced, not instructor-paced in a sense.  That distinction is why I would classify Students on the Move as a backchannel system, and I’m begun to think of backchannel systems as complementary to more instructor-paced systems.  I see comments from time to time indicating that having students engage in backchannel conversations (say via Twitter) during class can be a replacement for clickers.  I don’t think that’s the case, because the two kinds of classroom response systems (ones that are student-paced and ones that are instructor-paced) serve different, but complementary, functions in the classroom.

More on the relationship between backchannel and clickers later!  For now, I’ll end with a couple of questions about Students on the Move.

1. Does the system provide rich data to the instructor after class?  For instance, it would be useful to have time-stamped data available to match up with a recording of the class to determine what components of the lecture triggered particularly positive or negative reactions from students.
2. Does it make sense to show students the feedback visualization during class?  I’ve talked to a couple of i>clicker instructors who have accidentally displayed the results chart while voting was still open.  They both remarked how quickly the students converged to the same answer, like lemmings.  I wonder if Students on the Move would produce similar counterproductive behaviors.
3. How might instructors respond to this kind of feedback data during class?  That is, what kinds of “agile teaching” decisions might an instructor make?  Certainly, if most of the students say “slow down!” then it makes sense to slow down and ask students for questions.  (The Berkeley students wisely point out that their system doesn’t provide instructors with information on why students are asking for a slower pace, just how many of them are doing so.)  But what if students are split?  What then?  Or if students are asking for a faster pace, but the instructor hasn’t prepared to go that fast?

I’m glad I found out about Students on the Move.  If you can help me figure out how I found out about it, I would appreciate it!  Also, if you know of anything that resulted from this student project beyond the fall 2008 semester, please let me know.

Image: “In Control” by Flickr user Steve Snodgrass / Creative Commons licensed

Just over a year ago, I shared a story here about a clicker question I used in one of my math courses that didn’t go as planned during class.  I titled my post “Flexible Clicker Questions” because I wanted to make the point that clicker questions that seem to be poorly written can turn into real learning opportunities for one’s students.  If you put a poorly worded multiple-choice question on an exam, you’re in for a lot of student complaints and regrading.  However, in class, a poorly worded multiple-choice clicker question can, with a little agility on your part, turn out great.

I mention this because Mitch Keller recently described a similar incident in his math course over on his blog, Partially Ordered Thoughts.  He posed a particular clicker question with what he thought had a single correct answer.  His “correct answer” was indeed the most popular student response to the question, but more than 60% of students selected other answers.  Mitch wisely had his students discuss the question in small groups and then led a classwide discussion of the question.  Not only did he surface the correct reasoning for the “correct” answer, but he discussed the other answer choices, too.  It turned out that there were reasonable arguments for not one, but three of his answer choices.  Mitch writes:

A natural first reaction to a slip-up in a clicker question is almost always “Drats! I thought I’d done that perfectly.” However, it became a teachable moment. In reality, we were able to discuss far more aspects of generating functions than I intended with the question.

Have you used a clicker question that turned out to be poorly worded, yet resulted in valuable class discussions?  Please share below!

Image: “Untitled” by Flickr user Maurizio Polese / Creative Commons licensed

Over on the new Active Class blog, Sidneyeve Matrix recently discussed the idea of turning the college lecture into something like what the television industry calls “event programming.”  She did so in the context of encouraging students to come to class when the lecture is captured for later listening or watching by students.  She suggested instructors who are capturing their lectures incorporate a couple of elements to the in-class experience that aren’t replicated in the lecture capture: the use of video clips that help students remember and make sense of course content and the use of clickers for content and opinion questions.

Sidneyeve makes some great points about how the use of clickers can give students a sense of ownership over the in-class learning experience:

Clicker polls effectively personalize, customize, and socialize the class. Students know that poll results depend on who is in class that day and it is that indeterminacy that lends energy and anticipation to the lecture. Moreover, if the students see that their polling feedback is valued by the professor, and is connected to assessment, they too will value the activity of in-class participation as worthwhile.

Sidneyeve has hit upon a subtle, but important point about one of the roles that clickers (and more general classroom response systems, like backchannel tools) play in the classroom.  Using clickers turns students into co-creators, along with the instructor, of the in-class learning experience.  When the results of a clicker question are shown on-screen, something happens that would not have happened were those particular students not in the room participating that day.  Many instructors who use clickers practice what I call “agile teaching,” using the results of clicker questions to directly inform how they lead discussion or how they spend their class time.  Even when instructors don’t practice agile teaching, the results of a clicker question are still unique to the particular collection of students in the room at that time, and those results have at least some small impact on the students who view them.

Imagine the opposite, what some call “ballistic teaching,” a lesson plan that, once launched, cannot be altered by feedback from students in the room.  A video capture of this kind of lecture would be almost, if not more, valuable for students as attending this lecture in person.  Instructors considering lecture capture are often worried, as Sidneyeve points out, that students won’t come to class if lectures are placed online.  But if there’s virtually no difference between watching a lecture online and watching it from a seat in the classroom, why should students come to class?  What’s the point?

As Sidneyeve points out, including clicker questions in a lecture means that there is necessarily a difference between watching a lecture online and participating in the “live” version.  At the very least, watching a clicker question on video after the fact means that the student’s vote isn’t included in the results of the clicker question.  And if the clicker question is used to facilitate small-group or classwide discussion, then there’s even more difference–and more reason for the student to come to class and participate live!

I would like to take the television metaphor of “event programming” that Sidneyeve uses, and push it one step further.  A colleague of mine shared with me a recent Time essay by James Poniewozik titled “Twitter and TV: How Social Media Is Helping Old Media.”  In the essay, Poniewozik points out that many television viewers DVR their favorite shows to watch them later and skip the commercials.  This latter point is of particular interest to the television industry, since their revenue depends largely on advertisers.  Poniewozik argues, however, that social media like Facebook and Twitter can make certain television shows into “events” that viewers want to watch live.  He points to the live discussions that occurred online during the recent Winter Olympics and Academy Awards.  Participating in these live discussions was an incentive for people to watch these programs live.  I’ll attest to that–one of the reasons I watch Lost live as it airs each Tuesday night is so I can participate in online discussions about the show during and immediately after it!

Where am I going with this?  Well, to continue with the metaphor, a lecture that includes no interactive component and is captured for later viewing by students is like a television show that you DVR and watch when you get around to it.  A lecture that includes interactive components like the use of clickers and backchannel discussion (which is very much like the kind of online discussions Poniewozik describes) is more like an “event” television show that you just have to watch live as it airs.  Wouldn’t it be great if students refused to skip class (and watch the lecture video or borrow a friend’s notes instead) because they’ll miss their best opportunity for learning?

Image: “Empty” by Flickr user Shaylor / Creative Commons licensed

I was looking over my notes from the 2010 Health Professions Educational Research Symposium (HPERS) hosted back in January by Nova Southeastern University, and I was reminded of a couple of interesting points Jim Vanides of Hewlett-Packard made in his closing keynote.  Jim had shared several fascinating ways that educators receiving HP grants have been making use of tablet PCs in the classroom.  I had been familiar with ways in which laptops are often used in the classroom, but I hadn’t heard as much about uses for tablet PCs.

Jim described several examples in which the “digital ink” that tablets provide through their stylus interfaces played key educational roles.  For example, students in an engineering class can draw or design various diagrams on their tablet PCs, then send those to the instructor to be shared and discussed with the whole class.  Sure, students could prepare their diagrams before class for sharing, but sometimes having students work on “deliverables” during class–when the instructor and fellow students are around to provide feedback during the design process–is advantageous.

During the Q&A after Jim’s keynote, I asked him a question I’ve posed here on the blog several times: If students are submitting responses to free-response questions during class, how can an instructor process and use 20 or 30 or more student responses in a timely fashion during class?  With multiple-choice clicker questions, the bar chart is the perfect way to display the results.  How to handle free-response questions?

Jim made two very good points in response to my question:

1. Since humans process visual information very quickly, having students submit diagrams, drawings, photographs, and other visual responses means instructors can often make sense of dozens of student responses very quickly during class–more quickly than with textual responses.  As a big fan of using visual thinking tools in my work, I really liked this answer!  It also argues for using devices with touch interfaces, like tablets and smart phones, as part of classroom response systems used with open-ended questions.  (Jim also noted that in science, engineering, and mathematics, where people often think with pencils in hand, digital ink can be very useful.)
2. Instructors practicing the “usual” version of peer instruction, in which students respond to a clicker question on their own, then discuss it with peers, then vote again, can start processing student responses to open-ended questions during the peer discussion phase–while students are engaged in talking with each other.  This gives the instructor a little more time to make sense of student responses and decide how to discuss them with the class.  This is not unlike taking a “backchannel break” during class in which students brainstorm questions in small groups and submit them to the instructor via backchannel.  (I used this approach in a recent workshop on lecturing.)

As regular readers of this blog know, I’m very excited by the educational possibilities of having a class full of students with Web-enabled, app-enabled, touch-screen devices.  I was glad to hear a few new and very useful ideas on this topic from Jim at the HPERS Conference.  I think it’s often helpful to hear from people somewhat out of the usual academic circles I run in for great new ideas!

Image: “40+86 Tablet” by Flickr user bark

Classroom assessment techniques (CATs) are simple, non-graded, usually anonymous, in-class activities designed to give you and your students useful feedback on the teaching-learning process as it is happening.  The standard reference on CATs is Classroom Assessment Techniques: A Handbook for College Teachers by Thomas A. Angelo and K. Patricia Cross (Jossey-Bass, 1993). This book includes 50 CATs, each described in detail with examples from a variety of disciplines.  You’ve probably heard of a few of these, such as the minute paper, muddiest point exercise, and background knowledge probe.

CATs provide what is known as formative assessment, something I’ve frequently blogged about.  This is assessment of student learning intended to inform future teaching.  Formative assessment is often contrasted with summative assessment, which is performed in order to evaluate student performance.  Summative assessment comes at the end of a learning experience; formative assessment happens as the students are learning.  Feedback from formative assessment can provide instructors with useful insight into what students are understanding, what they are not understanding, and how they might target their teaching to their particular students.

At the recent EDUCAUSE Learning Initiative conference, Charlotte Briggs (University of Illinois-Chicago) and Deborah Keyek-Franssen (University of Colorado-Boulder) presented the results of a very useful study.  They combed through all 50 CATs in the Angelo and Cross book and determined that 23 of them could be used with clickers.  I’ve long thought of classroom response systems as a sort of “technoCAT,” a technology-enhanced classroom assessment technique, since they provide such useful formative assessment of student learning.  Charlotte and I connected via Twitter some time ago, and she had let me know that this analysis of the Angelo and Cross book was in the works.  I was very excited to see her work presented at the ELI meeting!

Charlotte and Deborah’s PowerPoint slides are available, as is their handout listing all 23 CATs that can be performed with clickers.  In their slides, they provide the following example of a CAT that can be used “as is” with clickers.

Background Knowledge Probe: Which of the following is NOT a characteristic of Romanticism?

1. Attention to “the natural”
2. Valued “folk” literature, such as fairy tales
3. Had a strong geographical center in Düsseldorf
4. Referred to “the blue flower” as a central symbol for longing
5. Valued medieval literature and art.

You can imagine asking this kind of clicker question at the start of a unit on Romanticism–or a unit that referenced Romanticism but didn’t focus on it.  If knowledge of Romanticism is important for participating in the discussion that followed, then this question will let instructors know how much time they need to spend reviewing Romanticism at the start of the unit.

The background knowledge probe CAT is one that I referenced in my book since it’s such a common use of clickers.  Where Charlotte and Deborah’s work gets more interesting is in their analysis of the other 49 CATs in the Angelo and Cross book!  For instance, they identify 12 other CATs that can be used “as is” with clickers, including such CATs as approximate analogies, problem recognition tasks, self-confidence surveys, and goal ranking and matching.  They also identify 10 CATs that can be modified to work with clickers.

For example, Angelo and Cross describe the “one sentence summary” CAT, in which students are asked to write a one-sentence summary of a reading or lecture using the WDWWWWHW format: Who Does What to Whom When Where How and Why.  Charlotte and Deborah note that students aren’t able to construct and submit these sentences using clickers.  However, they can be given a potential one-sentence summary and asked to identify its flaws.  The example they share in their slides is this one:

One-Sentence Summary: Find the errors in WDWWWWHW: A grand jury is a panel of judges (who) that decides if someone should be charged with a crime (does what to whom) when the offense might be a felony carrying prison time (when) if federal courts and most state courts (where) by listening to arguments by attorneys from both sides (how) so common sense and community perspectives are part of the criminal justice system (why).

1. Who and Why
2. When and Where
3. How and Why
4. Who and How
5. Does What to Whom and How

This clicker serves much the same purpose as a “traditional” one-sentence summary, in part because it’s a “multiple mark” style of question, asking students to identify not one, but two things wrong with the given summary.  If your clicker system allows actual multiple-mark questions, allowing students to select as many incorrect elements as they wish, the question becomes even more complex–and thus closer in usage to the one-sentence summary described by Angelo and Cross.

Charlotte and Deborah make a few very good points about modifying CATs to work with clickers.  They note that doing so “tends to down-grade the complexity” of the CAT itself.  With the one-sentence summary, for instance, you lose the ability to see what surprising things students might say in their constructed sentences.  However, Charlotte and Deborah point out that class discussion of the clicker question can restore that complexity.  As they write, “Instructors often get the most out of clickers when they are used to prompt discussion,” which is a point I always make when I talk about teaching with clickers.

Here’s one more great example along those lines.  Instead of asking students to write down the “muddiest point” of a lecture at the end of class, Charlotte and Deborah suggest in their handout the following:

List potential topics on slide and include an “other” option. Ask students to indicate the topic with which they had the most difficulty. If a significant proportion of the class selects “other”, probe the class to identify other “muddy” issues.

For other ideas on adapting CATs for use with clickers, take a look at their handout.  The Angelo and Cross CATs book is well-known in some educational circles (not so much in others, unfortunately), and Charlotte and Deborah’s work serves as a nice introduction to teaching with clickers for those familiar with the book.  Conversely, those already teaching with clickers are likely to find a few new ideas for using them as they explore the CATs framework.

Gardner Campbell and two of his Baylor University colleagues, librarian Ellen Filgo and first-year student Alexis Tracy, presented a talk at the recent EDUCAUSE Learning Initiative (ELI) conference on their use of Twitter in Gardner’s first-year seminar course on new media.  The talk, “Twitter Symbiosis: A Librarian, a Hashtag, and a First-Year Seminar,” is online (video + slides) thanks to ELI, which meant I could “attend” the presentation in spite of the fact that I didn’t go to the ELI conference.  I recently posted nine uses for backchannel in education, and Gardner’s talk provides another great example of the potential of Twitter-facilitated backchannel conversations in college teaching.

In a nutshell, here’s how Gardner incorporated Twitter in his course: As part of their class participation, Gardner’s students were encouraged to open Twitter accounts and participate in backchannel discussion on Twitter during class sessions, using a course-specific hashtag to make their tweets easy to find and follow.  Moreover, Ellen Filgo, a university librarian, participated in the Twitterstream, too, although she did not attend class sessions in general.  Instead, she followed the Twitter conversation from her office (by loading a column in her Tweetdeck application that searched for the course hashtag) and contributed resources and ideas to the backchannel discussion.

How did Gardner and his students use the backchannel?  I’ll use my “nine uses” as a framework here.  Gardner’s students engaged in notetaking, sharing resources with each other, commenting on the class discussion and presentations given by Gardner and by fellow students, asking questions of Gardner and each other, and helping one another by suggesting answers to those questions.  Also, Gardner was intentional about using the backchannel and other mechanisms (including student blogs “fed” into a course “mother blog” and social bookmarking via Delicious) to build community in his course.

Perhaps what is most interesting about this example is that the inclusion of librarian Ellen Filgo served to open the classroom to those not physically present.  In the talk, Ellen describes her participation in the backchannel as “librarian jazz,” referring to the improvisational quality of her interactions with the students.  She knew the topic of each class session’s conversation, but didn’t always have the readings ahead of time and couldn’t hear the verbal conversation in the room.  This meant that she had to suggest resources and answers to student questions based entirely on the Twitterstream in real time.  In the ELI talk, both Ellen and Gardner referred to “agile” teaching, one of my favorite terms, which made me smile!

Ellen noted that one positive outcome of this participation was that she was involved in the students’ research work at a much earlier point in that work than is typical for her work with students.  She was thus able to assist students in valuable ways, and the students’ understanding of the role of the library in their work was enhanced.

If you watch the talk online, be sure to listen to Gardner’s student, Alexis Tracy, describe her experiences in the course.  Using social media (Twitter, blogs, social bookmarking) in an academic setting was new to her, and she became very interested in Twitter in particular.  She’s remarkably reflective and well-spoken about the impact the backchannel had on her learning in the course.  I was impressed that she described herself as an “epistemologist”–that’s a word I didn’t learn until graduate school!

Here are a few other points that Gardner and his colleagues make in their talk:

• It’s important to use a course-specific hashtag.  That makes finding class tweets easy and helps to create a sense of community.
• Be sure to archive the class tweets using a service like Twapper Keeper which creates a permanent archive of all tweets using a particular hashtag.  They didn’t do this and regretted it later when they discovered that Twitter’s search function doesn’t go that far back.
• Gardner’s students all gave class presentations.  During the presentations, the other students participated in the backchannel as usual.  This provided a useful source of feedback to the presenting students, who would frequently read through those tweets after class.  I’m tempted to call this a “tenth” use of backchannel.  It falls under the category of students helping one another, but when the student being helped is the presenter, this use is, in a way, more significant.
• Near the end of the talk, Gardner says, “If you want your students to tweet well, then you need to tweet well.”  If not, that is, if you ask your students to engage in an activity in which you yourself do not engage, your students are likely to view it as busywork and not view it as a valuable learning activity.  Gardner has enough experience blogging and having his students blog that I consider this sound advice.

See the online archive of the talk for other points, including Gardner’s approach to grading backchannel participation, a great anecdote about how a question moved from the backchannel to the frontchannel, and some warnings about what can go wrong when students aren’t prepared well for this kind of participation.  Thanks to Gardner, Ellen, and Alexis for sharing their experiences with this very new form of classroom interaction!

Reference: DeBourgh, G. A. (2008). Use of classroom “clickers” to promote acquisition of advanced reasoning skills. Nurse Education in Practice, 8(2), 76-87.

Summary: Gregory DeBourgh provides a useful introduction to using clickers in nursing education, focusing on pedagogical strategies that use clickers to promote critical thinking.  His exploration of critical thinking in the context of nursing education is particularly interesting.  Here’s a sample:

“Reasoning is about using intellectual power to draw conclusions, form judgments, and make inferences based on evidence, education, and experience… The practical significance of acquiring skill in advanced reasoning is to move to the level of predictive clinical reasoning which enables one to anticipate both ideal and likely outcomes given a set of data.”

DeBourgh argues that using classroom response systems to engage students in high-level questions is an effective strategy for developing their critical thinking skills.  He supports this assertion by drawing on the literature on the roles of feedback and questioning in learning and by sharing concrete examples of clicker uses in nursing education.

Included are three sample questions, including a “one-best-answer” question that asks students to identify the likely cause of a particular symptom shown by a patient in a case study.  DeBourgh endorses the use of such questions since they better represent situations students are likely to encounter in clinical settings where they must deal with ambiguity.  He also suggests asking question sequences based around patient cases that “change the focus to add new variables,” noting that doing so also reduces the cognitive load students experience when familiarizing themselves with a new case.

DeBourgh makes a good argument for using clicker questions to model critical thinking skills for students:

“Anticipate likely incorrect responses and prepare ‘talking points’ for discussion as this facilitates ‘thinking on your feet’ and makes more visible to students how an expert uses heuristics, reasoning, and refined problem-solving skills to gain command of a clinical situation.”

Asking questions designed to provide an opportunity for the instructor to model critical thinking is one instance of many DeBourgh describes of crafting questions to meet particular teaching and learning objectives.  In doing so, DeBourgh draws on articles by Ian Beatty on good question design, transferring Ian’s advice to the context of nursing education.

DeBourgh also points out that clicker questions embedded in PowerPoint can be particularly useful in nursing, a field which frequently uses pictures, diagrams, sound clips, and video–media that can also be embedded in PowerPoint.  He also notes that nursing courses often involve discussion of nursing ethics and student opinions about ethical decisions, topics that lend themselves well to clicker questions.

The article also includes results from a study survey about clicker use.  Student responses to rating questions are summarized, and student responses to open-ended questions are presented, as well.

DeBourgh ends with a few challenges involved in teaching with clickers, two of which are particularly significant.  He notes that since instructors can track student performance in a class on a daily basis, expectations for students are raised, which is not popular with all students.  DeBourgh also speaks to the increased expectations for instructors:

“The greatest challenge is the new role for faculty to plan the curriculum and instruction around ‘deep comprehension’ rather than ‘covering content’ using a traditional lecture format.”

Comments: I read this article in advance of my presentation at the Health Professional Educational Research Symposium earlier in the month, and I was particularly impressed with Gregory DeBourgh’s eloquence in describing critical thinking in the context of nursing education and in describing ways that clicker pedagogies can foster those critical thinking skills.

As I’ve tried to capture above, DeBourgh describes a variety of ways of using clickers in nursing education, and he included one approach that was entirely new to me, one inspired by the 50-50 option in the television game show Who Wants to Be a Millionaire? DeBourgh suggests that before the correct answer to a clicker question is revealed to students, an instructor might ask the students which answer choices should be eliminated.  It’s a little unclear how DeBourgh implements this, but I can even imagine setting up a multiple-mark question with four answer choices, then asking students not to select the one correct answer but to select two incorrect answers.  This would offer a nice change of pace in question format and would help students focus on more than just the correct answer.  It’s often useful for students to consider why some answer choices are plausible on the surface but actually incorrect.

Hopefully it’s also above that DeBourgh puts an emphasis on teaching with case studies (multimedia case studies, at that) in his article.  I understand that case study methods are perhaps more common in nursing than they are in other disciplines, and I appreciated reading this article as a way to better understand why that was the case.  DeBourgh’s comments about using clickers for discussing ethics also helped me better understand the disciplinary context here.

If you’re a nursing educator, please share a thought or two about using clickers in your field in the comments section!

Update: Greg DeBourgh emailed me and clarified his 50-50 technique.  Here’s what he said:

I display the potential four-answers to a given question, then before the students “vote” with their clickers, I ask for a volunteer or select a student at random (my clicker system has this feature) and ask the student to eliminate 2 of the 4 potential answers and to explain why they are eliminating these two. This speaking out loud of their rationale for eliminating two of the potential answers that are not related to the question strengthens the students’ reasoning skills. They actually get quite good at it. If the student I called upon to answer hesitates or is reluctant to speak, I invite them to choose a “consultant” in the room to help them out. I hope this clarifies a bit for you.

I asked Greg what he does if the student eliminates the correct answer.  Here’s his response:

If the student eliminates one of the correct answers, it is still learning, and so I ask “does everyone agree with the 50/50 elimination?” If someone objects, I ask for their rationale. If no one objects, I just let the process go and during the “reveal and rationale” we talk about why each answer is incorrect or correct.

Thanks, Greg, for this clarification, and for this great use of clickers.