In spite of including experiences from not one, but two language instructors in my book, I still haven’t found any studies exploring the use of clickers in language classrooms for my bibliography.  And, if you check out the column to the right of this post, you’ll see the various disciplines I’ve covered here, and language instruction is not well represented.  (This very post will be only the second in that category.)  Since I’m pretty sure clickers have incredible potential in language instruction, you can imagine how glad I was to see a recent blog post about clickers in a Spanish class at Georgetown University!

The post is a report from Ellen Johnson, a PhD student in applied linguistics, who teaches and coordinates Spanish courses at Georgetown.  After hearing about clickers at a workshop hosted by Georgetown’s Center for New Designs in Learning and Scholarship (CNDLS), Johnson experimented with clickers in her language classroom.  Not only did she experiment with clickers, she collected some data useful in helping her judge their effectiveness.  Here’s what she did:

In a nutshell, 58 students enrolled in Beginning Spanish courses participated in the study on ser and estar. They were introduced to their uses in context, practiced answering questions using clickers with around 20 slides while viewing their performance in relation to their peers, and then completed posttests and reflection questionnaires.

Johnson also had colleagues observe the classes, and the feedback from both students and instructors about the use of clickers was very positive.  The students were particularly enthusiastic about getting immediate feedback on their learning, and Johnson’s fellow instructors thought the clickers had potential for helping them target their feedback to their students “in a more coherent manner.”

Students raised a couple of concerns in their feedback, however.  They thought that the clicker questions made it difficult for them to take notes during class.  Making clicker questions available to students after class is something I’ve mentioned here before, and there’s a little evidence that doing so is, in fact, very important since it allows students to review clicker questions later.  Knowing that clicker questions will be available online after class also frees students from having to take as many notes during class, which is likely to help them spend more time actually thinking during class.

Johnson’s students also noted that clicker questions don’t allow students to practice their speaking skills in a language class.  That’s a good point, but given the experiences of the language instructors I interviewed for my book, it would seem that clicker questions work very well for listening, reading, and writing skill development.

The main concerns raised by language instructors in Johnson’s study were logistical ones.  They worried that the technology would be difficult to start using or might fail during class.  I don’t know what system they use at Georgetown, but I know there are easy-to-use and reliable systems out there.  Also, Georgetown doesn’t seem to have a full-scale clicker implementation, one where students could be expected to purchase clickers at their bookstore, as is the case at many US colleges and universities.  That creates a logistical barrier, as well, since clickers would have to be distributed and collected each class session.

Thanks to Ellen Johnson for sharing her experiences with clickers.  I would be interested to hear more uses of clickers in language courses.  What kinds of questions and activities work well with clickers in those settings?  And why do you think that clickers aren’t more widely used in language instruction?

(I should also note that Ellen Johnson’s post appeared on a group blog from a group of instructors at Georgetown exploring the use of clickers this spring.  Take a look at previous blog posts for more interesting discussion of teaching with clickers.  This “community of practice” is another example of the value of fostering discussions about teaching and learning across the disciplines.)

Image: “Pink AC Bienvenidos” by Flickr user lopolis / Creative Commons licensed

Just over a year ago, I shared a story here about a clicker question I used in one of my math courses that didn’t go as planned during class.  I titled my post “Flexible Clicker Questions” because I wanted to make the point that clicker questions that seem to be poorly written can turn into real learning opportunities for one’s students.  If you put a poorly worded multiple-choice question on an exam, you’re in for a lot of student complaints and regrading.  However, in class, a poorly worded multiple-choice clicker question can, with a little agility on your part, turn out great.

I mention this because Mitch Keller recently described a similar incident in his math course over on his blog, Partially Ordered Thoughts.  He posed a particular clicker question with what he thought had a single correct answer.  His “correct answer” was indeed the most popular student response to the question, but more than 60% of students selected other answers.  Mitch wisely had his students discuss the question in small groups and then led a classwide discussion of the question.  Not only did he surface the correct reasoning for the “correct” answer, but he discussed the other answer choices, too.  It turned out that there were reasonable arguments for not one, but three of his answer choices.  Mitch writes:

A natural first reaction to a slip-up in a clicker question is almost always “Drats! I thought I’d done that perfectly.” However, it became a teachable moment. In reality, we were able to discuss far more aspects of generating functions than I intended with the question.

Have you used a clicker question that turned out to be poorly worded, yet resulted in valuable class discussions?  Please share below!

Image: “Untitled” by Flickr user Maurizio Polese / Creative Commons licensed

Over on the new Active Class blog, Sidneyeve Matrix recently discussed the idea of turning the college lecture into something like what the television industry calls “event programming.”  She did so in the context of encouraging students to come to class when the lecture is captured for later listening or watching by students.  She suggested instructors who are capturing their lectures incorporate a couple of elements to the in-class experience that aren’t replicated in the lecture capture: the use of video clips that help students remember and make sense of course content and the use of clickers for content and opinion questions.

Sidneyeve makes some great points about how the use of clickers can give students a sense of ownership over the in-class learning experience:

Clicker polls effectively personalize, customize, and socialize the class. Students know that poll results depend on who is in class that day and it is that indeterminacy that lends energy and anticipation to the lecture. Moreover, if the students see that their polling feedback is valued by the professor, and is connected to assessment, they too will value the activity of in-class participation as worthwhile.

Sidneyeve has hit upon a subtle, but important point about one of the roles that clickers (and more general classroom response systems, like backchannel tools) play in the classroom.  Using clickers turns students into co-creators, along with the instructor, of the in-class learning experience.  When the results of a clicker question are shown on-screen, something happens that would not have happened were those particular students not in the room participating that day.  Many instructors who use clickers practice what I call “agile teaching,” using the results of clicker questions to directly inform how they lead discussion or how they spend their class time.  Even when instructors don’t practice agile teaching, the results of a clicker question are still unique to the particular collection of students in the room at that time, and those results have at least some small impact on the students who view them.

Imagine the opposite, what some call “ballistic teaching,” a lesson plan that, once launched, cannot be altered by feedback from students in the room.  A video capture of this kind of lecture would be almost, if not more, valuable for students as attending this lecture in person.  Instructors considering lecture capture are often worried, as Sidneyeve points out, that students won’t come to class if lectures are placed online.  But if there’s virtually no difference between watching a lecture online and watching it from a seat in the classroom, why should students come to class?  What’s the point?

As Sidneyeve points out, including clicker questions in a lecture means that there is necessarily a difference between watching a lecture online and participating in the “live” version.  At the very least, watching a clicker question on video after the fact means that the student’s vote isn’t included in the results of the clicker question.  And if the clicker question is used to facilitate small-group or classwide discussion, then there’s even more difference–and more reason for the student to come to class and participate live!

I would like to take the television metaphor of “event programming” that Sidneyeve uses, and push it one step further.  A colleague of mine shared with me a recent Time essay by James Poniewozik titled “Twitter and TV: How Social Media Is Helping Old Media.”  In the essay, Poniewozik points out that many television viewers DVR their favorite shows to watch them later and skip the commercials.  This latter point is of particular interest to the television industry, since their revenue depends largely on advertisers.  Poniewozik argues, however, that social media like Facebook and Twitter can make certain television shows into “events” that viewers want to watch live.  He points to the live discussions that occurred online during the recent Winter Olympics and Academy Awards.  Participating in these live discussions was an incentive for people to watch these programs live.  I’ll attest to that–one of the reasons I watch Lost live as it airs each Tuesday night is so I can participate in online discussions about the show during and immediately after it!

Where am I going with this?  Well, to continue with the metaphor, a lecture that includes no interactive component and is captured for later viewing by students is like a television show that you DVR and watch when you get around to it.  A lecture that includes interactive components like the use of clickers and backchannel discussion (which is very much like the kind of online discussions Poniewozik describes) is more like an “event” television show that you just have to watch live as it airs.  Wouldn’t it be great if students refused to skip class (and watch the lecture video or borrow a friend’s notes instead) because they’ll miss their best opportunity for learning?

Image: “Empty” by Flickr user Shaylor / Creative Commons licensed

Monica Rankin has received some attention for her use of Twitter in the introductory history course she taught this spring at the University of Texas-Dallas, in part because Kim Smith, a UT-Dallas graduate student in the Emerging Media and Communication program, produced a video about Dr. Rankin’s “Twitter experiment” and posted it on YouTube.  I highly recommend you watch this five-minute video since it provides a useful overview of Dr. Rankin’s use of Twitter as backchannel during class.

Dr. Rankin has also posted some additional thoughts about her use of Twitter that are worth reading.  Following up on my musings about the use of Twitter in the classroom in an early blog post, I have a few comments and questions about Rankin’s experiment.

First, however, I’ll point out that given the fairly broad definition I like to use for “classroom response system,” the use of Twitter in the classroom is most definitely on-topic for this blog!  In case you were wondering…

Rankin notes that her Monday and Wednesday classes followed a traditional lecture model.  It was in her Friday classes that she used Twitter and required her students to read historical essays and primary source documents.  She had her students “do the reading” on Fridays to help them prepare to engage in small-group, large-group, and Twitter-based discussions.  (She gave an open-notes quiz at the start of class to hold them accountable for the reading.)  This aligns quite well with Eric Mazur’s transfer-assimilation model of learning: Rankin used the Monday and Wednesday lectures and the pre-class readings to transfer information to the students and the Friday discussions to help students assimilate that information.  I wonder, however, if she might have had her students “do the reading” prior to the Monday and Wednesday classes, opening up the option of discussions (small-group, large-group, and Twitter-based) in those classes.

Rankin also notes that most of her students were not already Twitter users at the start of the semester.  They were, however, familiar with social networking sites like Facebook and MySpace, so the concept behind Twitter wasn’t entirely alien to them.  However, this meant that most students had to sign up for new Twitter accounts for Rankin’s course.  One of the aspects of Twitter that Dan Cohen noted in his crowdsourcing-via-Twitter experiment is the “multiplier effect,” in which a comment made on Twitter is “retweeted” (forwarded, to use email lingo) by those who follow the person that made the comment.  This allows comments to spread very rapidly through social networks and can bring many people into a conversation on Twitter very quickly.  Since Rankin’s students were mostly new to Twitter, this meant they likely had few followers on Twitter and thus the multiplier effect apparently not much of a factor in her course.  Rankin notes that she plans to use Twitter again this fall.  I wonder if she and her students will have enough followers in the fall to see how the multiplier effect might enhance or detract from her use of Twitter in the classroom.

The upside of having students create Twitter accounts for the course was that students could “Tweet” their comments during discussion without worrying what their friends outside of the class thought about their comments.  Students already using Twitter who had friends following them on Twitter might have been more circumspect regarding their comments.  Rankin might find this more of an issue in the fall, as more students begin using Twitter.  I’ve heard many students comment that they don’t like faculty to contact them on Facebook because they see Facebook as their social space and they don’t appreciate faculty intruding into that space.  Right now, students don’t see Twitter as “their” space, I think, but that might change as more of them start using it.  There might be some pushback from students regarding educational uses of Twitter in the fall.

Rankin provides a few practical tips for using Twitter in the classroom, too.

• It’s important that students tag their posts with a “hashtag” so that they can be found easily by other students through Twitter’s search tool.  Rankin took the step of using a different hashtag each week during the course, providing a way for students to also find their peers’ comments on particular topics in the course more easily (during, say, exam review time).
• Rankin displayed her students’ Twitter stream during class on the classroom projector system via Tweetdeck, a program that provides a fair amount of flexibility for finding and displaying Twitter content.  This meant that students without laptops or smart phones could see the conversation unfold while contributing to it via text-messaging on their regular mobile phones.  Rankin points out that the font size on Tweetdeck was small, however, leading me to think there’s an opportunity here to develop a Twitter application customized for classroom display.
• I’ve mentioned several times on this blog that the problem with responses to open-ended questions collected via classroom response systems is that there’s no easy way to make sense of these responses on the fly during class.  (In contrast, the bar chart that shows the responses to a multiple-choice question aggregates those responses very nicely.)  How did Dr. Rankin get around this?  She had her TA monitor the Twitter conversation during class, post comments, respond to questions, and notify Rankin (who would circulate among the students during class) of Twitter comments worth addressing to the whole class.  I think this is a great solution, and it’s a nice example of team-teaching.  (In fact, those who are already team-teaching classes might consider trying this out!)
• Rankin and her TA also used Twitter’s “favorites” feature to mark student comments that were particularly insightful or useful.  This is another way to make sense of student responses to open-ended questions, particularly looking ahead to students’ use of these responses as they study after class.

What’s unclear in Rankin’s reflections is how the Twitter discussion impacted the small-group and classwide discussion and vice versa.  It’s clear that there were such impacts, but I think collecting some data on this would be useful.  Since Twitter provides a record of student comments (who said what when, as well as who replied to whom), it’s a great source of data for investigating how students learn in this environment and what the discussion dynamics are.

Rankin notes that she provided her students with discussion topics to frame the small-group and Twitter-based discussion, but she doesn’t go into detail about these topics.  I wonder if particular types of topics or discussion questions work better or worse for encouraging meaningful Twitter conversation.  There’s another project waiting for someone!

What are your thoughts on the use of Twitter in the classroom to faciliate discussion?

At a recent conference, I met Kevin Barhydt, an educational technologist at Union College in New York.  His presentation about clickers was great, but one thing that stood to me about Kevin was a pattern I observed in the questions he asked during other presentations.  He kept wanting to know if the clicker questions used by instructors during class were made available to students after class for review and study.

I think this is a great question.  There’s some evidence in an article by Bunce, VandenPlas, and Havanki (2006) that making clicker questions available to students after class can enhance student learning.  In that article, the impact on student learning of online quizzes was compared to that of in-class clicker questions.  It was found that the online quizzes helped students more, perhaps because those quiz questions were made available to students for study, whereas the clicker questions were not.

I recently learned of a new project out of the University of Ediburgh called Electronic Voting Analysis and Feedback for All (EVAF4All).  The goal of the project is to develop a vendor-independent way to provide students with useful feedback about clicker questions after class.  The feedback would include a student’s individual answer to a clicker question, the distribution of answers for the student’s classmates, and links to educational resources related to the topic of the clicker question.

There are some significant technical challenges that the EVAF4All project will have to overcome, including the development of an online application that will interface with any classroom response system (regardless of vendor) and any learning management system.  You can read a little more about these challenges in a blog post by Nitin Parmar of the University of Bath.  However, given the potential importance of the ways that students can learn from clicker questions after class, I think the EVAF4All’s project goal is a good one.  I would be interested in finding out what kind of assessment they’re planning for their project.  That might help discover the role of post-class review of clicker questions in student learning.

Reference: Bunce, D. M., VandenPlas, J., & Havanki, K. (2006). Comparing the effectiveness on student achievement of a student response system versus online WebCT quizzes. Journal of Chemical Education, 83(3), 488-493.

Summary: In the context of a 41-student chemistry course for nursing students, after certain topics were introduced during lecture, students were asked to respond to questions about those topics delivered via a classroom response system (CRS).  (In this case, the system used software running on wireless-enabled laptops that were loaned to students for this purpose.)  After class, students were asked to complete online quizzes on class topics prior to the next class.  The impact of these activities on student learning was assessed through performance on instructor-written hourly exams and a standardized final exam provided by the American Chemical Society.

Since some questions on the hourly and final exams featured topics covered in the CRS questions, some in the out-of-class quizzes, some in both, and some in neither, the authors were able to assess the impact on student learning of the two review mechanisms.  Students did better on hourly exam questions tied to the online quizzes than other questions, indicating that the online quizzes did the best job of helping students do well on these exams.  Results from the final exam indicate that neither the in-class CRS questions nor the out-of-class online quizzes helped students do better on the final, however.

The authors report a couple of problems that undercut these results.  One was that as students responded to the in-class CRS questions, they were able to see the bar chart showing the results as they came in.  This meant that, as one student put it, “People who don’t know the answer simply wait for the graph [to enter their responses] and no real learning occurs.”

The other was that the online quiz questions were made available to students after the quizzes for study purposes, and, according to a survey of students, students took advantage of this resource.  The in-class CRS questions, however, were not made available to students for review.

Students reported on a survey that the most useful features of the CRS were that it helped students reinforce what they learned in class and that giving them the opportunity to talk about class material with their peers helped them learn the material.

Comments: This article is a useful example of how difficult it can be to design a study about classroom response systems that provides meaningful results.  I’m glad the authors published this account, even though their results aren’t strong.  I think others designing similar studies are likely to learn from some of the design mistakes made in this study, in part because the authors did a great job of exploring the constraints of their study design in the conclusion of the article.

One of the primary advantages of having students respond via clickers or similar classroom response systems is that students are able to respond to a question independently.  Each student is asked to respond before he or she finds out what his or her peers think.  This can increase the level of engagement of the students with the question and any subsequent discussion.  The “bandwagon effect” seen in this study, in which students wait to find out what their peers think and then pick the most popular response, is why asking for a show of hands can be unproductive.  Since the classroom response system in this study was used in a such a way as to eliminate this primary advantage, it’s difficult to draw meaningful conclusions from the study.

What is clear from this study is that the availability of clicker questions for students to review after class is a potentially important variable, one that should be included in future research on the effects of classroom response systems.  This study presents some evidence that this may be a key ingredient in the impact of clicker questions on student learning.  I hope that future studies take a closer look at this variable.

Studies like this one that compare classroom response systems with alternatives (like online quizzes or asking for a show of hands) can get complicated very quickly.  I thought I might list just a few of the variables at play in this study to illustrate how difficult it can be to isolate the effects of a CRS.

• Independent Answers – In this case, students didn’t have to answer in-class questions independently; they could see their peers answers before responding.  This was probably not the case for the online quizzes; students likely had to respond on their own before seeing their peers’ responses.
• Peer Instruction - Students in this study were required to pair up and provide consensus answers to the in-class questions.  Students worked on their own when responding to the online quiz questions.  This seems like a significant difference between the two methods.
• Class Results - Students were shown the class results to the in-class questions.  It’s not clear if they were shown the class results to the online quiz questions, either online or in subsequent classes.  The impact of seeing these class results is something worth exploring in CRS research since these results aren’t typically immediately available when other response mechanisms are used.
• Immediacy of Feedback – With the in-class questions, students found out immediately whether they answered the questions correctly.  It’s not clear from the article if students had to wait to receive feedback on their online quizzes.
• Agile Teaching - In-class CRS questions that were missed by significant numbers of students were reviewed in some fashion during class, which meant that the “lecture” was responsive to student learning needs evidenced by the CRS.  It’s not clear if online quiz questions influenced class time in a similar fashion in this study.  The ability to practice “agile teaching” is a primary advantage of a CRS, so this is an important variable to consider.
• Question Type – Not much is said in this article about the nature of questions asked in class via CRS or out of class via online quizzes.  The two sets of questions were judged to be of similar difficulty levels by a panel of four chemical educators, however.  The students appeared to answer the online quiz questions correctly about 88% of the time, so the questions weren’t that difficult.  The format, difficulty, and learning goals associated with questions are potentially important variables.
• Availability for Review – The online quiz questions were made available to students for test review; the CRS questions were not.
• Grading Scheme – The article doesn’t state how either the in-class questions or the online quizzes were factored into students’ grades.  Given the role grades play in student motivation, this is a variable worth noting.

These are some of the variables I look for when I read about studies exploring the impact of classroom response systems.  I hope this partial list will be of use to researchers reading my blog.