Teaching with Classroom Response Systems

Resources for engaging and assessing students with clickers

Archive for May, 2010

Hacking the Academy is a project headed up by the Center for History and New Media at George Mason University.  The idea is to crowdsource an edited volume in a week, which is both innovative and ambitious.  It’s a little unclear how the peer review piece of this will work at this point, but it is clear that there was only a seven-day window for contributions.

I submitted two past blog posts to Hacking the Academy: Clickers, Lecture Capture, and Event Programming (a more conceptual piece) and Backchannel in Education – Nine Uses (a more practical piece).  I wanted to submit something original, as well, so I tried my hand at video production using Jing.  I edited a PowerPoint slide deck I used in a local, face-to-face workshop this spring, then plugged in my USB microphone and recorded a voiceover to go along with the slideshow.

The result is called “Revolution or Evolution? Changing Instructional Practices in the Academy” and you can see it here:

In the video, I compare the “traditional” college lecture format to a vision of the future, one involving all three kinds of backchannel as well as a few Google jockeys.  I argue that helping instructors move from the “traditional” model to this vision of the future will require evolution, not revolution.  One might say, it will require… hacking.

This revolution versus evolution theme is one that I’ll return to this fall at the POD Network conference.  I just found out this week that my proposal for a session on this topic, submitted jointly with Jim Julius of San Diego State University and Dwayne Harapnuik of Abilene Christian University, was approved!  The Hacking the Academy project has started me thinking of ways we can hack this conference session…

Sunday night, I delivered the opening keynote at Central Michigan University’s Great Lakes Conference on Teaching and Learning.  My presentation was titled “Class Time Reconsidered: Motivating Student Participation and Engagement.”  My goal was to share some frameworks and strategies for engaging students in the classroom by taking a few common assumptions about teaching and learning and flipping them on their heads.  Here’s my Prezi, complete with much flipping of things on their heads:

Some thoughts on the presentation:

One of my first clicker questions asked participants to identify a key challenge in motivating students to engage meaningfully during class.  Strangely, the even-numbered answer choices were by far the most popular-students are hesitant to speak up in front of their peers, students focus too much on grades and not enough on learning, and students don’t prepare adequately for class.  These results worked well for me, since I had been planning on addressing ways to reach students who are risk-averse or grade-focused and ways to motivate students to prepare for class in useful ways.

Participants engaged in a Think-Pair-Share activity in which they tried to identify six steps in a typical process their students might undertake to learn in their course.  This followed an introduction to the idea of a “time for telling,” so I asked participants to make sure that “telling” wasn’t the first step on their lists.  I also encouraged participants to force themselves to come up with six steps.  Coming up with three-step plans (take notes during class, figure things out in the homework, regurgitate on exams) is too easy.  Identifying a six-step process means you have think a little more intentionally about how your students learn.

Given the clicker question results indicating that lack of student preparation is a big challenge, we camped out for a while on the idea of a pre-class assignment.  I made two important points about these assignments: they should be graded, if only on effort, so that students will take them seriously and you should make use of these assignments in some way during class.  Otherwise students will see them as busywork, not connected with the “real” work of the course.  One participant shared her approach-she has students create outlines of their pre-class readings, then share and compare their outlines in small groups during class.

Monday morning (the day after my presentation), I saw on the book raffle table that there’s a new book on Just-in-Time Teaching, Just in Time Teaching: Across the Disciplines and Across the Academy edited by Scott Simkins and Mark Maier (Stylus, 2010).  I wish I had known that Sunday night-I would have mentioned it during that section of my presentation!

My third and final clicker question asked participants to identify one of five in-class engagement strategies they wanted to try soon.  While I wasn’t intending the presentation as a pitch for clickers, perhaps my biases couldn’t be hidden-clickers was the number one answer!  This result might have also been because clickers are new and different, but not so different as to require a complete rethinking of one’s teaching approach.  I’m convinced that the return on investment for teaching with clickers is high-one can make small changes in one’s teaching methods that yield significant results.

At the end of the presentation, I had the participants generate questions for me at their tables.  Most of the tables had at least one person with a Web-enabled device (such as the iPads several of the CMU staff hosting the event were sporting).  They used these to submit their tables’ questions via Google Moderator.  I asked them to vote on other tables’ questions, as well, providing me with a ranked list of the most popular questions.  This served as a reasonable demo of Google Moderator as a backchannel tool, but unfortunately I didn’t have time to address the questions that emerged through this process.  My plan is to address the more popular questions with Google Moderator since, as the creator of this Moderator session, I can leave comments on individual questions.  You can see the questions submitted by the group here.

The conference continues through Tuesday morning.  I was able to attend most of the conference on Monday, and I live-tweeted a couple of the sessions.  You can read my tweets here.  Joy Mighty of Queen’s University in Ontario delivered the Monday lunch keynote, and she made a strong case that by not paying attention to matters of diversity in our classroom, we run the risk of fostering inequity.  It was a thought-provoking keynote for me.

Thanks to Central Michigan University for having me as part of their conference and for some great conversations about student engagement!

Blogs, Blogs Everywhere

A couple of weeks ago, I began an experiment, one that’s working out okay so far.  I’ve been blogging here at http://derekbruff.com/teachingwithcrs for almost two years now, but lately I’ve wanted to blog about topics other than classroom response systems.  Instead of cluttering up this blog with off-topic posts, I’ve started blogging at my main website, http://derekbruff.com.

I’m not going to stop blogging here about clickers and other classroom response systems, since those topics are still very much of interest to me.  And to (hopefully) make things simple, I’ve figured out how to syndicate the feed from this blog into my main website, so all of my clickers posts will show up on both blogs.  (I’m also syndicating some of my posts from the Vanderbilt Center for Teaching blog on the main site, too.)

What does this all mean for you?

  • If you’d like to read all my blogs posts–the ones about clickers as well as the ones about other topics–read my main website, http://derekbruff.com.  In fact, feel free to subscribe to the RSS feed from the main site in your favorite RSS reader.
  • If, however, your interests lie with classroom response systems and you aren’t interested in my posts on other topics, just keep reading this blog right here at http://derekbruff.com/teachingwithcrs.  It won’t bother me, and you don’t have to do anything different!

I thought about just moving all of my blogging over to my main website, but I’ve heard others mention that when they switched blogs, they lost readers.  My experiment in syndication is an attempt to avoid that.  Also, I’ve found my extensive set of categories on this blog very useful, and I didn’t want to lose those.  I’m going to keep the set of categories fairly small on the main site.

This all may be more information than you want, but for those of you who have been reading this clickers blog, I wanted to make sure you knew that I was starting to expand my blogging a bit.  Thanks for being a part of this blog, and I hope to continue to see your comments here!

Image: “Pipes” by Flickr user .:Axle:. / Creative Commons licensed

  • Filed under: Blog Post
  • More from my round-up of articles on clickers in the health professions.  This time, another article that doesn’t add much to the literature, but raises an interesting idea.  Again, your comments are invited…

    Reference: Williams, B., & Boyle M. (2008). The use of interactive wireless keypads for interprofessional learning experiences by undergraduate emergency health students. International Journal of Education and Development Using ICT, 4(1).

    Notes: This article has features results from a survey of students using clickers in a “foundations of health” course taken by emergency health students as well as students majoring in “nursing, physiotherapy, occupational therapy, midwifery, health science, and social work.”  The authors refer to this as interprofessional education, “learning that represents a way of fostering collaborative and seamless, integrated patient-care education.”  I first heard about this approach from my POD Network colleague Marilla Svinicki, who is involved in interprofessional education at the Clinical Education Center affiliated with the University of Texas-Austin.  The CEC is an impressive initiative.

    I was interested to hear how clickers would play out in this setting, one featuring students with a diverse set of backgrounds and career goals.  However, the course is a first-year course, so the students weren’t likely to have differentiated themselves yet.  Moreover, there’s no attention paid to these different majors in the survey results that are reported here.  (The survey results are very positive, however, and are in keeping with other surveys I’ve mentioned here on the blog before.)

    Have you used clickers in a course that included groups of students from different majors?  I can imagine forming heterogeneous student groups, then giving each group a single clicker as part of small-group activities during class.  How would you teach a course like this?

    Update: Just a couple of days after posting this, I learned that Vanderbilt University has an interprofessional program something like the one at UT-Austin.  The Vanderbilt program involves medical and nursing students from Vanderbilt and social work students from Tennessee State University.  The students work together (with mentors) in clinical settings half a day per week and participate in classroom-based learning (using reflective exercises and case study activities) half a day a week, as well.  The rest of the week they participate in their respective programs as normal.  Given the complex nature of health care today, this seems like an incredibly sensible approach to health professions education.

    Image: “Colourful Army” by Flickr user maistora / Creative Commons licensed

    More from my round-up of articles on clickers in the health professions.  A short, but interesting post today.  Your comments are invited…

    Reference: Zurmehly, J., & Leadingham, C. (2008). Exploring student response systems in nursing education. CIN: Computers, Informatics, Nursing, 26(5), 265-270.

    Notes: This short article is another introduction to teaching with clickers, although I found it a little too prescriptive for my tastes.  There’s nothing here you won’t find in other articles with one very interesting exception:

    To date, there has been no evidence of hacking or compromise to the SRS systems that were evaluated.  As a safeguard against tampering, a computer printout of responses can be generated and saved, to be used as a record for future references and to check for any attempted manipulation of grades.

    Wow!  I’ve never heard this worry before.  Have you had to worry about students hacking into their clicker grades?

    Image: “Me on Computer” by Flickr user Brian Lane Winfield Moore / Creative Commons licensed

    More from my round-up of articles on clickers in the health professions…

    Reference: Kenwright, K. (2009). Clickers in the classroom. TechTrends, 53(1), 74-77.

    Notes: This short paper from Kathy Kenwright (University of Tennessee Health Science Center) serves as a concise introduction to teaching with clickers, complete with a brief review of the literature.  As with Cain and Robinson (2008), the lit review isn’t comprehensive, but Kenwright does a good job of discussing the major benefits of clickers in the context of reported studies.  Most of her observations are not specific to any one discipline.  For example, she notes that clickers facilitate formative assessment of student learning, as well as agile teaching.  She mentions the importance of the display of results of a clicker question and the use of clickers to facilitate in-class quizzes on pre-class readings.

    I have concerns about a couple of Kenwright’s recommendations, however.  She notes that many students in the health professions must pass national board exams, and uses this to support her claim that one shouldn’t ask too many clicker questions during class.

    Asking too many questions during the lecture leaves less time to convey important content.  In a curriculum such as the Clinical Laboratory Science program, there is a defined body of knowledge that must be delivered to the students.

    She’s speaking of a coverage model of education here, which is problematic, as I’ve mentioned here before.  I would argue that since students will be required to excel at the multiple-choice questions seen on these national board exams, they should spend plenty of class time practicing these kinds of questions.  Clicker questions based on these exam questions work well for that.

    Kenwright also notes that asking clicker questions “on the fly” during class can take too much class time:

    If they are added during class the class will be kept waiting while the instructor is typing in the question and answer choices… There is nothing wrong with reverting to an old-fashioned show of hands, or calling on a particular student for an answer.

    Asking “on the fly” questions doesn’t require you to type questions into your clicker system–asking them verbally usually does the trick.  Moreover, if there was nothing wrong with a show of hands, there wouldn’t be any reason to use clickers to begin with.  Why are clickers better than a show of hands?  Because students don’t answer questions independently when you go with a show of hands (Stowell and Nelson, 2007).

    What’s your view on the coverage issue?  Is a lot of active learning possible in health professions education?

    Image: “Stethoscope” by Flickr user vitualis / Creative Commons licensed

    Back in January I gave a keynote talk at the Health Professionals Education Research Symposium hosted by Nova Southeastern University.  Part of my preparation for that talk included reading some of the articles from related disciplines in my clickers bibliography.  Shortly after the conference, I blogged about one great article about using clickers to promote critical thinking in nursing (Debourgh, 2008), and I’ve been meaning to post some notes about the other articles I read.  Let’s get started…

    Reference: Cain, J., & Robinson, E. (2008). A primer on audience response systems: Current applications and future considerations. American Journal of Pharmaceutical Education, 72(4), 77.

    Notes: The literature review is the highlight of this article.  It’s not as comprehensive as other lit reviews, but it does a great job of describing a few studies of the use of clickers in the health professions with particularly positive results.  For example, Slain et al. (2004) report that students in clicker sections of two pharmacy courses scored significantly higher on exams than students in non-clickers sections.  Similar results were found by Schackow et al. (2004) in classes for family medicine residents and by Pradhan, Sparano, and Ananth (2005) in classes for obstetrics and gynecology residents.  These references are listed in my bibliography.  Hopefully, I’ll find some time to read and blog about them soon.

    Cain and Robinson also include a useful exploration of some of the logistical aspects of teaching with clickers.  Instead of making recommendations, they describe the various choices a department might make and their pros and cons.  They note that any clickers initiative should make sense given an institutions teaching philosophy and technology plan.

    For example, a pharmacy school with a mandatory laptop program may highly value an ARS that can utilize laptops as response devices, rather than basing the decision on other features.

    They also recommend purchasing a set of clickers available to faculty and staff to check out for one-shot events, like continuing education programs and faculty meetings.

    The section on recommendations for future research is a strong one.  Cain and Robinson write, “Any effects from using an instructional medium do not come from the use of the media itself, but from the instructional methods employed.”  That’s something I’ve argued here before.  Cain and Robinson call for research that explores the effects of very particular instructional strategies involving clickers, including strategies useful for facilitating discussion about matters of ethics and morality.  While ethical issues are present in every discipline, they are often particularly important in professional education.

    Cain and Robinson make an interesting statement in their section on student considerations: “Finally, appropriate application of the ARS in the curriculum should be defined and encouraged.”  I understand the interest in encouraging instructors to use clickers in appropriate ways.  It’s the “defining” piece that makes me wonder if pharmacy education is a bit more top-down than the kinds of programs you find in, say, colleges of arts and science.  I find that faculty members in undergraduate liberal arts departments tend to have a high degree of autonomy when it comes to their teaching decisions.  They might not be comfortable having appropriate uses of clickers “defined” for them.  Am I reading too much into this word choice?  Does your department (whatever your discipline) set policy on educational technology use?

    Image: “Rx, San Antonio, TX” by Flickr user Tadson / Creative Commons licensed

    I’ll be giving the Sunday evening keynote at the Great Lakes Conference on Teaching and Learning hosted by Central Michigan University on May 23-25, 2010.  The keynote will address methods for increasing student participation and engagement in the classroom, including, but not limited to clickers.  Here’s the abstract:

    Class Time Reconsidered: Motivating Student Participation and Engagement

    Whether you have 20 students in your class or 200, motivating students to engage meaningfully with course material during class can be a challenge.  In fact, motivating students to come to class at all can sometimes be tough.  How can instructors make their lectures more dynamic?  What in-class activities help students grapple with tough questions-and which of these scale up well to large classes?  What out-of-class activities can prepare students to participate more intentionally during class?  In this keynote, we’ll explore some ways to rethink what you do-and what you have your students do-during class with the goal of increasing student attendance, participation, and engagement.

    I’ll also be leading a concurrent session the next morning that will indeed focus entirely on teaching with clickers.  Here’s the abstract for that:

    Teaching with Clickers: Engaging Students with Classroom Response Systems

    Classroom response systems (“clickers”) are technologies that enable teachers to rapidly collect and analyze student responses to multiple-choice questions during class.  These systems can be used in a variety of ways to engage students in learning, particularly in large classes.  In this session, we’ll explore the kinds of questions and activities that make the most of these systems, including ways to foster small-group and classwide discussion, turn quizzes into learning experiences for students, practice more “agile” teaching, and make class time more enjoyable.

    If you’re anywhere near Michigan, I encourage you to come to the conference!

    Image: “Breaking Waves” by Flickr user Tom Gill / Creative Commons licensed

  • Filed under: Conference
  • In spite of including experiences from not one, but two language instructors in my book, I still haven’t found any studies exploring the use of clickers in language classrooms for my bibliography.  And, if you check out the column to the right of this post, you’ll see the various disciplines I’ve covered here, and language instruction is not well represented.  (This very post will be only the second in that category.)  Since I’m pretty sure clickers have incredible potential in language instruction, you can imagine how glad I was to see a recent blog post about clickers in a Spanish class at Georgetown University!

    The post is a report from Ellen Johnson, a PhD student in applied linguistics, who teaches and coordinates Spanish courses at Georgetown.  After hearing about clickers at a workshop hosted by Georgetown’s Center for New Designs in Learning and Scholarship (CNDLS), Johnson experimented with clickers in her language classroom.  Not only did she experiment with clickers, she collected some data useful in helping her judge their effectiveness.  Here’s what she did:

    In a nutshell, 58 students enrolled in Beginning Spanish courses participated in the study on ser and estar. They were introduced to their uses in context, practiced answering questions using clickers with around 20 slides while viewing their performance in relation to their peers, and then completed posttests and reflection questionnaires.

    Johnson also had colleagues observe the classes, and the feedback from both students and instructors about the use of clickers was very positive.  The students were particularly enthusiastic about getting immediate feedback on their learning, and Johnson’s fellow instructors thought the clickers had potential for helping them target their feedback to their students “in a more coherent manner.”

    Students raised a couple of concerns in their feedback, however.  They thought that the clicker questions made it difficult for them to take notes during class.  Making clicker questions available to students after class is something I’ve mentioned here before, and there’s a little evidence that doing so is, in fact, very important since it allows students to review clicker questions later.  Knowing that clicker questions will be available online after class also frees students from having to take as many notes during class, which is likely to help them spend more time actually thinking during class.

    Johnson’s students also noted that clicker questions don’t allow students to practice their speaking skills in a language class.  That’s a good point, but given the experiences of the language instructors I interviewed for my book, it would seem that clicker questions work very well for listening, reading, and writing skill development.

    The main concerns raised by language instructors in Johnson’s study were logistical ones.  They worried that the technology would be difficult to start using or might fail during class.  I don’t know what system they use at Georgetown, but I know there are easy-to-use and reliable systems out there.  Also, Georgetown doesn’t seem to have a full-scale clicker implementation, one where students could be expected to purchase clickers at their bookstore, as is the case at many US colleges and universities.  That creates a logistical barrier, as well, since clickers would have to be distributed and collected each class session.

    Thanks to Ellen Johnson for sharing her experiences with clickers.  I would be interested to hear more uses of clickers in language courses.  What kinds of questions and activities work well with clickers in those settings?  And why do you think that clickers aren’t more widely used in language instruction?

    (I should also note that Ellen Johnson’s post appeared on a group blog from a group of instructors at Georgetown exploring the use of clickers this spring.  Take a look at previous blog posts for more interesting discussion of teaching with clickers.  This “community of practice” is another example of the value of fostering discussions about teaching and learning across the disciplines.)

    Image: “Pink AC Bienvenidos” by Flickr user lopolis / Creative Commons licensed


    Recent Comments