Teaching with Classroom Response Systems

Resources for engaging and assessing students with clickers

Archive for the ‘Conference’ Category

If you’re heading to the Turning Technologies User Conference at Harvard University in October or the Lilly Conference at Miami University in November, you’ll see me there! I’m delivering keynote talks at both conferences. Abstracts can be found below.

Connecting with Participatory Culture: Clickers and Deep Learning

Turning Technologies User Conference
Harvard University
October 10, 2010

Today’s students vote for their favorite contestants on American Idol, “like” a friend’s wall post on Facebook, comment on news and events on Twitter, and engage in robust online discussions about World of Warcraft. We live in a participatory culture, one in which voting, commenting, creating, and sharing are the norm and people prefer being contributors to being consumers. Teaching with clickers is one way to tap into this culture, engaging students in ways that motivate them to participate during class in meaningful ways. In this talk, Derek Bruff will explore ways that using clickers connects with our students’ participatory culture and how those connections can be leveraged to promote deep learning.

Clickers and Backchannel: Engaging Students with Classroom Response Systems

Lilly Conference on College Teaching
Miami University of Ohio
November 18, 2010

Classroom response systems offer instructors useful options for encouraging student participation and engagement during class. For instance, “clickers” enable instructors to rapidly collect and analyze student responses to multiple-choice questions. Backchannel conversation tools can help more students share questions and ideas with instructors during class and provide new venues for in-class discussion among students. In this session, we’ll explore the kinds of activities and questions that make the most of these systems, including ways to foster small-group and class-wide discussion, turn quizzes into learning experiences for students, practice more “agile” teaching, and make class time more enjoyable.

Image: “Suitcase” by Flickr user EssjayNZ, Creative Commons licensed

  • 0 Comments
  • Filed under: Conference
  • Heads and TailsContinuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

    “Using Prediction and Classroom Voting via Clickers to Address Students’ Overreliance on the Representativeness Heuristic,” Tami Dashley, University of Texas-El Paso [Slides]

    Tami Dashley is a graduate student in math education and a student of Kien Lim, one of the organizers of the contributed paper session. She shared some of her thesis research, an investigation into the connection between classroom voting with clickers and certain misconceptions students have about probability. Her work focuses on the representativeness heuristic, which she defines as “determining the likelihood for events based on how well an outcome represents some aspect of its parent population.”

    Tami gave the following example: Suppose you toss a coin six times, getting a sequence of heads (H) and tails (T). Which of the following is more likely to occur: TTHHTH or HTTHHH? Someone using the representativeness heuristic would say that TTHHTH is more likely to occur since it includes an equal amount of heads and tails, just like the coin does. The other option includes more heads than tails, so it would not seem as likely to someone using the representativeness heuristic. Actually, both of those outcomes are equally likely (each occurring with probability 1/64), so the representative heuristic is a misleading one in this example.

    The issue is that the representativeness heuristic is useful in some cases, but not useful in all cases. The misconception that many students have is that it’s always useful.

    How to help students stop over-relying on the representativeness heuristic? Tami has been investigating the use of prediction questions, ones that ask students to predict an outcome or probability without actually computing anything. For example, students might be asked to determine which of several outcomes is most likely to occur. Since students need not be as precise when responding to prediction questions, they have some cognitive processing power freed up to focus on concepts. Clicker questions are a natural match here, since they allow students to commit to their predictions and compare their predictions to those of their peers. Then discussion of the incorrect answer choices provides an opportunity to deal with misconceptions.

    Tami conducted her research in a high school setting, using three groups of students. Her “control” group received a lesson exploring the representativeness heuristic that didn’t ask the students to predict any probabilities. A second group was asked several prediction questions but didn’t use clickers to respond to the questions. The third group used clickers to respond to prediction questions during the lesson. Tami used pre- and post-tests to determined the efficacy of these three different lessons.

    Tami found that her “control” group did pretty well on the post-test compared to the two experimental groups. However, most of their success came from what she called a “learned response.” In this case, many of the students picked up on the fact that “all of the above outcomes are equally likely” is often the correct answer to questions exploring the representativeness heuristic. (These are what students might call trick questions!) When Tami looked at performance on questions where “all of the above outcomes are equally likely” was, in fact, not the correct answer, the prediction-with-voting group performed better than the control and prediction-only groups.

    I was very impressed with Tami’s research design and the subtlety with which she explored student misconceptions in this teaching context. I don’t believe that Tami has published this work yet, but I look forward to reading it when she does.

    Image: “Heads and Tails” by Flickr user canonsnapper, Creative Commons licensed

    The summer meeting of the American Association of Physics Teachers (AAPT) wraps up in Portland, Oregon, today. There were several talks on teaching physics with clickers at the meeting, including one by Ian Beatty of the University of North Carolina at Greensboro physics education research group. Ian was the subject of my first podcast interview, and he’s been doing great work helping science instructors at the K12 and post-secondary levels teach effectively with clickers.

    In Ian’s presentation, he identified and addressed several common concerns instructors express about teaching with clickers. For each concern, Ian identifies a belief about teaching and/or learning that likely underlies the concern, as well as an alternate belief that can be adopted to address the concern productively. Ian also includes some practical strategies and example clicker questions for each of these alternate beliefs.

    For example, when many instructors hear about teaching with clickers, they’re concerned with having sufficient class time to cover what they need to cover in their courses given the time required by having students discuss and respond to clicker questions. Ian notes that this concern is likely a result of the following belief: “I must explicitly address in class everything students will be held accountable for.” Ian then presents an alternate perspective on this idea: “I can use class time to focus on core ideas and big-picture understanding, and charge students with filling in the details outside class.” This alternate perspective is, perhaps, non-intuitive to many instructors, but it’s a reasonable and useful perspective to have. Adopting this perspective leads to a shift from what Ian calls an understanding of class as a place to present content to an understanding of class as a place to help students digest content. Ian then shares five tips and techniques for implementing this shift in the classroom.

    Ian addresses other concerns in a similar manner, including concerns about having enough time to write good clicker questions, concerns about poor student participation during class, and concerns about changing one’s teaching style. His visuals, which use the online presentation tool Prezi, are included below and are well worth checking out.

    Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

    “Preservice Elementary Teachers’ Perceptions of Clicker Use in their College Mathematics Course,” Travis K. Miller, Millersville University of Pennsylvania [Slides]

    In my last post, I mentioned that Janet White first used clickers in her courses for pre-service teachers at Millersville University of Pennsylvania. Another speaker in the contributed paper session back in January was her colleague, Travis Miller, who shared results of a student survey he conducted in the pre-service teacher course he taught. Travis used clickers for only six lessons during that course in each of the four sections he was teaching. His clicker questions weren’t graded, and he followed the “classic” peer instruction model each time, having students vote individually, then discuss the question in small groups, then vote again.

    Travis’ students overwhelmingly (96%) liked using clickers in the course. Travis mentioned that there are very few things he does as a teacher that are as uniformly popular with his students! Almost as many students (89%) believed that the clicker activities helped them learn the material in those six lessons. Travis drilled down on this, asking students to say why the clickers were useful. The number one answer (59% of students) was that the clicker questions provided students with an opportunity to discuss and think about course content. The number two answer (23%) was that the clickers provided a sense of accountability and involvement.

    Travis didn’t stop there, either. He asked his students which topics they understood better because of the clicker activities. Of the six topics that Travis addressed using clickers, sets and Venn diagrams was cited by 52% of the students as the one that most benefited from clickers. Numeration / base arithmetic was a distant second with 15%, and deductive reasoning came in third with 13%. When sharing these data, Travis floated a very interesting hypothesis. He wondered if the fact that the number one topic (sets and Venn diagrams) was a visual one led to the students selecting it as most benefited by clicker questions. I’m a big fan of visual thinking, so this comment caught my attention. Is there something special about peer instruction with clickers and visual thinking?  I’d appreciate your thoughts in the comments.

    Travis’ other interesting hypothesis was that his more competitive students liked the competitive aspects of clickers (being the first to answer, answering correctly more frequently than other students, and so on), while the non-competitive students didn’t mind those aspects since they were essentially opt-in. That is, the students who didn’t want to compete could still participate fully with the peer instruction and voting process without feeling any pressure to treat it like a game. Graham, Tripp, Seawright, & Joeckel (2007) found that most students who are hesitant to participate in class liked clickers as well as those who were fine with participating, but I don’t think I’ve seen any research that compared competitive students with non-competitive students. That would make for an interesting research question.

    Travis also taught some sections of his pre-service teacher course without using clickers, and he surveyed students in these sections about the potential advantages and drawbacks of using clickers. What concerns did they have about using clickers? They worried about the cost of the devices, that clickers weren’t necessary in small classes, that clicker activities take up too much class time, and that the technology might not be reliable. I found it interesting that these are among the common concerns of faculty members not already using clickers, too!

    Image: “Happy Pi Day!” by Flickr user Mykl Roventine / Creative Commons licensed

    Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

    “Using Personal Response Systems (Clickers) in Liberal Arts Mathematics Courses to Support a Lecture Format,” Janet A. White, Millersville University of Pennsylvania [Slides]

    Just like Jean McGivney-Burelle and Kimberly Burch, Janet White shared her experiences teaching with clickers in a “liberal arts” mathematics course taken by non-majors. Unlike Jean and Kimberly, who teach relatively small sections of this kind of course, Janet teaches in a large lecture hall with 75 students per section. Janet had used clickers in courses for pre-service math teachers in the past and found them useful, so when it was her turn to teach this larger course, she decided to use them again. A classroom response system was hardly the only technology Janet used in this course: She also had students complete online homework and quizzes and she annotated her PowerPoint lecture slides using an Interwrite Mobi.

    Janet used clickers on a daily basis in her course, usually either to assess students’ prior knowledge or to assess their understanding of a topic taught during lecture.  Her questions came from a bank of multiple-choice questions provided by her textbook publisher.  She counted the clicker questions as part of her students’ participation grades, but in a low-stakes manner.  Given her use of the questions as well as the source of the questions, many were on the lower levels of Bloom’s taxonomy, aimed at recall and application of procedural knowledge.  She shared an example of a prior knowledge question that asked students to find the measure of an angle that complements a 36 degree angle.  A slightly harder question aimed at assessment of something taught during the course asked students to identify the cut edge in a given graph (or to assert that the graph had no cut edge).

    Student survey results indicated that 85% of Janet’s students who used clickers regularly liked using them, and 71% said that using clickers helped them learn the material.  Students who used clickers regularly during the course ended up with higher grades in the course than students who didn’t, but, of course, that can’t necessarily be attributed to the use of the clickers.  (And since clicker questions were factored in the course grade, students who participated more frequently in clicker questions would almost certainly have higher grades in the course anyway.)

    Student comments about the clickers were generally positive.  My favorite one was, “I liked getting the wrong answer anonymously.”  Other comments addressed the usual points that students like about clickers: They liked the interactivity, they liked discussing questions with classmates, they liked seeing where they stood relative to their peers, and they liked the feedback on their own learning the clicker questions provided.  The only significant negative aspect for the students was the cost, about $50 in Janet’s case.

    Janet found that having students discuss clicker questions in small groups led to very engaged students, even in the large auditorium environment.  In the future, she plans to write more of her own questions, instead of relying on ones from the textbook’s question bank.  She hopes to write more difficult questions that will generate even more engaged discussion during class.  She’s also hoping to find ways to reduce the technology cost to the students, either by selecting a different vendor or facilitating the resale of clickers after each semester to students taking the course the next semester.

    Also, Janet mentioned that the earth science faculty at Millersville are big users of clickers.  Earth science instructors looking for advice on using clickers might want to investigate!

    Image: “Recursive Daisy” by Flickr user gadl / Creative Commons licensed

    Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

    “Clickers in the Classroom,” Kimberly J. Burch, Indiana University of Pennsylvania [Slides]

    Kimberly teaches a “Math 101″ survey course called “Foundations of Mathematics.”  Topics covered include set theory, graph theory, and counting methods (among others), and Kimberly shared several interesting clicker questions on each of these topics.  For example, here’s one of her questions from the unit on graph theory:

    How many vertices are there in a tree with 19 edges?

    1. 19
    2. 18
    3. 20
    4. Not enough information given

    Kimberly practices the “classic” peer instruction technique of having students vote individually first, then discuss the questions in small groups, then vote again.  She finds that students often converge to the correct answer on the second vote.

    In the example above, her students were split between 18 and 20 on the first vote, but after the peer discussion time, most students went with the correct answer, 20.  I found this interesting because the “Not enough information given” seemed to be the obvious wrong answer to this question.  A graph with 19 edges might have any number of vertices, but a tree with 19 edges can only have 20 vertices.  Students who don’t realize that trees are graphs with very specific properties might be tempted to go for the “Not enough information given” option.

    I suspect that Kimberly used this question after the students learned the relationship between the number of edges and number of vertices in a tree and that this question was meant to assess whether students remembered that relationship.  Some students likely remembered that one of these numbers was one more than the other but weren’t sure which one was higher.  That would account for the split vote between 18 and 20.  Had this question been asked as an exploratory question and not a review question, I’m betting the split would have been between 20 and “Not enough information given.”

    Kimberly also mentioned that she uses her clicker system’s priority ranking questions to have her students decide what topics should be emphasized during exam review sessions.  Kimberly gives her students a list of 8-10 exam topics, and the students indicate the top three or four toughest topics in order.  Kimberly said that this helps her make good use of limited exam review time by focusing on the topics the students find the most difficult.

    Kimberly also shared some data from a quasi-control group experiment she conducted.  She taught two sections of this survey course and alternated which topics she covered with clickers in the two sections.  For example, section A might cover topic 1 with clickers while section B covered topic 1 without.  Then for topic 2, section B used clickers and section A didn’t.  She then compared test scores for the two sections by topic.  For some topics, students using clickers performed better on exams but for other topics, the students not using clickers performed better.  And for other topics, there was no difference.  The data was generally favorable to using clickers, but the “quasi” part of this quasi-control group experiment made it difficult to draw firm conclusions.

    Image: “Point Marian Bridge” by Flickr user timmenzies / Creative Commons licensed

    Deep Down Inside, We All Love Math

    File this under “Better Late Than Never.”  Back in January 2010, I coordinated (with Kelly Cline and Kien Lim) a contributed paper session on teaching with clickers at the Joint Mathematics Meetings in San Francisco.  Shortly after the conference, I blogged about some clickers talks that didn’t fall in our session, but I never got around to blogging about the talks in our session!  Five months later, I’m finally getting around to sharing my notes from those talks…

    “The Evolution of Classroom Voting in Contemporary Mathematics,” Jean McGivney-Burelle, University of Hartford [PowerPoint Slides]

    Jean teaches a “math for the liberal arts” course called Contemporary Mathematics taken by music and arts majors among others.  She finds her students come to the course with relatively little interest or self-reported ability in mathematics, so it’s a tough crowd to teach.  A few years ago, she started teaching with clickers in order to appeal to what she calls the “thumb generation”–students used to spending a lot of time sending text messages.

    Jean interspersed some clicker questions throughout her lectures and encouraged students to discuss them in small groups before voting.  She and the students liked this, but she found that most of her questions were answered correctly by most of her students  and that the small group discussions didn’t involve much debate among students.  The next year, Jean decided to ask tougher questions.  She calls them QEDs–Questions to Encourage Discussion.  She aimed for the analysis, synthesis, and evaluation levels of Bloom’s Taxonomy.

    For example, in Jean’s first year using clickers she gave her students a preference schedule–a list of how each voter in an electorate (only four of them to keep things simple) ranked all of the candidates.  She then asked her students to determine which candidate would win the election using the instant run-off voting scheme.  This is a straight-forward application of a particular algorithm.

    The next year, Jean asked another question in which she shared a preference schedule with her students and asked them to analyze it.  However, this time, she asked the question at the beginning of the unit on voting schemes and asked her students to indicate which of the candidates had the best case for winning the election.  There’s no single correct answer to this question since winner of an election (well, one involving at least three candidates) depends on what scheme you use to count the votes.  This is a great example of a one-best-answer question (since students are asked to select the one answer they think is best among multiple reasonable answers) used to create a time for telling (since it’s used to make the point that which voting scheme you use matters).

    Jean found that these more challenging and ambiguous questions generated longer and more engaged small group discussions as well as more “horizontal” bar graphs–ones indicating significant disagreement among the students.  Looking ahead, she plans to build on this success by writing questions designed to develop mathematical habits of mind–an important goal of this course.  For example, here’s a sample question she shared aimed at pointing students towards the notions of proof and counterexamples:

    Suppose there is a majority winner in an election. Will all of the voting methods we have studied thus far always pick that winner?  Yes or no?  If you answer yes, prepare to defend your answer. If you answer no, have a counterexample ready.

    I really like this question.  It has a degree of ambiguity that students often find disconcerting, but it also reminds students of how they’ll need to defend their answers, which should help put their minds at ease.  As Jean noted in her talk, in a course like this one, it’s more important students develop mathematical habits of minds (like the ones surfaced by this question) than learn particular math content areas.  I hope this kind of question helps with this objective.

    Stay tuned to the blog for more notes on these talks over the coming days…

    Image: “Deep Down Inside, We All Love Math” by Flickr user Network Osaka / Creative Commons licensed

    I’ve scheduled this post to appear on the blog just as I’m starting my keynote at the University of Louisville clickers conference in Louisville, Kentucky.

    • For those of you not at the conference, you can get a sense of what I’m talking about right now by checking out my Prezi below.  You’re welcome to weigh in on Twitter about these ideas.  Just tag your tweets with #ULclickers so I’ll see them.
    • For those of you at the conference, you’ll find below links to a few resources mentioned in my talk.  Feel free to explore these after the keynote!  (Or during… I’m cool with that.)

    My talk is titled “Connecting with Participatory Culture: Clickers and Deep Learning.”  Here’s the abstract:

    Today’s students vote for their favorite contestants on American Idol, “like” a friend’s wall post on Facebook, comment on news and events on Twitter, and engage in robust online discussions about World of Warcraft.  We live in a participatory culture, one in which voting, commenting, creating, and sharing are the norm and people prefer being contributors to being consumers.  Teaching with clickers is one way to tap into this culture, engaging students in ways that motivate them to participate during class in meaningful ways.  In this talk, Derek Bruff will explore ways that using clickers connects with our students’ participatory culture and how those connections can be leveraged to promote deep learning.

    And here’s my Prezi:

    Finally, some relevant resources:

    Like Buttons / Student Perspective Questions

    • Matthew Freeman’s perspective questions come from this article: Campt, D., & Freeman, M. (2009). Talk through the hand: Using audience response keypads to augment the facilitation of small group dialogue. The International Journal of Public Participation, 3(1), 80-107.  Here’s my summary.

    Text-to-Vote / Peer Assessment Questions

    • A description of Kori Street’s use of clickers for peer assessment can be found on pages 94-96 of my book.

    Serious Fans / Misconception Questions

    Event TV / Critical Thinking Questions

    Volunteerism

    For more on the notion of a participatory culture, read Henry Jenkins’ white paper, “Confronting the Challenges of Participatory Culture: Media Education for the 21st Century” [PDF].  Also, here’s my blog post that got me started thinking along these lines, the one that references the Campt and Freeman article.

    What are your thoughts on the ideas in my keynote?  Do we, especially our students, live in a participatory culture?  What consequences does that have for how we teach?

    Image: “skates” by Flickr user marythom / Creative Commons licensed

    Sunday night, I delivered the opening keynote at Central Michigan University’s Great Lakes Conference on Teaching and Learning.  My presentation was titled “Class Time Reconsidered: Motivating Student Participation and Engagement.”  My goal was to share some frameworks and strategies for engaging students in the classroom by taking a few common assumptions about teaching and learning and flipping them on their heads.  Here’s my Prezi, complete with much flipping of things on their heads:

    Some thoughts on the presentation:

    One of my first clicker questions asked participants to identify a key challenge in motivating students to engage meaningfully during class.  Strangely, the even-numbered answer choices were by far the most popular-students are hesitant to speak up in front of their peers, students focus too much on grades and not enough on learning, and students don’t prepare adequately for class.  These results worked well for me, since I had been planning on addressing ways to reach students who are risk-averse or grade-focused and ways to motivate students to prepare for class in useful ways.

    Participants engaged in a Think-Pair-Share activity in which they tried to identify six steps in a typical process their students might undertake to learn in their course.  This followed an introduction to the idea of a “time for telling,” so I asked participants to make sure that “telling” wasn’t the first step on their lists.  I also encouraged participants to force themselves to come up with six steps.  Coming up with three-step plans (take notes during class, figure things out in the homework, regurgitate on exams) is too easy.  Identifying a six-step process means you have think a little more intentionally about how your students learn.

    Given the clicker question results indicating that lack of student preparation is a big challenge, we camped out for a while on the idea of a pre-class assignment.  I made two important points about these assignments: they should be graded, if only on effort, so that students will take them seriously and you should make use of these assignments in some way during class.  Otherwise students will see them as busywork, not connected with the “real” work of the course.  One participant shared her approach-she has students create outlines of their pre-class readings, then share and compare their outlines in small groups during class.

    Monday morning (the day after my presentation), I saw on the book raffle table that there’s a new book on Just-in-Time Teaching, Just in Time Teaching: Across the Disciplines and Across the Academy edited by Scott Simkins and Mark Maier (Stylus, 2010).  I wish I had known that Sunday night-I would have mentioned it during that section of my presentation!

    My third and final clicker question asked participants to identify one of five in-class engagement strategies they wanted to try soon.  While I wasn’t intending the presentation as a pitch for clickers, perhaps my biases couldn’t be hidden-clickers was the number one answer!  This result might have also been because clickers are new and different, but not so different as to require a complete rethinking of one’s teaching approach.  I’m convinced that the return on investment for teaching with clickers is high-one can make small changes in one’s teaching methods that yield significant results.

    At the end of the presentation, I had the participants generate questions for me at their tables.  Most of the tables had at least one person with a Web-enabled device (such as the iPads several of the CMU staff hosting the event were sporting).  They used these to submit their tables’ questions via Google Moderator.  I asked them to vote on other tables’ questions, as well, providing me with a ranked list of the most popular questions.  This served as a reasonable demo of Google Moderator as a backchannel tool, but unfortunately I didn’t have time to address the questions that emerged through this process.  My plan is to address the more popular questions with Google Moderator since, as the creator of this Moderator session, I can leave comments on individual questions.  You can see the questions submitted by the group here.

    The conference continues through Tuesday morning.  I was able to attend most of the conference on Monday, and I live-tweeted a couple of the sessions.  You can read my tweets here.  Joy Mighty of Queen’s University in Ontario delivered the Monday lunch keynote, and she made a strong case that by not paying attention to matters of diversity in our classroom, we run the risk of fostering inequity.  It was a thought-provoking keynote for me.

    Thanks to Central Michigan University for having me as part of their conference and for some great conversations about student engagement!

    I’ll be giving the Sunday evening keynote at the Great Lakes Conference on Teaching and Learning hosted by Central Michigan University on May 23-25, 2010.  The keynote will address methods for increasing student participation and engagement in the classroom, including, but not limited to clickers.  Here’s the abstract:

    Class Time Reconsidered: Motivating Student Participation and Engagement

    Whether you have 20 students in your class or 200, motivating students to engage meaningfully with course material during class can be a challenge.  In fact, motivating students to come to class at all can sometimes be tough.  How can instructors make their lectures more dynamic?  What in-class activities help students grapple with tough questions-and which of these scale up well to large classes?  What out-of-class activities can prepare students to participate more intentionally during class?  In this keynote, we’ll explore some ways to rethink what you do-and what you have your students do-during class with the goal of increasing student attendance, participation, and engagement.

    I’ll also be leading a concurrent session the next morning that will indeed focus entirely on teaching with clickers.  Here’s the abstract for that:

    Teaching with Clickers: Engaging Students with Classroom Response Systems

    Classroom response systems (“clickers”) are technologies that enable teachers to rapidly collect and analyze student responses to multiple-choice questions during class.  These systems can be used in a variety of ways to engage students in learning, particularly in large classes.  In this session, we’ll explore the kinds of questions and activities that make the most of these systems, including ways to foster small-group and classwide discussion, turn quizzes into learning experiences for students, practice more “agile” teaching, and make class time more enjoyable.

    If you’re anywhere near Michigan, I encourage you to come to the conference!

    Image: “Breaking Waves” by Flickr user Tom Gill / Creative Commons licensed

  • 2 Comments
  • Filed under: Conference
  • Categories

    Recent Comments