The summer meeting of the American Association of Physics Teachers (AAPT) wraps up in Portland, Oregon, today. There were several talks on teaching physics with clickers at the meeting, including one by Ian Beatty of the University of North Carolina at Greensboro physics education research group. Ian was the subject of my first podcast interview, and he’s been doing great work helping science instructors at the K12 and post-secondary levels teach effectively with clickers.

In Ian’s presentation, he identified and addressed several common concerns instructors express about teaching with clickers. For each concern, Ian identifies a belief about teaching and/or learning that likely underlies the concern, as well as an alternate belief that can be adopted to address the concern productively. Ian also includes some practical strategies and example clicker questions for each of these alternate beliefs.

For example, when many instructors hear about teaching with clickers, they’re concerned with having sufficient class time to cover what they need to cover in their courses given the time required by having students discuss and respond to clicker questions. Ian notes that this concern is likely a result of the following belief: “I must explicitly address in class everything students will be held accountable for.” Ian then presents an alternate perspective on this idea: “I can use class time to focus on core ideas and big-picture understanding, and charge students with filling in the details outside class.” This alternate perspective is, perhaps, non-intuitive to many instructors, but it’s a reasonable and useful perspective to have. Adopting this perspective leads to a shift from what Ian calls an understanding of class as a place to present content to an understanding of class as a place to help students digest content. Ian then shares five tips and techniques for implementing this shift in the classroom.

Ian addresses other concerns in a similar manner, including concerns about having enough time to write good clicker questions, concerns about poor student participation during class, and concerns about changing one’s teaching style. His visuals, which use the online presentation tool Prezi, are included below and are well worth checking out.

I was quoted this morning in “At Universities, Is Better Learning a Click Away?“, an Associated Press story on the future of classroom response systems by AP reporter Eric Gorski.  The story features Michael Dubson, who teaches physics with clickers at the University of Colorado-Boulder.  CU-Boulder, and its physics education research group in particular, has been very active in the world of clickers (including contributing to these great videos), and I was glad to hear Michael Dubson’s perspectives on the technology in the AP piece.

CU-Boulder is an i>clicker campus, and Dubson makes the case in the AP story that a simple, dedicated clicker device is preferable in most instances to more flexible systems based on smart phone apps.  Indeed, i>clicker devices have only six buttons–an on/off button and buttons labeled A, B, C, D, and E.  This is a very simple system, but, as inventor Tim Stelzer argued at the Louisville clicker conference back in 2008, multiple-choice questions with five answer choices work very well for the kinds of formative assessment and peer instruction many instructors use clickers to implement.

Gorski places me on the other side of a somewhat-artificial divide:

Derek Bruff, assistant director of Vanderbilt University’s Center for Teaching, said simple clickers are great at multiple choice questions. But he’s more excited about using smart phones, which allow students to ask questions of instructors, hold back-channel discussions with each other and respond in their own words.

Regular readers of this blog know that I’m definitely excited by the possibilities of using smart phones as “super-clickers” or to facilitate backchannel discussion in the classroom.  It’s true that I’m more excited by smart-phone systems than I am by simple clickers like i>clicker, but that’s largely because I’ve been involved in teaching with clicker with several years and I’m eager to leverage that experience to consider new kinds of technology-facilitated classroom dynamics.  (For one thoughtful perspective on those potential dynamics, consider Sean Seepersad’s recent post on moving away from clickers.  I hope to blog about Sean’s post soon!)

I’ve spent plenty of time thinking about the pedagogy of multiple-choice questions (while writing my book, blogging about clickers here, and giving talks on the subject around the country), and I think the multiple-choice format is often underrated.  I even have an article coming out (soon, I hope!) titled, “Multiple-Choice Questions You Wouldn’t Put on a Test: Promoting Deep Learning with Clickers.”  So I definitely get where Michael Dubson is coming from: Five-answer multiple-choice clicker questions are incredibly useful in all kinds of courses.

All this to say that one of the principles I attempted to uphold when writing my book was that everyone’s teaching context is different–different students, different disciplines, different institutions, different teaching styles and experiences.  I’m interested in helping instructors think more intentionally about their teaching choices, exploring the pros and cons of choices both traditional and innovative.  So while I may be more excited myself about smart phone systems, I always encourage instructors to select technologies and teaching practices that make the most sense in their particular teaching contexts.

I’m glad for clickers to receive the attention of the Associated Press.  The story has been all over Twitter today, and I hope it makes its way into print and online newspapers across the country.  And I’m glad that I could help Eric Gorski out as he was researching this story.  Eric also contributed to a short video piece to accompany his article, and he blogged about the story on the AP’s Facebook page.

Thoughts on the AP story?

Image: “The Nabla System (Forgotten Seed)” by Flickr user Syntopia

A math colleague of mine, who blogs under the name Doc Turtle, recently blogged about his use of a calculus worksheet that helps his students “guide themselves through the algebraically intense process of partial fractions.”  Doc Turtle reports that his students look forward to this kind of work, and he’s planning to develop more activities along these lines.

I’ve heard from several instructors who have students engage in this kind of active, self-directed learning in class (through worksheets, clicker questions, and so on) that some students complain that the professor isn’t doing any work.  I suspect that these are the students who expect to come to class, take a lot of notes, and figure the material out while working through their homework.  They can sometimes push back when their instructor isn’t presenting course content in the way they expect.

Of course, instructors who design and implement activities like Doc Turtle’s worksheet activity aren’t avoiding the hard work of teaching.  Instead, they’re being intentional about what they want their students to learn and they’re planning and facilitating experiences designed to help their students learn those things.

As Ian Beatty wrote over on his blog, “It’s not really creating [clicker] questions that’s tough.  The hard part is figuring out what I want my students to learn from the class, and casting that in terms of what I want my students to be able to do.”  Once he’s done that, he says it’s relatively easy for him to write effective clicker questions.  “Just formulate a question asking them to do that (in a particular context), and then much of the class activity is me helping them struggle through the process as they learn how.”

What struck me about Doc Turtle’s post was how excited his students are to engage in this kind of active learning.  As I mentioned above, not all students see this kind of learning as valuable.  Did Doc Turtle just get lucky with a batch of exceptional students?  I suspect not.  I’m guessing that he’s been teaching his students to learn this way since the first day of classes so that by this point in the semester, his students are perfectly willing to see this kind of activity as valuable.  I think that’s an important takeaway: If we’re asking our students to learn in a “new” way, then we need to help them learn how to learn in that way.

Do you find that your students push back when you ask them to engage in active learning in class?  How do you help them see the value in this kind of learning over time?

Reference: Crouch, C. H., & Mazur, E. (2001). Peer instruction: Ten years of experience and results. American Journal of Physics, 69(9), 970-977.

Summary: In this now-classic article, Catherine Crouch and Eric Mazur present data on ten years of the use of peer instruction in introductory physics courses.  Included is a description of Mazur’s teaching practices for these courses, including ConcepTests (multiple-choice questions that help students develop conceptual understanding independent of computational skills), pre-class reading quizzes (used to motivate students to read their textbooks before class, allowing Mazur to shift the transfer of information outside of class, freeing up classtime to work on the assimilation of information), and peer instruction with and without clickers.

For assessment, Crouch and Mazur compare student performance on pre- and post-tests–the Force Concept Inventory (FCI), a widely used multiple-choice test of conceptual understanding in first-semester physics–before Mazur began using peer instruction and after.  They use normalized gain as their metric, which is determined by the formula (post-pre)/(100%-pre).  Thus, if a student scores a 70% on the pre-test and an 80% on the post-test, their normalized gain is (80-70)/(100-70), which is approximately 0.33.  Another student who moved from a 90% to a 95% would have a gain of 0.5, indicating that the student gained 50% of the improvement s/he could have gained from pre-test to post-test.

Using normalized gain on the FCI as a metric enables Crouch and Mazur to make comparisons to national data.  In Richard Hake’s 6000-student study of “traditional” and “interactive” physics courses, the average normalized gain for students in traditional courses was 0.23, whereas the average for students in interactive courses was 0.48, a very significant difference.  The semester before Mazur started using peer instruction, his normalized gain was 0.25, consistent with Hake’s findings for “traditional” lecture courses.  The first semester Mazur used peer instruction, his normalized gain was 0.49, also consistent with Hake’s findings.

Perhaps most interesting is that as Mazur gained experience with these teaching methods (and made refinements to them, like the replacement of flash cards with clickers in his second year using peer instruction), his normalized gain increased by several percentage points each year, hitting 0.74 the sixth time he implemented peer instruction.  Thus he was, in a sense, three times as effective in helping his students master concepts in first-semester physics.

Comments: I tend to review more recent articles on teaching with clickers on this blog, but I couldn’t resist posting something about this classic article.  Mazur’s peer instruction technique is the most commonly used approach to teaching with clickers, and that’s in large part to the persuasiveness of the data he has collected on its impact in his courses.  This article presents solid evidence that having students read their textbooks before class and grapple with tough conceptual understanding questions during class in small groups is a superior way to teach first-semester physics.

It’s also worth noting that Mazur’s normalized gain improved over time.  I’ll occasionally read an article by an instructor who taught a section of a course with clickers and a section without and student performance in the two sections to find that using clickers had little or no impact on student performance.  These experiments often have a variety of design problems, but, regardless, it’s important to note that instructors can improve in their use of a particular teaching method over time.  Expecting great results the first or second time out is sometimes unrealistic, and big learning gains are sometimes only possible after a few semesters experience.

Back in January, I blogged about a New York Times article describing MIT’s Technology Enhanced Active Learning (TEAL) classrooms.  Just today, Diana Senechal blogged about the article, too, as well as her own experiences as an adult student in a physics class that uses clickers.  A few important questions were raised in Diana’s post and in the comments that followed it–questions about the prep time teachers need to teach with clickers and about which students we should be trying to benefit through our teaching.  I weighed in on those questions on Diana’s blog post, but I thought I would reproduce my comments here in case my readers would like to weigh in, too.

I’ve taught math courses with clickers for five years now, and (full disclosure) I’ve written a book on teaching with clickers, one that draws upon interviews I conducted with 50 faculty members in different disciplines, including physics.  As you might expect, I have a few thoughts about the questions raised here!

The first thing I noticed reading this post and its comments was the juxtaposition of the MIT student’s comment that using clicker-facilitated active learning during class means professors don’t have to prepare as much and Mike Anderson’s comment that using the IFAT quizzes he describes took more, not less, preparation time.

I think Mike’s hit the nail on the head: Figuring out what misconceptions students are likely to have, which is required for coming up with plausible wrong answers to multiple-choice questions, is challenging work.  And doing what the MIT physics professors are doing–designing intensive learning experiences that help students resolve misconceptions and build their knowledge–is even more challenging.  It requires a great deal of understanding of student learning and motivation.

Speaking of student motivation, the question was raised above asking which students are benefited by more active classroom learning experiences.  I would argue that as teachers, we have a responsibility to try to motivate and teach all our students, not just the ones that are self-motivated or the ones who learn best by listening to a lecture.  I think it’s great that Diana enjoys and benefits from a great lecture.  Evidence points to the fact that such students are in the minority.  Combining lectures with more participatory learning experiences is likely to benefit more students’ learning.

I’ll also point out that the pedagogy behind Mike’s IFAT quizzes is very similar to the pedagogy behind effective instruction with clickers–getting students to actively engage with problems and to discuss those problems with peers and their instructors, and providing instructors with useful feedback on student learning, feedback that can inform future instruction.  As Ricki points out, it’s the pedagogy that counts more than the technology.

That being said, clickers provide a few advantages that other technologies don’t.  Clickers allow me to hold my students accountable for their class participation since the system tracks individual student responses.  However, clickers also provide students with a level of anonymity since their peers can’t see who they responded, making it safer for them to take risks and be wrong.  (Asking a question to a class of students and taking the first student response privileges those students who are quicker, more confident, and more experienced.  It leaves all the other students out of the loop, unfortunately.)  And the instant display of results (in the form of a bar graph) provides the instructor with useful information for making on-the-fly teaching choices and can have an impact on student motivation.  If, for instance, students see that most of them answered a question incorrectly, they’re more likely to pay attention to the explanation that follows.

So, dear readers, what say you?  Any thoughts on the prep time issue or the question of which students are most benefited from active engagement teaching techniques?

A couple of weeks ago, Stephanie Chasteen posted a series of blog entries on her ScienceGeekGirl blog from an American Association of Physics Teachers (AAPT) conference.  One entry describes a session she attended that focused on interactive lecture demonstrations in physics courses.  If you’ll think back to a physics or chemistry course you’ve taken, you can probably remember a class session or two in which your instructor performed some kind of demonstration at the front of the classroom.  Research data shared at the AAPT session indicate that students learn a lot more from these demos when they do more than just watch them.  Having students interact with the demo somehow increases learning.

One way to have students interact with a lecture demo is to have them respond to a clicker question that asks them to predict the outcome of the demo.  This helps create a “time for telling” about the demo, particularly if most students predict incorrectly.  Since the students have thought about the demo and have committed to their prediction, when that prediction turns out to be incorrect, the students are ready (cognitively and affectively) to hear an explanation of the demo.

One of the presenters shared an interesting result.  Stephanie writes:

However, the learning gains don’t seem quite as high when they use clickers.  They conjecture that the clickers don’t require students do actually do ray tracing, etc., as much as when they don’t have clickers.  (My thought on that is that you shouldn’t present the clicker answer choices until they’ve done the ray tracing and other cognitive work required to arrive at an answer).

Stephanie’s suggestion would, I think, be echoed by Jennifer Imazeki, the economics instructor I blogged about recently, who takes that very approach with some of her clicker questions.

Stephanie also writes that one of the presenters at the AAPT session, David Sokoloff, is “looking for people who would like to use some of their clicker interactive lecture demonstration.”  Email him at sokoloff at uoregon dot edu if you’re interested.

If you follow the world of classroom response systems, you’ve likely heard of Harvard physics professor Eric Mazur.  (I’ve certainly mentioned him often here on this blog.)  Mazur’s book, Peer Instruction, and related talks and videos introduced the most commonly used clicker pedagogy (peer instruction) to many instructors in the sciences and other disciplines.  For a concise introduction to this pedagogy, as well as what motivated its development by Mazur, see his recent article, “Peer Instruction: An Overview.”

Here are a few key statements from that article:

It became apparent that many students were simply memorizing algorithms without understanding the underlying physics.

Mazur’s physics education research has helped make the case that students are quite capable of solving computational problems without understanding the associated concepts.  When planning assessment and learning activities for students in quantitative disciplines, this is an important point to remember.  Mazur addresses this issue by including both computational and conceptual questions on his exams and spending class time helping students make sense of conceptual clicker questions he calls ConcepTests.  See his article for examples.

Here’s another good idea:

Preclass reading assignments from the book first introduce the material. Next, lectures elaborate on the reading, address potential difficulties, deepen understanding, build confidence, and add additional examples.

In his talks, Mazur often talks about learning as a two-step process: transfer and assimilation.  Instead of using lectures to transfer information and after-class problem sets to help students assimilate that information, Mazur has his students read the textbook before class for the transfer step.  He then uses class time to help them assimilate that information.  Why?  In part because it makes sense to use  class time to do things that take advantage of the fact he and his students are all together.  He makes class time a collaborative learning experience for the students by using clickers to facilitate peer instruction, a pedagogy he outlines clearly in the article.

One more point from the article:

I view the results from the poll without displaying the results to the students. If more than 30% and less than 70% responded correctly, I give the students one to two minutes to convince their neighbors of the correct answer.

This is a change in practice from Mazur’s early use of peer instruction.  He didn’t used to “hide” the results of the first vote from the students.  However, he and others, including me, have begun to think that showing the results of the first vote to students should only be done when those results are likely to motivate students to engage more seriously in discussions with their peers.  If most of the students agree on the answer to a clicker question (whether they’re right or wrong), they’re less likely to think the question is worth discussing.  As a result, I’ve started only showing my students the results of the first vote if those results indicate disagreement among the students.  I think that kind of results motivates them to take the discussion more seriously.

If you get the chance to hear Eric Mazur give a talk, take advantage of it.  The story he tells about how his teaching has evolved over time is a compelling one.  You might also check out some of his publications on peer instruction, listed below.

• Fagen, A.P., Crouch, C.H., & Mazur, E. (2002). Peer instruction: Results from a range of classrooms. The Physics Teacher, 40(4), 206-209.
• Crouch, C. H., & Mazur, E. (2001). Peer instruction: Ten years of experience and results. American Journal of Physics, 69(9), 970-977.
• Lasry, N., Mazur, E., & Watkins, J. (2008). Peer instruction: From Harvard to the two-year college. American Journal of Physics, 76(11), 1066-1069.
• Mazur, E. (2009). Farewell, lecture? Science, 323(5910), 50-51.

A couple of weeks ago, Stephanie Chasteen shared a series of blog posts on teaching with clickers in upper-division physics courses: Part 1, Part 2, Part 3, and Part 4.  I’m often asked if clickers work well in upper-division courses, yet I’ve not met many faculty members who use them in such courses.  So I was glad to see this series by Stephanie.  It’s adapted from a talk she gave at the American Association of Physics Teachers conference a few months ago, and it includes videos that feature interviews with faculty and students about teaching and learning with clickers.  Here are some highlights from Stephanie’s posts…

One of the students interviewed in the video in Part 1 of the series says that she likes clicker questions because they allow her to take a concept and metaphorically put in her pocket.  I like that metaphor.  It indicates that the clicker question allows her to confirm that she understands a concept, which is useful during class since it helps prepare her for what follows.  This idea that clicker questions allow students to test themselves on concepts during class is one that shows up often in student surveys as a positive aspect of using clickers.  This self-testing is a type of formative assessment, and Stephanie notes it’s important to include even in small classes.

Another type of formative assessment is also mentioned in the same video.  Steven Pollock, whom I interviewed for my book, mentions that prior to using clickers he found himself making assumptions about what his students did and did not understand.  He notes that clickers provide him actual data on his students’ learning so he doesn’t have to rely on his assumptions.  I wonder if this aspect of using clickers is even more important in upper-level courses since common student misconceptions in these courses may not be as well known as in lower-level courses.

Several different types of clicker questions are mentioned in Stephanie’s series: conceptual questions, application questions, review questions used at the start of class, procedural questions asking students to identify the next correct step in a derivation.  I like the conceptual question Steven shared that distinguishes between students approaching physics from a classical mechanics point of view and those using a quantum mechanics approach.  I can imagine this kind of question is particularly useful for students making the transition to an upper-level course like quantum mechanics.

One of the arguments against using clickers in upper-level courses that Stephanie says she hears is that students in these courses are sophisticated learners.  They don’t need the structure of clicker-facilitated peer instruction to help them learn.  Stephanie presents a strong counter-argument, that since these students are more sophisticated learners, they actually get more out of the peer instruction method, more seriously engaging in small-group and classwide discussions.

Stephanie also shares some interesting data on student perceptions of clickers in upper-level courses.  Students who took a non-clicker upper-level course were asked how they would feel if they had taken the course with clickers.  They were resistant, arguing that clickers were for lower-level courses.  However, students who actually went through a clicker-enhanced upper-level course were extremely enthusiastic about their use.  Stephanie points out that students aren’t always able to predict how they’ll respond to a particular teaching approach, which is an important point to remember when trying out new approach in one’s teaching.

Take a look at Stephanie’s blog posts for more thoughts on using clickers in upper-level courses, including thoughts on their role in creating “times for telling.”  Stephanie also contributes to the clickers efforts at the Carl Wieman Science Education Initiative at the University of British Columbia, where they’ve put together a 36-page guide to using clickers in the sciences.

I have (indirectly) disturbed Bill McKeachie.  Last month, I mentioned here that James Rhem interviewed me for a lead article in the National Teaching & Learning Forum newsletter he edits.  That article has recently been reproduced on the Tomorrow’s Professor Blog.  One way or another, Bill McKeachie, author of the well-known McKeachie’s Teaching Tips, read the article.  He emailed a response to James Rhem, who forwarded the email me.  McKeachie writes:

I’m always disturbed when I read an article about clickers and no mention is made of Dick Brandt, Professor of Physics here at Michigan, who introduced clickers in his classes in the late 1940′s or early 1950′s.  I think that was the origin of the idea.

I’ll admit I’m not a clickers historian.  I’ve focused my energies on best practices with the current technology and haven’t read much about early versions of clickers.  However, I’m not familiar with Dick Brandt’s work, nor is his work mentioned in the histories of clickers with which I’m familiar, notably Judson and Sawada, 2002.

Have you heard of Dick Brandt’s work?  Before I ask Bill McKeachie if he has a reference to which he can point me, I thought I might query my blog readers.  Judson and Sawada point to some early wired classroom response systems in the 1960s and 70s, as well as some wired systems the United States military used with filmed instructional material.  Does anyone know of use of classroom response systems in colleges or universities prior to the 60s?

Reference: Nagy-Shadman, E., & Desrochers, C. (2008). Student response technology: Empirically grounded or just a gimmick? International Journal of Science Education, 30(15), 2023-2066.

Summary: This article reports the results of a survey of 350 students (mostly elementary education majors) in 13 earth and physical sciences courses taught by five different instructors at California State University, Northridge (CUSN).

One interesting approach the authors took to their study was to include four questions from the National Survey of Student Engagement (NSSE) in their survey instrument. These questions were chosen, in part, because CUSN’s performance on these questions was below average among their peer institutions. It was apparently hoped that using clickers might help overcome this performance gap. That was indeed the case, as the NSSE questions indicated that students in these courses were more engaged that students in other courses at the institution in the following ways:

• coming to class having completed readings and assignments,
• receiving prompt feedback from instructors, and
• working harder than students thought they could to meet the instructors’ standards.

Nagy-Shadman was one of the five instructors whose classes were studied. She apparently made frequent use of conceptual and application questions.  She frequently used clickers to facilitate peer instruction and classroom games and often discussed correct and incorrect answers to clicker questions after voting.  In one of her games, each team of students not only submitted team answers but were rewarded for responding more quickly than other teams, an activity that apparently worked well for exam preparation.

It is unclear from the article what kinds of questions and activities the other four instructors implemented.  Many of the outcomes measured in this survey are likely dependent on instructor variables, including implementation choices regarding clickers, that are largely not discussed in the article. One exception is that the article provides evidence that instructors with more experience teaching and with more experience using clickers had students who were generally more positive about the use of clickers.

Notable survey results where students were in almost complete agreement, regardless of instructor, included the following.

• Clickers had a neutral or beneficial effect on students’ attendance and tardiness.
• Clickers were not particularly effective in fostering students’ problem-solving skills, relative to other effects.  (One should note that it’s not clear from the article what kinds of clicker questions were asked by the instructors of the courses surveyed.  Question type would have an impact on this measure.)
• Clickers were not particularly effective in preventing “daydreaming” by students during class, relative to other effects.  (This result was true across all the courses surveyed, indicating that even the highly interactive uses of clickers employed by Nagy-Shadman didn’t eliminate student daydreaming.)
  

Following is a list of aspects of clickers that students liked, taken from an analysis of the student responses to open-ended questions on the survey.

• Clickers added “variety / fun / interaction” to class.
• Clickers allowed for anonymous responses.
• Clickers provided immediate feedback to students on their learning.
• Clickers helped students review material.
• Clickers were easy to use.

Student complaints were focused on technical difficulties, “taking too much of class time” with clicker questions, “waiting for other students to answer,” and “difficulty reading the screen from the back of the room.”

Additionally, the article features well-researched discussions of the history of classroom response systems, writing multiple-choice questions, and the role of feedback in learning.

Comments: I think the use of NSSE questions here was a great choice, in part because the results are stronger because of the ability to compare them with national data.  It is not clear from the article, however, whether the NSSE results varied across the five instructors included in the study. It would be interesting to know if particular instructors or particular approaches to using clickers were more or less effective in improving these measures.

The finding that clickers improve attendance and reduce tardiness is a positive one.  However, it would have been useful to know what attendance and participation policies were used by the instructors in this study. Given that complaints about grading and monitoring were not that common in the students’ responses to open-ended survey questions, it’s unlikely that clickers were used to take attendance or judge participation in all five instructors’ courses. (See Graham et al. (2007) for a similar study where students complained about these issues.) If that’s the case, then this is an encouraging result—that students feel more likely to attend even without tracking clicker responses.

The authors summarize advice from the literature on writing multiple-choice questions.  However, I would argue, that the existing literature on writing multiple-choice questions (for quizzes and tests, for the most part) is only partially useful for writing in-class clicker questions. There are a variety of types of questions that function well in class that would not function well on summative assessments, including questions designed to help students explore new topics, questions for which there aren’t single correct answers, student opinion and personal experience questions, questions about the teaching and learning process, and questions that ask student to assess each other’s work.  This is one reason that instructors can have difficulty imagining how clickers might work in their classes: they’re used to writing multiple-choice questions for exams, not for in-class use.