Earlier this week, I gave a virtual presentation at the Muskegon Community College Math and Technology Workshop organized by Maria Andersen. The participants were all math instructors spending the week at MCC learning from Maria and others about various uses for educational technology in math instruction.

I’ve blogged often about teaching math with clickers here, but I don’t think I’ve shared slides from any of my presentations on this topic. Since Maria asked me to put my slides on Slideshare for the workshop participants, I thought I would share them here.

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Using Prediction and Classroom Voting via Clickers to Address Students’ Overreliance on the Representativeness Heuristic,” Tami Dashley, University of Texas-El Paso [Slides]

Tami Dashley is a graduate student in math education and a student of Kien Lim, one of the organizers of the contributed paper session. She shared some of her thesis research, an investigation into the connection between classroom voting with clickers and certain misconceptions students have about probability. Her work focuses on the representativeness heuristic, which she defines as “determining the likelihood for events based on how well an outcome represents some aspect of its parent population.”

Tami gave the following example: Suppose you toss a coin six times, getting a sequence of heads (H) and tails (T). Which of the following is more likely to occur: TTHHTH or HTTHHH? Someone using the representativeness heuristic would say that TTHHTH is more likely to occur since it includes an equal amount of heads and tails, just like the coin does. The other option includes more heads than tails, so it would not seem as likely to someone using the representativeness heuristic. Actually, both of those outcomes are equally likely (each occurring with probability 1/64), so the representative heuristic is a misleading one in this example.

The issue is that the representativeness heuristic is useful in some cases, but not useful in all cases. The misconception that many students have is that it’s always useful.

How to help students stop over-relying on the representativeness heuristic? Tami has been investigating the use of prediction questions, ones that ask students to predict an outcome or probability without actually computing anything. For example, students might be asked to determine which of several outcomes is most likely to occur. Since students need not be as precise when responding to prediction questions, they have some cognitive processing power freed up to focus on concepts. Clicker questions are a natural match here, since they allow students to commit to their predictions and compare their predictions to those of their peers. Then discussion of the incorrect answer choices provides an opportunity to deal with misconceptions.

Tami conducted her research in a high school setting, using three groups of students. Her “control” group received a lesson exploring the representativeness heuristic that didn’t ask the students to predict any probabilities. A second group was asked several prediction questions but didn’t use clickers to respond to the questions. The third group used clickers to respond to prediction questions during the lesson. Tami used pre- and post-tests to determined the efficacy of these three different lessons.

Tami found that her “control” group did pretty well on the post-test compared to the two experimental groups. However, most of their success came from what she called a “learned response.” In this case, many of the students picked up on the fact that “all of the above outcomes are equally likely” is often the correct answer to questions exploring the representativeness heuristic. (These are what students might call trick questions!) When Tami looked at performance on questions where “all of the above outcomes are equally likely” was, in fact, not the correct answer, the prediction-with-voting group performed better than the control and prediction-only groups.

I was very impressed with Tami’s research design and the subtlety with which she explored student misconceptions in this teaching context. I don’t believe that Tami has published this work yet, but I look forward to reading it when she does.

Image: “Heads and Tails” by Flickr user canonsnapper, Creative Commons licensed

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Preservice Elementary Teachers’ Perceptions of Clicker Use in their College Mathematics Course,” Travis K. Miller, Millersville University of Pennsylvania [Slides]

In my last post, I mentioned that Janet White first used clickers in her courses for pre-service teachers at Millersville University of Pennsylvania. Another speaker in the contributed paper session back in January was her colleague, Travis Miller, who shared results of a student survey he conducted in the pre-service teacher course he taught. Travis used clickers for only six lessons during that course in each of the four sections he was teaching. His clicker questions weren’t graded, and he followed the “classic” peer instruction model each time, having students vote individually, then discuss the question in small groups, then vote again.

Travis’ students overwhelmingly (96%) liked using clickers in the course. Travis mentioned that there are very few things he does as a teacher that are as uniformly popular with his students! Almost as many students (89%) believed that the clicker activities helped them learn the material in those six lessons. Travis drilled down on this, asking students to say why the clickers were useful. The number one answer (59% of students) was that the clicker questions provided students with an opportunity to discuss and think about course content. The number two answer (23%) was that the clickers provided a sense of accountability and involvement.

Travis didn’t stop there, either. He asked his students which topics they understood better because of the clicker activities. Of the six topics that Travis addressed using clickers, sets and Venn diagrams was cited by 52% of the students as the one that most benefited from clickers. Numeration / base arithmetic was a distant second with 15%, and deductive reasoning came in third with 13%. When sharing these data, Travis floated a very interesting hypothesis. He wondered if the fact that the number one topic (sets and Venn diagrams) was a visual one led to the students selecting it as most benefited by clicker questions. I’m a big fan of visual thinking, so this comment caught my attention. Is there something special about peer instruction with clickers and visual thinking?  I’d appreciate your thoughts in the comments.

Travis’ other interesting hypothesis was that his more competitive students liked the competitive aspects of clickers (being the first to answer, answering correctly more frequently than other students, and so on), while the non-competitive students didn’t mind those aspects since they were essentially opt-in. That is, the students who didn’t want to compete could still participate fully with the peer instruction and voting process without feeling any pressure to treat it like a game. Graham, Tripp, Seawright, & Joeckel (2007) found that most students who are hesitant to participate in class liked clickers as well as those who were fine with participating, but I don’t think I’ve seen any research that compared competitive students with non-competitive students. That would make for an interesting research question.

Travis also taught some sections of his pre-service teacher course without using clickers, and he surveyed students in these sections about the potential advantages and drawbacks of using clickers. What concerns did they have about using clickers? They worried about the cost of the devices, that clickers weren’t necessary in small classes, that clicker activities take up too much class time, and that the technology might not be reliable. I found it interesting that these are among the common concerns of faculty members not already using clickers, too!

Image: “Happy Pi Day!” by Flickr user Mykl Roventine / Creative Commons licensed

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Using Personal Response Systems (Clickers) in Liberal Arts Mathematics Courses to Support a Lecture Format,” Janet A. White, Millersville University of Pennsylvania [Slides]

Just like Jean McGivney-Burelle and Kimberly Burch, Janet White shared her experiences teaching with clickers in a “liberal arts” mathematics course taken by non-majors. Unlike Jean and Kimberly, who teach relatively small sections of this kind of course, Janet teaches in a large lecture hall with 75 students per section. Janet had used clickers in courses for pre-service math teachers in the past and found them useful, so when it was her turn to teach this larger course, she decided to use them again. A classroom response system was hardly the only technology Janet used in this course: She also had students complete online homework and quizzes and she annotated her PowerPoint lecture slides using an Interwrite Mobi.

Janet used clickers on a daily basis in her course, usually either to assess students’ prior knowledge or to assess their understanding of a topic taught during lecture.  Her questions came from a bank of multiple-choice questions provided by her textbook publisher.  She counted the clicker questions as part of her students’ participation grades, but in a low-stakes manner.  Given her use of the questions as well as the source of the questions, many were on the lower levels of Bloom’s taxonomy, aimed at recall and application of procedural knowledge.  She shared an example of a prior knowledge question that asked students to find the measure of an angle that complements a 36 degree angle.  A slightly harder question aimed at assessment of something taught during the course asked students to identify the cut edge in a given graph (or to assert that the graph had no cut edge).

Student survey results indicated that 85% of Janet’s students who used clickers regularly liked using them, and 71% said that using clickers helped them learn the material.  Students who used clickers regularly during the course ended up with higher grades in the course than students who didn’t, but, of course, that can’t necessarily be attributed to the use of the clickers.  (And since clicker questions were factored in the course grade, students who participated more frequently in clicker questions would almost certainly have higher grades in the course anyway.)

Student comments about the clickers were generally positive.  My favorite one was, “I liked getting the wrong answer anonymously.”  Other comments addressed the usual points that students like about clickers: They liked the interactivity, they liked discussing questions with classmates, they liked seeing where they stood relative to their peers, and they liked the feedback on their own learning the clicker questions provided.  The only significant negative aspect for the students was the cost, about $50 in Janet’s case.

Janet found that having students discuss clicker questions in small groups led to very engaged students, even in the large auditorium environment.  In the future, she plans to write more of her own questions, instead of relying on ones from the textbook’s question bank.  She hopes to write more difficult questions that will generate even more engaged discussion during class.  She’s also hoping to find ways to reduce the technology cost to the students, either by selecting a different vendor or facilitating the resale of clickers after each semester to students taking the course the next semester.

Also, Janet mentioned that the earth science faculty at Millersville are big users of clickers.  Earth science instructors looking for advice on using clickers might want to investigate!

Image: “Recursive Daisy” by Flickr user gadl / Creative Commons licensed

Continuing my reports from the contributed paper session on teaching with clickers I helped coordinate at the Joint Mathematics Meetings back in January…

“Clickers in the Classroom,” Kimberly J. Burch, Indiana University of Pennsylvania [Slides]

Kimberly teaches a “Math 101″ survey course called “Foundations of Mathematics.”  Topics covered include set theory, graph theory, and counting methods (among others), and Kimberly shared several interesting clicker questions on each of these topics.  For example, here’s one of her questions from the unit on graph theory:

How many vertices are there in a tree with 19 edges?

1. 19
2. 18
3. 20
4. Not enough information given

Kimberly practices the “classic” peer instruction technique of having students vote individually first, then discuss the questions in small groups, then vote again.  She finds that students often converge to the correct answer on the second vote.

In the example above, her students were split between 18 and 20 on the first vote, but after the peer discussion time, most students went with the correct answer, 20.  I found this interesting because the “Not enough information given” seemed to be the obvious wrong answer to this question.  A graph with 19 edges might have any number of vertices, but a tree with 19 edges can only have 20 vertices.  Students who don’t realize that trees are graphs with very specific properties might be tempted to go for the “Not enough information given” option.

I suspect that Kimberly used this question after the students learned the relationship between the number of edges and number of vertices in a tree and that this question was meant to assess whether students remembered that relationship.  Some students likely remembered that one of these numbers was one more than the other but weren’t sure which one was higher.  That would account for the split vote between 18 and 20.  Had this question been asked as an exploratory question and not a review question, I’m betting the split would have been between 20 and “Not enough information given.”

Kimberly also mentioned that she uses her clicker system’s priority ranking questions to have her students decide what topics should be emphasized during exam review sessions.  Kimberly gives her students a list of 8-10 exam topics, and the students indicate the top three or four toughest topics in order.  Kimberly said that this helps her make good use of limited exam review time by focusing on the topics the students find the most difficult.

Kimberly also shared some data from a quasi-control group experiment she conducted.  She taught two sections of this survey course and alternated which topics she covered with clickers in the two sections.  For example, section A might cover topic 1 with clickers while section B covered topic 1 without.  Then for topic 2, section B used clickers and section A didn’t.  She then compared test scores for the two sections by topic.  For some topics, students using clickers performed better on exams but for other topics, the students not using clickers performed better.  And for other topics, there was no difference.  The data was generally favorable to using clickers, but the “quasi” part of this quasi-control group experiment made it difficult to draw firm conclusions.

Image: “Point Marian Bridge” by Flickr user timmenzies / Creative Commons licensed

File this under “Better Late Than Never.”  Back in January 2010, I coordinated (with Kelly Cline and Kien Lim) a contributed paper session on teaching with clickers at the Joint Mathematics Meetings in San Francisco.  Shortly after the conference, I blogged about some clickers talks that didn’t fall in our session, but I never got around to blogging about the talks in our session!  Five months later, I’m finally getting around to sharing my notes from those talks…

“The Evolution of Classroom Voting in Contemporary Mathematics,” Jean McGivney-Burelle, University of Hartford [PowerPoint Slides]

Jean teaches a “math for the liberal arts” course called Contemporary Mathematics taken by music and arts majors among others.  She finds her students come to the course with relatively little interest or self-reported ability in mathematics, so it’s a tough crowd to teach.  A few years ago, she started teaching with clickers in order to appeal to what she calls the “thumb generation”–students used to spending a lot of time sending text messages.

Jean interspersed some clicker questions throughout her lectures and encouraged students to discuss them in small groups before voting.  She and the students liked this, but she found that most of her questions were answered correctly by most of her students  and that the small group discussions didn’t involve much debate among students.  The next year, Jean decided to ask tougher questions.  She calls them QEDs–Questions to Encourage Discussion.  She aimed for the analysis, synthesis, and evaluation levels of Bloom’s Taxonomy.

For example, in Jean’s first year using clickers she gave her students a preference schedule–a list of how each voter in an electorate (only four of them to keep things simple) ranked all of the candidates.  She then asked her students to determine which candidate would win the election using the instant run-off voting scheme.  This is a straight-forward application of a particular algorithm.

The next year, Jean asked another question in which she shared a preference schedule with her students and asked them to analyze it.  However, this time, she asked the question at the beginning of the unit on voting schemes and asked her students to indicate which of the candidates had the best case for winning the election.  There’s no single correct answer to this question since winner of an election (well, one involving at least three candidates) depends on what scheme you use to count the votes.  This is a great example of a one-best-answer question (since students are asked to select the one answer they think is best among multiple reasonable answers) used to create a time for telling (since it’s used to make the point that which voting scheme you use matters).

Jean found that these more challenging and ambiguous questions generated longer and more engaged small group discussions as well as more “horizontal” bar graphs–ones indicating significant disagreement among the students.  Looking ahead, she plans to build on this success by writing questions designed to develop mathematical habits of mind–an important goal of this course.  For example, here’s a sample question she shared aimed at pointing students towards the notions of proof and counterexamples:

Suppose there is a majority winner in an election. Will all of the voting methods we have studied thus far always pick that winner?  Yes or no?  If you answer yes, prepare to defend your answer. If you answer no, have a counterexample ready.

I really like this question.  It has a degree of ambiguity that students often find disconcerting, but it also reminds students of how they’ll need to defend their answers, which should help put their minds at ease.  As Jean noted in her talk, in a course like this one, it’s more important students develop mathematical habits of minds (like the ones surfaced by this question) than learn particular math content areas.  I hope this kind of question helps with this objective.

Stay tuned to the blog for more notes on these talks over the coming days…

Image: “Deep Down Inside, We All Love Math” by Flickr user Network Osaka / Creative Commons licensed

Just over a year ago, I shared a story here about a clicker question I used in one of my math courses that didn’t go as planned during class.  I titled my post “Flexible Clicker Questions” because I wanted to make the point that clicker questions that seem to be poorly written can turn into real learning opportunities for one’s students.  If you put a poorly worded multiple-choice question on an exam, you’re in for a lot of student complaints and regrading.  However, in class, a poorly worded multiple-choice clicker question can, with a little agility on your part, turn out great.

I mention this because Mitch Keller recently described a similar incident in his math course over on his blog, Partially Ordered Thoughts.  He posed a particular clicker question with what he thought had a single correct answer.  His “correct answer” was indeed the most popular student response to the question, but more than 60% of students selected other answers.  Mitch wisely had his students discuss the question in small groups and then led a classwide discussion of the question.  Not only did he surface the correct reasoning for the “correct” answer, but he discussed the other answer choices, too.  It turned out that there were reasonable arguments for not one, but three of his answer choices.  Mitch writes:

A natural first reaction to a slip-up in a clicker question is almost always “Drats! I thought I’d done that perfectly.” However, it became a teachable moment. In reality, we were able to discuss far more aspects of generating functions than I intended with the question.

Have you used a clicker question that turned out to be poorly worded, yet resulted in valuable class discussions?  Please share below!

Image: “Untitled” by Flickr user Maurizio Polese / Creative Commons licensed

A few weeks ago, Jason B. Jones, one of the editors at my favorite group blog, ProfHacker, invited readers who teach in math and science disciplines to contribute articles to the blog.  I took Jason up on his offer, and I’ve now contributed two posts to ProfHacker.

In the first post, “Multiple-Choice Questions on Exams,” I describe some of the reasons I use multiple-choice questions on my exams.  As regular readers of my blog know, I find multiple-choice questions very useful as clicker questions.  I think many instructors underestimate their use in the classroom.  However, as I write in my ProfHacker post, I’ve found multiple-choice questions very effective for assessing students’ conceptual understanding in my courses.  Moreover, since I have my students spend a good chunk of each class session grappling with multiple-choice questions, it seems appropriate that these kinds of questions would show up on their exams!

In my second ProfHacker post, “Getting Students to Do the Reading: Pre-Class Quizzes on WordPress,” I describe the rationale, use, and implementation of pre-class reading quizzes in my math courses.  As I’ve mentioned on this blog, I ask my students to read their textbook before coming to class as a first encounter with the course material.  This frees up some class time for more active learning (such as clicker-facilitated peer instruction), and it also allows me to practice “just-in-time teaching” by letting student responses to pre-class quiz questions inform my lesson plans so that our use of class time is more responsive to student learning needs.  See the ProfHacker post for lots more details and a great set of comments by other ProfHacker readers.

A math colleague of mine, who blogs under the name Doc Turtle, recently blogged about his use of a calculus worksheet that helps his students “guide themselves through the algebraically intense process of partial fractions.”  Doc Turtle reports that his students look forward to this kind of work, and he’s planning to develop more activities along these lines.

I’ve heard from several instructors who have students engage in this kind of active, self-directed learning in class (through worksheets, clicker questions, and so on) that some students complain that the professor isn’t doing any work.  I suspect that these are the students who expect to come to class, take a lot of notes, and figure the material out while working through their homework.  They can sometimes push back when their instructor isn’t presenting course content in the way they expect.

Of course, instructors who design and implement activities like Doc Turtle’s worksheet activity aren’t avoiding the hard work of teaching.  Instead, they’re being intentional about what they want their students to learn and they’re planning and facilitating experiences designed to help their students learn those things.

As Ian Beatty wrote over on his blog, “It’s not really creating [clicker] questions that’s tough.  The hard part is figuring out what I want my students to learn from the class, and casting that in terms of what I want my students to be able to do.”  Once he’s done that, he says it’s relatively easy for him to write effective clicker questions.  “Just formulate a question asking them to do that (in a particular context), and then much of the class activity is me helping them struggle through the process as they learn how.”

What struck me about Doc Turtle’s post was how excited his students are to engage in this kind of active learning.  As I mentioned above, not all students see this kind of learning as valuable.  Did Doc Turtle just get lucky with a batch of exceptional students?  I suspect not.  I’m guessing that he’s been teaching his students to learn this way since the first day of classes so that by this point in the semester, his students are perfectly willing to see this kind of activity as valuable.  I think that’s an important takeaway: If we’re asking our students to learn in a “new” way, then we need to help them learn how to learn in that way.

Do you find that your students push back when you ask them to engage in active learning in class?  How do you help them see the value in this kind of learning over time?

Here are a few interesting ideas shared during the first set of talks about teaching with clickers at the Joint Mathematics Meetings earlier today.  (This post’s title inspired by the following books seen at the exhibits: Calculus Gems, Mathematical Diamonds, More Mathematical Morsels, and Biscuits of Number Theory.)

Kathryn Ernie (University of Wisconsin-River Falls) shared ways that she and her colleagues use clickers in their college algebra courses.  One use that she mentioned was to warm students up before “traditional” in-class quizzes.  By asking students a clicker question or two, then discussing those questions prior to a graded quiz, the students are able to approach the quiz with a little more confidence.

Ben Galluzzo (Shippensburg University) also talked about graded quizzes.  However, in his case, clicker questions aren’t warm-ups for the quiz; the clicker questions are the quiz.  Clickers allow Ben to turn his quizzes into learning experiences for his students.  After each quiz question, he discusses the question with the class before moving on to the next question.  This can work particularly well when he has more than one quiz question of the same type.  Students who miss the first one can learn from the discussion of that question and apply what they’ve learned to the subsequent question.  Students like this because they appreciate the chance to redeem themselves.

Aprillya Lanz (Virginia Military Institute) mentioned that teaching with clickers help students stay awake and engaged during class.  This is particularly important for her since many of her students are freshmen (“rats” as they’re called at VMI) who are required to participate in all kinds of strenuous physical activities, particularly on Sunday nights.  This can make for some very sleepy students in Monday morning classes.

Daniel Joseph (also VMI) described a problem that those who teach calculus often see: His calculus students often struggle because of pre-calculus misconceptions.  They can’t tackle the calculus because they get tripped up by algebra and other pre-calc topics.  He described several methods he’s tried to combat this, but he finds that the students’ over-confidence trips keeps these methods from working.  The students say they “know” all the pre-calculus material because they’ve studied it in the past.  Daniel appreciates how clickers provide his students with frequent evidence that they don’t know it as well as they think they do.

Daniel shared one approach to attacking this problem–using clicker questions in a pre-semester pre-calculus course for incoming freshmen.  He’s interested in hearing ideas for hitting this issue, with or without clickers, in the calculus course itself.  Any ideas?

(By the way, Daniel used the phrase “attack this problem” at least five times in his presentation.  Given that he teaches at a military institute, I figured that was language that comes naturally to him.  Thus my use of the verbs tackle, combat, attack, and hit above!)