I’m back from the Inaugural Conference on Classroom Response Systems, hosted by the Delphi Center for Teaching and Learning at the University of Louisville.  I had a great time meeting people who I knew only by their research, and some of the sessions were very well done.  Thanks to the Delphi Center, particularly director Gale Rhodes and associate director Marianne Hutti, for putting together such an enriching conference.  I’m already looking forward to next year!

Tim Stelzer, research associate professor of physics at the University of Illinois and one of the founders of i>clicker, presented the morning keynote address.  His presentation was very engaging, featuring a nice blend of information and humor.  His slides were particularly impressive, having been designed in the Presentation Zen style.  Many of his slides consisted primarily of one fullscreen image and just a few words of text.  This put the focus on Stelzer and his message while reinforcing that message visually.  He included a few well-chosen video clips and animations that helped him make his points (with a little humor) as well as an ongoing clicker-enabled game that kept his audience engaged.

One point Stelzer made that stood out to me was that in the past, being highly educated was correlated strongly with remembering lots of facts.  This is still true today to some extent.  Consider Ken Jennings, the guy who won all those Jeopardy gameshows.  He’s considered highly intelligent, but not for higher-order thinking skills (problem solving, critical thinking, etc.), just for remembering lots of trivia.

Stelzer made the point that with all the information available to students via the Internet, factual recall doesn’t play the same role it used to play in learning.  The challenge now in higher education is to develop students’ higher-order thinking skills, and Stelzer feels that classroom response systems can facilitate pedagogies that help teachers meet that challenge.  This is a valid point, and it’s one of the reasons I included in my book so many examples of clicker questions aimed at higher-order thinking skills.

Stelzer also described the genesis of the i>clicker classroom response system.  The first electronic system he used at Illinois (after abandoning the flash card method due to poor student participation, caused by lack of accountability) involved hardwiring jacks in all of the seats in a lecture hall.  The students connected their TI-83 calculators to these jacks.  This system allowed a couple of neat features not available in current systems to my knowledge.  One was that it allowed the instructor to call up a seating chart showing each student’s name and how that student voted in response to a clicker question.  This allowed an instructor to say, for instance, “John, I see that you answered B but the students sitting next to you answered C.  Why don’t you discuss this question with them and see if you can come to a consensus?”

Another feature of the system was that when a student answered a multiple-choice question, the system could be programmed to send the student a response determined by the student’s answer choice.  So if a student selected choice A, the system might reply, “Have you considered X?”, where X would be some example or concept relevant to the answer choice.  I imagine this feature would be very useful in helping students think more deeply about their answer choices.  It’s also a feature that could be implemented in some of the systems that use cell phones, smart phones, and laptops available now.  In fact, this might already be a feature of some of these systems.  If you know that to be the case, please let me know.

(Coincidentally, Friday night at the conference reception, I met Kevin Patton of St. Charles Community College in Missouri.  He and I were talking about clickers and somehow hit upon this very same idea–giving feedback to students after their answers right on their clickers, feedback tailored to their particular answers.)

One of the research findings Stelzer shared was particularly interesting, too.  According to surveys of students at the University of Colorado at Boulder, where over 17,000 clickers are in use, the factor mostly highly correlated with negative student reactions to clickers was “sporadic use.”  If clickers aren’t used very often, students tend not to like them.  That’s pretty good evidence that students see some value in the use of clickers, I think.

Stelzer’s talk was videotaped and will be posted online, probably at the i>clicker site.  His top ten tips for using clickers are also available at the i>clicker site.

I have more thoughts from the conference I’ll be posting in the next few days.

Inside Higher Ed posted a story yesterday on a session at the annual meeting of the Association of the Study of Higher Education that explored the ethics of “double dipping,” which is described as faculty “using the same scholarly material in multiple formats and settings.”  For instance, some would consider presenting the same talk at two different conferences unethical.

I found it interesting that Marybeth Gasman and Kristen Renn, the presenters of the session, used clickers to poll the audience, asking for their opinions on the ethics of various actions.  For example,

Is it ethical to submit a proposal to present the same paper to the annual meetings of both ASHE and the American Educational Research Association , which follows the higher education scholarly meeting by a few months? Yes, if the proposal is rejected by ASHE first, said 14 respondents. Yes, even if ASHE accepts it, said two. No under any circumstances, said three others.

This is a great example of the use of clickers to ask opinion questions.  In this public setting where the audience members (faculty members and educational researchers in this case) don’t know each other well and where the topic of the session is a controversial one, asking for a show of hands just might not do the trick.  The anonymity provided by the clickers helps encourage honest answers, and the results of the clicker questions displayed on-screen provide data to fuel the group discussion.

I would like to reminder my readers that the Inaugural Conference on Classroom Response Systems is happening this Saturday, November 15th, at the University of Louisville in Kentucky.  As I mentioned back in August,

The conference features keynote presentations by physicist Tim Stelzer (one of the inventors of the i>clicker system) and astronomer Douglas Duncan (author of Clickers in the Classroom).  Over a dozen other sessions are on the schedule, including one by yours truly that features a few activities based on the taxonomy of clicker questions and activities I developed while writing my book.

Let me know if you’re planning to be at the conference.  It would be great to meet my blog readers face-to-face.

Also, I’m interested in interviewing a couple of people at the conference and podcasting those interviews here on the blog.  If you’ll be at the conference and would be interested in being interviewed about your uses of clickers, please let me know.

I’ll be sure to blog about the conference when I get back, too.

Jeanne Dillon of Argosy University recently emailed me to ask if I knew of any research on the role of gender in the impact of the use of classroom response systems.  Since something is preventing me from replying to her email, I thought I would respond here.  (Jeanne, if you’re reading this, please email me again or leave a comment below.  Thanks!)

Taking a look through my clickers bibliography, I found several research articles that discussed gender differences.  Several studies–Freeman and Blayney (2005), MacGeorge et al (2007), Nicol and Boyle (2003), Rice and Bunz (2003)–found no statistically significant difference between male and female students’ perceptions of the use of classroom response systems (satisfaction, perceived benefits, etc.).  Scornavacca and Marshall (2007) found the same result when looking at a text-messaging-based classroom response system.

Len (2007) found that some of his astronomy students were “self-testers,” preferring to answer clicker questions independently, and some were “collaborator,” preferring to discussion clicker questions with peers before answering.  He found that gender had no impact on the likelihood that a student was a self-test or a collaborator, which is a somewhat surprising result given commonly held beliefs about gender differences and collaborative learning.

The only study I found that looked at the role of gender in the impact of the use of clickers on student learning (as opposed to student perceptions) was Reay, Li, and Bao (2008).  In that study, it was found that the gain for male students from pre-test to post-test was statistically greater than the gain for females when clickers weren’t used.  When clickers were used, the gains were the same, indicating that clickers (and the question sequence pedagogy the instructors used) reduced the performance gap between male and female students.  As I noted in my post about this study, it’s tough to isolate the effect of the clickers here from the overall effect of the question sequence pedagogy used.

In summary, there’s some evidence that gender doesn’t play a role in how student perceive the usefulness of teaching with clickers.  There’s almost no evidence for or against a gender role in the impact of clickers on student learning at this point.  I would encourage Jeanne and others conducting research on classroom response systems to look for any gender differences.  This would seem to be an area of research with great potential.

If you know of any other research along these lines, please let me know.  Thanks!

On the final day of the EDUCAUSE Annual Conference, I attended a session titled “Growing and Sustaining Student Response Systems at Large Campuses: Three Stories” presented by Christopher Higgins of the University of Maryland, Nancy O’Laughlin of the University of Delaware, and Michael Arenth of the University of Pittsburgh.  The presenters’ slides are available, and Inside Higher Ed ran a story on the session, too.

University of Maryland

There were three classroom response systems in use at the University of Maryland as of a few years ago, so the IT office got together with Undergraduate Studies and the Center for Teaching Excellence to form a review committee that recommended the adoption of the TurningPoint system.  Key factors included keeping student data on campus (because of FERPA), cost to students, integration with PowerPoint and Maryland’s course management system, and reporting options.

They now have over 12,000 clickers in the system with at least 75 faculty members using clickers, mostly with courses in business and the natural sciences.  (Some departments purchased their own sets, so IT isn’t sure how many faculty are using clickers in these departments.)  I haven’t spoken with many business faculty about how they use clickers, although the business and management section of my bibliography is one of the larger ones.  I might try to track down a couple of Maryland business faculty to find out how they are using clickers.

Challenges included registering student clickers, which required two different registration systems for a while.  Also, the software isn’t as robust on Macs, which poses a problem for some faculty.  They also went from 10 classrooms with receivers and software to 150 in a single semester which was challenging!  TurningPoint’s receivers also needed upgrading last academic year, which posed some logistical problems.

Currently, the IT office handles technical support for faculty using clickers, while the Center for Teaching Excellence handles training and promotion.  The two units seem to work well together, offering joint training sessions that have gone over well.  IT finds it necessary to have a staff member devoted almost entirely to clicker support at the start of a semester.

Christopher Higgins is particularly excited about TurningPoint’s new ResponseWare Web system, which enables any Web-enabled device (laptop, iPhone, etc.) to function as a clicker.  He likes the fact that the system leverages existing hardware that can also perform other functions, as well as the fact that the Web system is cheaper–$20 per student per year or $40 per student for four years.  Christopher found that many students took advantage of an Apple promotion this fall to purchase iPod Touches and iPhones along with their Mac laptops so a lot of students at Maryland have devices that can run the new TurningPoint system.

University of Delaware

The adoption committee at Delaware included not only faculty members and IT staff, but staff from the assessment offices and, I think, students, as well.  (I may have misheard that last point.)  They standardized on Interwrite PRS and spent the summer of 2006 training faculty and installing receivers and software in all classrooms with at least 75 seats.  (They now have receivers and software in all classrooms with at least 35 seats, which is most of the classrooms on campus.)  By the fall semester about 3,600 students and 40 faculty were using clickers.  More faculty started using clickers in the fall of 2007, but this year there are relatively few faculty new to clickers since most faculty have heard about them and decided whether or not to use them.

Clickers are popular in courses in the natural sciences, as well as psychology, political sciences, and nursing.  Many first-year undergraduate courses use clickers, which means that faculty teaching “downstream” courses are now more likely to use clickers, as well, since most of their students already own the devices.

Clickers are used in non-academic settings on campus, too.  Residential Life uses them to collect information on student experiences and opinions in the dorms.  The library and the office of assessment use them, as well.

Challenges to the support of classroom response systems on campus included a move to a new unique student identifier.  The Interwrite PRS system allows students to enter and store their unique identifiers on their clickers, but it took some work to have all the students request a new unique identifier on the Delaware Web site.  Other challenges included handling new versions of the software and a move from one course management system (WebCT) to another (Sakai).

One process Nancy mentioned that I particularly liked is that when faculty request clickers for their courses from the bookstore, there’s a checkbox on the form that asks them if they are new to using clickers.  Faculty who check this box are then sent resources by Nancy’s office and added to Nancy’s mailing list.  This helps faculty connect to useful pedagogical and technical resources and helps Nancy know who’s using clickers on campus.

Nancy also mentioned that she’s found it helpful to give faculty members their own receivers so they can practice as much as they need to outside of the classroom.  She finds that students know when their teachers aren’t comfortable with a technology, so time for practice is important.

Another point Nancy made was that the code of student conduct at Delaware has been amended to mention clickers.  Students are to respond for themselves, not on behalf of other students.  She indicated that faculty appreciate having this clause in the code since it means there’s a process they can follow if they suspect students of cheating by bringing other students’ clickers to class.

University of Pittsburgh

Things at Pittsburgh have been a little more chaotic.  A review committee consisting of IT staff, facilities staff, and registrar staff decided in 2003 not to adopt a single system on campus.  As a result there are now a few systems in use on campus now.  There’s now some move toward standardizing on eInstruction, but there doesn’t seem to be a central decision-making office that enforces that decision so faculty are still free to use other systems.

Clickers are popular in biological sciences, physics, nursing, and pharmacy.  Also, the School of Social Work uses them frequently in their gambling addiction counselor program.  I wouldn’t mind talking to some of those faculty to find out how they use clickers in that setting.

Michael Arenth named a few challenges they’ve faced at Pittsburgh, including managing faculty expectations (particularly for faculty who get excited by clickers but don’t plan on the time necessary to learn the systems), cheating (students who bring other students’ clickers to class to cheat on attendance grades), and set-up between classes since until recently, they haven’t been installing systems in classrooms.

I believe Michael said that Pittsburgh has still been using infrared clicker technologies until fairly recently switching to radio frequency.  (Most people I’ve talked to made the switch a couple of years ago.)  He noted that the IT group on campus had to approve the use of radio frequencies for this purpose.  I hadn’t heard of this kind of approval before, so I found this point interesting.

Common Issues

All three campuses have surveyed faculty and students about clickers, and they used some common questions to enable comparisons among the three campuses.  They found that faculty frequently use clickers to measure student comprehension, measure student opinion, obtain anonymous responses, monitor attendance, and facilitate quizzes.  The presenters spoke only briefly about these results, and it was unclear to me the extent to which faculty use comprehension or opinion questions to generate small-group or classwide discussion or to practice “agile teaching” by responding to the results of clicker questions during class.  I was, however, happy to see that clickers were used more for formative assessment (measuring comprehension and opinions) than summative assessment (quizzes and tests) since I think that’s where clickers really shine.

An audience member at the presentation asked about the student response to clickers.  The panel indicated that students like the interactivity that classroom response systems provide.  They confirmed what I’ve now heard from multiple sources, that students want to see some value added to their learning experience as a result of the clickers.  If a faculty member just asks a question and quickly moves on, there’s no interactivity and little impact on student learning.  Students don’t respond well to this.

Finally, I spoke with Danny Sohier of Université Laval in Québec after the session.  His school is using clickers to conduct end-of-semester course evaluations during class.  They found that online course evaluations resulted in low response rates, a problem I’ve heard about from many institutions.  They now use clickers to collect student responses to multiple-choice evaluation questions during class in some courses, inviting students to respond to open-ended questions online outside of class.  Danny indicated that this arrangement is working pretty well.  I might follow up with him to learn more about this process.

That’s it for my notes on this session.  I was glad to see a clicker session on the agenda at EDUCAUSE.  I was a little surprised at the number of audience members who asked questions at the end of the session and at the nature of those questions.  It seems there are a lot of institutions that are still just starting to work on adoption and support issues.  That indicates to me that use of classroom response systems will continue to grow over the next few years.

Today was more about networking and visiting vendor booths (including Qwizdom, eInstruction, and SMART Technologies) than attending sessions.  Here are a few clickers-related highlights…

• I got a chance to talk some more with the folks from Abilene Christian University (George Saltsman, Kyle Dickson, and Bill Rankin) about their iPhone initiative.  One idea we discussed for free-response questions was to have students respond to a free-response question, then push the list of student responses back to the students in some fashion so that each student has an electronic version of the list of ideas generated by the whole class.  Then students, individually or in groups, could start working with that list, grouping, prioritizing, exploring the ideas on the list.  Imagine having the sociology majors in an interdisciplinary course turn the list of student suggestions into a concept map, while also having the history majors and biology majors in the same class generate their own concept maps, then discussing similarities and differences in the various maps.
• I ran into Kevin Yee of the University of Central Florida.  He’s working with an undergraduate student on a project exploring the use of clicker questions trees.  With question trees, instructors design a set of branching clicker questions.  The responses to a given question determine which of several possible clicker questions are asked next.  If you’re about my age (as Kevin is), you can think of these as “Choose Your Own Adventure” clicker questions.  (I’ve tried using the CYOA metaphor in clickers workshops in the past, but apparently, if you’re not about my age, it really doesn’t mean anything!)  I haven’t found many people using clickers in this way, so I was glad to hear that Kevin is engaged in this kind of project.
• Here’s another interesting use of clickers.  Peter Saunders of Oregon State University told me that he once observed an art history professor using clickers to help students develop critical thinking skills.  She had a matrix with artistic movements along the top and elements of a painting (type of brush stroke, etc.) running down the side.  She would show her students a painting and ask her students who the painter was, what movement the painting belonged to, and what elements (brush stroke, etc.) they used to determine the movement.  All of these questions were clicker questions, and she used them to help her students better understand how experts analyze paintings.  This activity struck me as very appropriate for the discipline and for the use of clickers.

That’s it for today.  There were several other topics that come up repeatedly throughout the day, especially learning space design and the UMW Blogs project, but I thought I would keep the focus here on classroom response systems!

Following up on my last post, I wanted to share one interesting way free-response questions are being used at Abilene Christian University.

Kyle Dickson, in his portion of the ACU presentation today, described a tool they’ve developed for their iPhone-enabled classroom response system.  An instructor can pose a free-response question such as “What 20th Century event has had the most impact on American society?”  (Kyle used a question like this in his example.)  Students then submit their responses to this question using their iPhone.  The instructor can then view those responses, select a subset of them (perhaps the most popular, perhaps ones she’s particularly interested in discussing), and turn that subset into the answer choices for a multiple-choice question (with the same question stem) to be asked of all the students.

This sounds like a great tool to me.  Often, instructors aren’t sure how students will respond to a given question, so letting it be open-ended is helpful to see the range of student responses.  However, once an instructor gets a better feel for those responses, a multiple-choice question based on those responses is useful for polling an entire class more systematically.  This process often happens outside the bounds of a single class session, with instructors constructing multiple-choice questions based on student responses to free-response questions asked as part of pre-class homework.  The ACU tool enables this entire process to happen during class, which sounds pretty exciting.

It’s getting late here in Orlando, so I’ll wrap up this series of blog posts.  More tomorrow!

Update: EDUCAUSE Connect has posted some resources for the Abilene session I attended, including links to videos demonstrating the iPhone tools ACU has developed.

I stopped by the Turning Technologies booth this morning.  Turning is one of a few classroom response system vendors with booths at the conference.  I’m hoping to check out the others tomorrow.  At the Turning booth, I saw a demo of their new ResponseWare Web application.  This application makes it possible for students to use laptops, smart phones, and any other Internet-enabled device to respond to polling questions during class.  One of the advantages of using this kind of application is that it makes it easier for students to respond to free-response questions since they have keyboards and other ways of quickly entering text.

It occurred to me that while interviewing faculty for my book, I found very few instructors who were using the free-response options provided by many classroom response systems in part because the methods that most clickers provide for entering text are pretty slow.  (I’ve met a few instructors who frequently ask numeric-response questions, which are a little easier for students to answer than text-response questions.)  I’ve also sensed that some faculty, upon hearing about clickers, reject the idea of using clickers because they feel that multiple-choice questions are too limiting.  I would argue that multiple-choice questions can be used productively in just about any class, but, regardless, there’s still this sense.

While at the Turning booth, I hypothesized that a system like their ResponseWare Web–one that made it easy for students to quickly respond to free-response questions–might appeal to faculty who don’t see much value in asking multiple-choice questions during class.

Later in the day, I spoke briefly with George Saltsman, who, along with his Abilene Christian University colleagues Kyle Dickson and Bill Rankin, presented a session today on ACU’s iPhone initiative.  At ACU, instructors are able to ask free-response questions and have students respond quickly and easily using their iPhones, so I asked George if many faculty members there were taking advantage of this ability.  He said yes and then confirmed my hypothesis from earlier in the day–this ability was a big draw for a lot of his faculty members, ones that might not have been interested in asking multiple-choice questions using clickers.

I’m eager to hear more about the experiences of faculty at ACU and elsewhere who are using classroom response systems to ask free-response questions.  The vast majority of work exploring the roles and impact of clickers in the classroom has focused on the use of multiple-choice equestions.  I think there’s a lot of potential for exploring the use of free-response questions, too.

Today was the first full day of the EDUCAUSE Annual Conference here in Orlando.  I caught several great sessions, and I thought I might share a few clickers-related highlights here on the blog.

First, I’ll follow up on yesterday’s thought about the ways in which Web 2.0 tools such as blogs and wikis can raise the bar for student participation by making students more publicly accountable for their work.  I attended a session presented by Gardner Campbell and Jim Groom in which they described UMW (University of Mary Washington) Blogs, a multi-user blogging platform available to all faculty, staff, and students at UMW.

Gardner noted that most of the scholarly work done by college students is neither public nor permanent.  Students turn in term papers, those papers are graded and returned to students, and that’s the end of them in most cases.  Having students post their work to blogs means the work is more public and more permanent.  Their work is seen not only by their instructors but by other students in the course and, as Gardner conveyed in a couple of anecdotes, students who took the course in previous semesters and sometimes the authors and artists about which students write in their blog posts.

The public nature of the blogging platform increases accountability for students and, at least at UMW, seems to help students engage more deeply in the learning process.  This talk helped reinforce for me the important role accountability plays in teaching and learning, whether that accountability is provided by blogs, wikis, or classroom response systems.  (For more on the UMW Blogs initiative, check out one of Gardner’s recent blog posts.)

In the interest of keeping my blog posts concise, I’ll stop here for this post and share further thoughts on the conference in my next post.

After my first day at my first EDUCAUSE Conference, I’ve learned a few things…

• The Orange County Convention Center is gigantic.  This may be the largest building I’ve ever been in.  It’s so large the conference staff ride around on Segues.
• Every other vendor exhibition at every other conference I’ve attended is small potatoes compared to the vendor exhibition here.  See some pictures at Christine Sexton’s blog.
• The free soft drinks provided by the conference on breaks are great, in part because the soda machines at the OCCC charge three dollars for a 20 ounce Pepsi.  That’s pricier than Disney World!

I went to one of the pre-conference seminars today and had a bit of a clickers-related brainstorm during a presentation on Web 2.0 tools in college and university teaching…

Let’s say you’re using a wiki with your course.  I’ve known for a while now that most wikis will allow you to see exactly how your students contribute to the wiki–how often they edit pages, what kinds of edits they make, etc.  I’ve been thinking of this kind of data in terms of what’s often called the scholarship of teaching and learning–asking questions about student learning and answering those questions by systematically collecting and analyzing evident of student learning.  The contribution data that wikis automatically generate seems to me to be a goldmine for understanding how students interact with course content.

I’ve also heard from many faculty members who use classroom response systems that the fact that students can be held accountable for their participation by such systems helps motivate students to participate during class.  They can no longer hide in the back of the class and let other students answer the instructor’s questions.  Not only that, but the use of clickers can communicate to students that their instructors are interested in hearing what they have to say.  Since all students are given a voice of sorts by the clickers, all students are given a chance to contribute to the class dialog.  As a result, the use of clickers raises the bar for class participation and many students respond very positively to this.

During the seminar today, I connected these two ideas.  Doing so helped clarify for me the value in Web 2.0 tools like wikis.  The tools that wikis provide for tracking contributions by students help to make public (to their instructors, to their peers, perhaps to the whole world) the work that the students do.  This helps hold the students accountable for their work, which can increase their participation levels.  Likewise, the invitation to contribute to a class wiki helps give more students a voice in the class, which can also enhance participation.

I’ve heard several instructors say that the accountability factor is a key ingredient to the success of clickers in promoting student engagement.  It seems to me that leverage that aspect of Web 2.0 tools like wikis will help those tools generate student engagement as well.

More tomorrow!