The Chronicle of Higher Education recently reported (briefly) on a new survey from CDW, a “leading provider of technology products and services for business, government, and education,” indicating some differences in how faculty and IT staff view the role of technology in higher education. Here’s what caught my eye from the Chronicle story:

“The most popular tools cited by professors were e-textbooks and online documents, with faculty members reporting far less enthusiasm for other electronic tools. Under a quarter of faculty members surveyed use wikis or blogs in their teaching…”

While I’m an active user of blogs in my courses and I see a lot of value in wikis for student collaboration, the bit about e-textbooks and online documents doesn’t interest me that much. Those two technologies are more about content delivery than interaction. Sure, they have their uses, but they’re not as likely to lead to active learning experiences as more interactive tools.

What bugs me about this Chronicle story is that there’s no mention of classroom response systems. I mentioned this on Twitter, and a couple of people there poked fun at my tweet about this omission. Sure, I’m going to notice whether or not clickers are included in a survey like this. I did write a book about teaching with clickers, after all. However, it’s not that clickers weren’t addressed in the survey. In fact, they were listed right along with many other educational technologies as response options in the survey itself, and the publicly available report from CDW notes that 34% of institutions support the use of clickers by faculty. (More on that statistic below.)

I think what bothers me is that clickers rarely seem to rate a mention in stories like the Chronicle‘s on educational technology. Sure, e-book readers are all the rage these days and there are plenty of people in academia talking about the potential of e-textbooks. Blogs and wikis get a lot of attention, too, which is great since they are useful tools for fostering out-of-class interactions among students. But what about technologies that enhance the in-class experience for students? Yeah, I know I’m biased, but those are the technologies that I see as having the greatest potential to have a positive impact on higher ed. Why? Because what happens during class still looks a lot like it did 20, 50, or even 100 years ago. There’s great potential for growth there, and in-class, interactive technologies like classroom response systems can be a big part of that.

Back to the survey: Only 34% of IT professionals surveyed indicated that they support faculty use of clickers? That seems low to me, given that it seems that every campus I hear about has at least a couple of faculty members teaching with clickers. Perhaps at many of those places, that’s all there is: a couple of faculty members using clickers without any formal IT support. That would explain the 34% statistic.

This bothers me, too, particularly when, according to the CDW survey, 59% of IT professionals consider lecture capture technologies “essential” to the 21st century classroom. That’s just more content delivery. It doesn’t do much to increase student engagement and interaction. Yes, it’s true that students who know they can watch a lecture after class might take fewer notes and have more mental bandwith for paying attention and engaging during class. And lecture capture tools that allow students to collaboratively mark-up and share lectures after class have a lot of potential for outside-of-class interaction and learning. But why not put some more support behind a technology like clickers that’s designed to support formative assessment and student engagement during class?

Image: “Overhead Projectors at US Grant High School in Oklahoma City” by Flickr user Wesley Fryer / Creative Commons licensed. I’ve been wanting to use this image for a while now. What we consider an essential classroom technology one year can be a recycling challenge the next!

Back in January I gave a keynote talk at the Health Professionals Education Research Symposium hosted by Nova Southeastern University.  Part of my preparation for that talk included reading some of the articles from related disciplines in my clickers bibliography.  Shortly after the conference, I blogged about one great article about using clickers to promote critical thinking in nursing (Debourgh, 2008), and I’ve been meaning to post some notes about the other articles I read.  Let’s get started…

Reference: Cain, J., & Robinson, E. (2008). A primer on audience response systems: Current applications and future considerations. American Journal of Pharmaceutical Education, 72(4), 77.

Notes: The literature review is the highlight of this article.  It’s not as comprehensive as other lit reviews, but it does a great job of describing a few studies of the use of clickers in the health professions with particularly positive results.  For example, Slain et al. (2004) report that students in clicker sections of two pharmacy courses scored significantly higher on exams than students in non-clickers sections.  Similar results were found by Schackow et al. (2004) in classes for family medicine residents and by Pradhan, Sparano, and Ananth (2005) in classes for obstetrics and gynecology residents.  These references are listed in my bibliography.  Hopefully, I’ll find some time to read and blog about them soon.

Cain and Robinson also include a useful exploration of some of the logistical aspects of teaching with clickers.  Instead of making recommendations, they describe the various choices a department might make and their pros and cons.  They note that any clickers initiative should make sense given an institutions teaching philosophy and technology plan.

For example, a pharmacy school with a mandatory laptop program may highly value an ARS that can utilize laptops as response devices, rather than basing the decision on other features.

They also recommend purchasing a set of clickers available to faculty and staff to check out for one-shot events, like continuing education programs and faculty meetings.

The section on recommendations for future research is a strong one.  Cain and Robinson write, “Any effects from using an instructional medium do not come from the use of the media itself, but from the instructional methods employed.”  That’s something I’ve argued here before.  Cain and Robinson call for research that explores the effects of very particular instructional strategies involving clickers, including strategies useful for facilitating discussion about matters of ethics and morality.  While ethical issues are present in every discipline, they are often particularly important in professional education.

Cain and Robinson make an interesting statement in their section on student considerations: “Finally, appropriate application of the ARS in the curriculum should be defined and encouraged.”  I understand the interest in encouraging instructors to use clickers in appropriate ways.  It’s the “defining” piece that makes me wonder if pharmacy education is a bit more top-down than the kinds of programs you find in, say, colleges of arts and science.  I find that faculty members in undergraduate liberal arts departments tend to have a high degree of autonomy when it comes to their teaching decisions.  They might not be comfortable having appropriate uses of clickers “defined” for them.  Am I reading too much into this word choice?  Does your department (whatever your discipline) set policy on educational technology use?

Image: “Rx, San Antonio, TX” by Flickr user Tadson / Creative Commons licensed

Michael Bugeja, director of the Greenlee School of Journalism and Communication at Iowa State University, has written another opinion piece for the Chronicle of Higher Education.  You may recall that last year, Bugeja used a Chronicle essay to criticize classroom response systems as not worth the cost, although as I pointed out in my blog post about the essay, he didn’t really consider the benefits of clickers in his cost-benefit analysis, just costs.  This time, Bugeja’s essay, “Reduce the Technology, Rescue Your Job,” makes the argument that in a time of budget-cutting, if colleges and universities don’t scale back on their use of technology, “secretaries, janitors, adjuncts, advisers, and professors” are likely to lose their jobs.

As I wrote in my response to Bugeja’s last essay, I agree that colleges and universities should consider the cost of new educational technologies, not just their potential benefits.  And Bugeja points out some costs, such as the cost of local tech support required by some technologies, that may not be immediately obvious to some involved in these kinds of decisions.  However, Bugeja’s argument in support of his main point-that colleges and universities should scale back their use of technology-is unclear in places, and I find that the evidence he provides is only somewhat relevant to his argument.

My main concern with Bugeja’s argument is that it confuses teaching about digital technologies (such as journalism courses on uses of social networking sites) with teaching with digital technologies (such as teaching a journalism course on, say, ethics using classroom response systems or Twitter).  For instance, he begins his piece by stating that “professors [have] embraced the pedagogy of engagement” by using technologies to “woo” students.  That sounds like he’s focusing on teaching with technology.  However, when he shares changes his school has made to reduce technology, almost every change he mentions concerns streamlining his school’s curriculum so that there are fewer courses about technology.

This mixing of teaching about technology with teaching with technology muddies the issues involved.  For instance, it might be a poor choice (from a cost-benefit standpoint) to add courses to a major that are focused on learning about particular technologies if those technologies (as subjects of study, not as pedagogical tools) can be incorporated into existing courses.  However, that doesn’t imply that using educational technology in courses is a poor choice.  Those are two separate choices.

Even within each of these areas (teaching about technology and teaching with technology), Bugeja’s evidence is of questionable relevance.  For instance, he points to the great numbers of courses and degree programs in video-game design as an example of new courses designed “to accompany the gadgets that students brought with them” in an attempt to engage those students.  However, given that the United States computer and video game industry brought in $11.7 billion in revenue in 2008 (compared with “only” $9.8 billion for the motion picture industry in the same period), it seems to me that teaching students to create video games is a way to prepare them to enter an incredibly productive sector of the US economy, not just an attempt to tap into their leisure activities.

Regarding teaching with technology, Bugeja again omits any mention of possible or proven benefits to student learning provided by educational technologies.  He also blurs together several very different educational technologies, including classroom response systems (“clickers”), virtual worlds such as Second Life, and the microblogging platform Twitter.  These technologies are used to facilitate student learning and engagement in very different ways and at very different price points.

For example, using Twitter for in-class or out-of-class engagement with students is essentially free, since the service itself has no charge, it is simple enough so that instructors and students using it need very little tech support, and its bandwidth needs are minimal given that all the messages it facilitates are limited to 140 characters.  Virtual worlds like Second Life, on the other hand, typically require significant resources (mostly time) to prepare for use with students.  The upfront and ongoing costs are much more significant than with Twitter.

Even though I’m no big fan of Second Life in higher education, I should point out that in some settings, particularly ones in which the skills to be learned can be practiced meaningfully in virtual environments, virtual worlds can be very effective ways to promote student learning.  They can even be cost-effective, given that virtual simulations can be less expensive than real-world simulations in some cases.  Thus, the cost-benefit analysis for Second Life can sometimes come out in favor of its use.  When looking at other technologies like Twitter and clickers, where the costs are much less, it’s all the more likely that a cost-benefit analysis will lead to adoption.

Given that Bugeja seems to be against the use of technology in almost any way in higher education (going so far as to rail against libraries subscribing to digital versions of journals!), it’s not surprising that he would criticize teaching centers for conducting workshops on educational technologies.  He compares teaching centers to “brand managers” for technology vendors.  However, many teaching center workshops focus more on using technology to engage students (to “deep critical thinking and inspired commitment,” to use Bugeja’s definition of engagement) than they do on promoting particular vendors.  Certainly, when I give workshops on teaching with clickers, I focus on ways to ask clicker questions that foster higher-order thinking skills and that engage more students more actively in class discussions.  When particular vendors are mentioned in these kinds of workshops, it is usually because (a) instructors interested in tech-enabled pedagogies need some basic level of tech support that is vendor-specific and (b) a single vendor has been adopted across the campus in order to reduce costs, both good reasons for mentioning particular vendors.

Bugeja lists two changes at his school of journalism that are more about teaching with technology than teaching about technology.  He mentions a renewed emphasis on “the fundamentals of teaching excellence-preparation, organization, and mastery of the subject matter,” as well as an emphasis on “spending more face time than Facebook time with students” in the service of academic advising.  Let me be clear: preparation, organization, and mastery of the subject matter are indeed fundamentals of teaching excellence.  (Take a look at students evaluations in a course where one of these is missing to see how important each of these is!)  And when it comes to academic advising, face time with students is usually a good thing.

However, I would argue that there are at least a couple of other “fundamentals of teaching excellence,” including formative assessment (gauging student learning prior to evaluations of their learning in order to provide feedback to students and inform teaching decisions) and skill in planning and facilitating active learning experiences for students.  Ignoring these two areas can sometimes privilege certain students-students who learn in particular ways, often the ways we learned.  For instance, a well-prepared, well-organized lecture demonstrating mastery of the subject matter can work very well for some students-those who learn deductively, sequentially, aurally, and reflectively, for instance.  (I’m drawing on the Felder-Silverman learning styles model here.)  Students who learn better inductively, globally, visually, or actively, however, might not benefit as much from this learning experience.

Since educational technology can often be very helpful in facilitating formative assessment and active learning experiences, it seems a shame not to explore the possibilities technology might provide in these areas-especially in teaching center workshops aimed at promoting student learning and engagement!

Similarly, most students might benefit from academic advising conducted face-to-face, but is it really that hard to imagine students who might benefit more from advising conducted via Facebook, Twitter, or email?  Commuting students, for instance, who are rarely on campus, or students who can express themselves more effectively in writing than while speaking or students who only warm up to faculty members once they get to know them on a slightly more personal level first?  Should we privilege other students over these types of students by discouraging the use of digital communication technologies?

Given Michael Bugeja’s comments on my previous blog posts, I suspect that he’ll read this one and respond to it.  I’m glad for him to do so, because the issues he raises are important ones.  Given his advice to those making technology decisions to assess the costs and benefits of technologies they implement, I would be interested in hearing from him how his school is evaluating their decision to reduce teaching with technology.  Are they investigating whether or not focusing on preparation, organization, and mastery of the subject is indeed “instill[ing] in learners a commitment to make a difference in society”?  Does more face time and less Facebook time with students actually lead to higher retention rates?  These are genuinely interesting questions to me; they are just as important to ask as questions about the cost of technology.  I would welcome more insight into how they might be answered at Iowa State.

For a few more good thoughts on Bugeja’s essay, check out Britt Watwood’s blog post, “What Walls Need Tearing Down?“  What was your response to the essay?

My last couple of blog posts here have focused on pedagogical possibilities of a class full of students with smart phones or laptops.  While I’m excited by those possibilities, I also think it’s worth considering some of the more practical aspects of using mobile devices in the classroom.  Jason B. Jones, who teaches English at Central Connecticut State University, recently shared on the Prof. Hacker blog his experience teaching a summer course in which every student was given an iPod Touch.  His “lessons learned” focused mostly on practical issues:

• Plan to deal with access issues.  For instance, some of his students didn’t have their own computers, making it difficult for them to sync their iPod Touches with new content.
• Don’t assume that your students are as tech savvy as you might think they are.  As D. Askey said in the comments on Jason’s post, “The ability to use Facebook adeptly does not indicate technical or Internet savvy in any way whatsoever.”
• On the other hand, students can come up with uses you might not consider.  Make sure you learn from their ingenuity.
• Jason’s campus network didn’t allow peer-to-peer wireless connections, which rendered some of the apps he wanted to use inoperative.  Watch out for network issues like this.
• If you’re lending devices to students for a course, make sure you have a plan for getting them back at the end of the semester!

Are you experimenting with using mobile devices in the classroom?  If so, what lessons have you learned?

Jennifer Imazeki teaches a 500-student microeconomics course at San Diego State University, and she recently blogged about her use of clickers in this course.  In her post, she describes some advantages of campus standardization, her grading scheme, and her students’ (generally positive) response to learning with clickers.  She also describes a couple of teaching choices she’s made that I find particularly interesting.

Last semester, I also made a quiz available on Blackboard that students could take if they missed class; I take the higher of their clicker score or quiz score for a given day.

Imazeki notes that by giving her students the option of taking an online quiz, she minimizes the number of students who show up to class just to get their participation points.  Since these students are often somewhat disruptive in class, this works out well for the students who attend class while also giving Imazeki a way to keep tabs on the students who don’t attend class.

Imazeki also describes her use of the “pick-a-random-student-who-responded” feature of her classroom response system.

I tend to use this when I have asked the class to brainstorm examples or asked them a question that doesn’t really have a ‘wrong’ answer. Although students don’t love it, they don’t seem to hate it either.

As I’ve mentioned before, this feature can help prevent cheating with clickers (one student responding with an absent student’s clicker).  In a class of 500 students, it also offers a useful way to “cold call” students without having them feel like they’ve been singled out.

Imazeki also notes that she’ll have students draft responses to an open-ended question, then display a clicker question with preset answer choices.  Students are then asked to select the answer choice that matches their draft response.  With graphing questions such as the one she notes as an example (“Use a supply and demand graph to show what happens to price and quantity if X happens”), this approach seems particularly useful since drawing the right graph is a harder task than selecting the right graph from a set of options.

In a subsequent blog post, Jennifer Imazeki shares more results from a survey of her students about her use of clickers.  She notes what percentage of her students agreed with a variety of statements about the positive impact of clickers.  Then she writes:

The percentage agreeing with these statements has risen each of the three semesters I’ve taught the large lecture and the percentage disagreeing has fallen.

I think this is an important point.  Often when we try something new in our teaching, it doesn’t work out as well as we would like.  We haven’t really figured out how to make it work, and the students are used to it either.  Sometimes a teacher trying out something new and receiving poor feedback from her students gives up the innovation and reverts to her previous teaching methods.  However, as Imazeki’s data show, our use of instructional techniques can improve over time with practice and feedback.

(I’ve just noticed that this is the first time I’ve blogged about using clickers in the discipline of economics.  I have a couple of economics examples in my book, however, and I’ve heard from several economics instructors, including ones who teach very large classes like the one described here, that clickers can work very well in their discipline.)

Teaching with clickers provides instructors with a wealth of information they can use to learn about the effectiveness of their own teaching.  Finding out that half your students don’t understand a topic (as evidenced by wrong answers to a clicker question) minutes after you’ve explained it can be disappointing, but it’s better to know your students are confused than to assume they’re following along.  More direct clicker questions (like, “How well are you following my lecture right now? Very, somewhat, a little, or not at all?”) can also provide formative feedback on one’s teaching.

But what about using clickers for more summative evaluation of one’s teaching?  Might clickers be used in place of the handwritten or online end-of-semester course evaluations?  I haven’t heard of many schools doing so, but I did speak with someone involved in such an effort back at EDUCAUSE in the fall of 2008:

I spoke with Danny Sohier of Université Laval in Québec after the session.  His school is using clickers to conduct end-of-semester course evaluations during class.  They found that online course evaluations resulted in low response rates, a problem I’ve heard about from many institutions.  They now use clickers to collect student responses to multiple-choice evaluation questions during class in some courses, inviting students to respond to open-ended questions online outside of class.  Danny indicated that this arrangement is working pretty well.

These seems sensible to me.  The advantage of handwritten course evaluations completed during class is that response rates are fairly high, since often most of the students enrolled in a course show up on the day evaluations are completed.  The disadvantage is that handwritten evaluations take more work to analyze.  Using clickers during class to ask these kinds of questions keeps response rates high and yields data that are easy to use.

The advantage of online course evaluations is that students can take their time and compose thoughtful and lengthy replies to open-ended questions about one’s course or teaching.  The disadvantage is that relatively few students do so!  The system described above allows motivated students to submit thoughtful responses to open-ended questions after class, while hearing from all (or almost all) the students in a course on some useful multiple-choice questions during class.

This topic has been on my mind since Nira Hativa of Tel Aviv University posted an inquiry to the POD Network listserv about using clickers for formal teaching evaluation.  Kevin Owens of Turning Technologies outlined one way to do so:

1. Evaluation questions are entered ahead of time into a PowerPoint presentation (via our TurningPoint software which is free) and saved onto the memory stick or network drive.
2. Instructor leaves room
3. Selected facilitator (staff member, student worker, etc.) loads interactive PowerPoint presentation and distribute clickers to students
4. Selected facilitator administrates the questions giving 5 to 10 seconds on each question to allow students time to submit responses
5. When finished, facilitator saves data onto memory stick or network drive for future reporting.
6. Saved data can produce up to 30 automated reports within our TurningPoint software or can produce raw data available for export into existing reporting tools on your campus.

Mark Scarbecz of the University of Tennessee College of Dentistry pointed to a poster he presented about UT Dentistry’s experiences using clickers for evaluating teaching.  His conclusions?

Response rate and acceptance of the ARS for course evaluation were greater than for a web-based system. The ARS was effective and efficient for data collection. Random selection of keypads provided anonymity. ARS software had multiple formats for data reporting. Limitations of the ARS are the following: 1) a small question set reduces the length of evaluation sessions and student boredom, but also information collection; 2) student conversations during sessions may bias responses; 3) the ARS provided no mechanism for open-ended feedback; and 4) development/ presentation of question sessions and dissemination of data are timeconsuming and labor-intensive.

Nira Hativa replied to the listserv to note that her situation has a particular challenge.  The courses she’s evaluating rotate instructors every two or three class sessions, so waiting until the end of the semester to collection feedback on those instructors isn’t practical.  She discussed this challenge with Mike Theall of Youngstown State University, and it appears that she doesn’t have the staff power to send a facilitator in these classes every two or three class sessions to conduct this evaluation, the students might not provide honest feedback if the instructor is the one administering the evaluations, and some of her instructors might object to one of the students in the class proctoring the evaluations (as has often been done in the past with paper-based evaluations).

If the student proctor option is off the table, I’m not sure Nira’s problem has a solution.  However, I throw the question to you now: Any ideas for helping Nira?  And do you have any experience with or ideas about using clickers for course evaluations?

At a recent conference, I met Kevin Barhydt, an educational technologist at Union College in New York.  His presentation about clickers was great, but one thing that stood to me about Kevin was a pattern I observed in the questions he asked during other presentations.  He kept wanting to know if the clicker questions used by instructors during class were made available to students after class for review and study.

I think this is a great question.  There’s some evidence in an article by Bunce, VandenPlas, and Havanki (2006) that making clicker questions available to students after class can enhance student learning.  In that article, the impact on student learning of online quizzes was compared to that of in-class clicker questions.  It was found that the online quizzes helped students more, perhaps because those quiz questions were made available to students for study, whereas the clicker questions were not.

I recently learned of a new project out of the University of Ediburgh called Electronic Voting Analysis and Feedback for All (EVAF4All).  The goal of the project is to develop a vendor-independent way to provide students with useful feedback about clicker questions after class.  The feedback would include a student’s individual answer to a clicker question, the distribution of answers for the student’s classmates, and links to educational resources related to the topic of the clicker question.

There are some significant technical challenges that the EVAF4All project will have to overcome, including the development of an online application that will interface with any classroom response system (regardless of vendor) and any learning management system.  You can read a little more about these challenges in a blog post by Nitin Parmar of the University of Bath.  However, given the potential importance of the ways that students can learn from clicker questions after class, I think the EVAF4All’s project goal is a good one.  I would be interested in finding out what kind of assessment they’re planning for their project.  That might help discover the role of post-class review of clicker questions in student learning.

Reference: Freeman, M., Bell, A., Comerton-Forder, C., Pickering J., & Blayney, P. (2007). Factors affecting educational innovation with in class electronic response systems. Australasian Journal of Educational Technology, 23(2), 149-170.

Summary: Fourteen faculty and staff involved in a clickers pilot project reflected on their experiences in the project (some in writing, some in structured interviews) using questions based on Rogers’ model of the diffusion of innovation. Each question was tied to one of the five factors identified by Rogers that drive adoption considerations–relative advantage, cultural compatibility, complexity, trialability, and visibility. Two non-adopters were included among the fourteen. This article presents an analysis of these reflections.

• Relative Advantage – The article includes a nice inventory of reasons clickers can improve student learning and faculty job satisfaction according to the instructors studied. These advantages, however, were offset by the time (time to learn the technology, time to learn the pedagogy, time to set up before class each day) and risks (risks of technical failures, risk of being surprised at what faculty learned about student study habits and misconceptions) involved in using clickers. Faculty not already using active learning strategies in the classroom found that the disadvantages generally outweighed the advantages.
• Cultural Compatibility – When discussing this factor, the authors expand on the notion that faculty already using active learning strategies (in support of constructivist pedagogies) find clickers more compatible with their current teaching practices than faculty not using such techniques. The authors also note a few aspects of faculty culture (promotion and reward systems, for instance) that are risk-averse to teaching innovations, particularly those involving technology.
• Complexity – Clickers presented some hardware issues, but problems learning to use the software were even more significant, particularly for faculty not already using PowerPoint.
• Trialability – Given how difficult the system was to learn and to set up on a daily basis, it was difficult for faculty to trial the system.
• Visibility – The private nature of teaching limited the visibility of faculty use of clickers among other faculty. Also, the pilot participants had on the whole poor experiences with the technology, so they were less inclined to tell their colleagues about it.

The authors provide some advice for dealing with the challenges to adoption outlined above.

1. Provide technical and administrative support to faculty using clickers.
2. Promote faculty with constructivist pedagogies to leadership positions.
3. Utilize project teams to support faculty.

Following up on the third point, the authors provide some advice for such project teams:

1. Include a mix of faculty and staff.
2. Include not only innovators and early adopters, but also “early majority” faculty (to use Rogers’ terms).
3. Include mostly faculty who are constructivist in their pedagogies.
4. Include discussion of research on instructional technology.

Comments: It’s important to note that this pilot project was conducted in 2004-05. Classroom response system hardware and software have improved in ease of use and reliability greatly since then.

I agree with the authors that instructors already using constructivist approaches to teaching are more likely to see clickers as compatible with their teaching. However, adopting clickers can often be seen as a “low threshold activity,” as least as far as changes to one’s teaching practice are concerned. (Technical hurdles can sometimes be more significant, as mentioned in the article.) Instructors used to lecturing can fairly easily start to use clickers effectively by asking an application clicker question a couple of times during a class session.  Doing so gives students a chance to apply what they’ve just heard about and provides the instructor with a useful assessment of student learning, while not requiring the instructor to change his or her teaching practice in big ways.

This idea that faculty already using student-centered teaching approaches are more likely to make use of clickers and use them well was mentioned in Fies and Marshall (2008).  The question I asked in my post about that article is relevant here, too: How can instructors using clickers be helped / encouraged / supported in using student-centered approaches to teaching with clickers?  I think Ian Beatty’s suggestion in the comments on that post was a useful one–share and discuss with faculty examples of clicker questions designed to meet useful pedagogical goals.

Interestingly, there’s not much mention of student resistance to clickers in the article as a compatibility issue or a risk of using clickers. In my experience, student resistance can be significant if clickers aren’t used in ways that students perceive as contributing to their learning, particularly if students are shouldering the cost of the hardware.

The authors make a good point about the difficulty of trying out clickers for a class period or two. Many teaching and technology centers and offices make available sets of clickers for faculty to borrow for this purpose.  Perhaps it makes sense to also offer the services of a “clicker facilitator” who would actually set up and run the system during class for the faculty member. That way, the faculty member could try out the system without having to learn much at all about it.

I think the visibility issue the author raise is also important.  It is too rare that college and university instructors visit each other’s classrooms.  In some places, classroom visits only happen for evaluation purposes or when an instructor is in trouble of some kind.  This is a wasted opportunity, however, since all instructors can benefit from learning about each other’s approaches to teaching.  I would bet that instructors on the fence about using clickers who visit the classroom of an instructor experienced in using clickers would benefit from that visit by seeing in very concrete ways how clickers can be used to engage students and “uncover” student misconceptions.

Update: I just read a great post by Stephanie Chasteen exploring Rogers’ model of diffusion of innovation.  She discusses how the factors mentioned above affect the adoption of educational innovations, and even makes the point that I did that clickers can be used in ways that are compatible with the traditional lecture method of teaching.

Okay, one more post in the series, then it’s back to some discussion of “old-fashioned” clickers.  One interesting aspect of the ConnectEd Summit was that everyone at the conference had an iPhone or iPod Touch.  The conference organizers made loaner iPod Touches for attendees who didn’t already have one of these devices.  They also created an iPhone application for the conference.  This app featured a place for conference announcements, an attendee directory (complete with photos of attendees taken during registration), and a copy of the program.  Most of the sessions at the conference also had dedicated collaboration spaces within the app, with areas to add relevant links and simple discussion forums.  We encouraged to contribute to these space during and between sessions as a kind of backchannel for the conference.

One issue that quickly emerged was that the local wireless network had trouble handling 400 people trying to access it simultaneously.  Since we were also using our mobile devices to respond to “clicker” questions during some of the keynotes, there were several occasions when all 400 of us attempted to go online in the same 60-second window.  This didn’t always work, although the conference organizers did their best to work on this problem.  One solution that seemed to help was to ask iPhone users to turn off their wireless and use their 3G cellular connections.  ACU has two 3G AT&T towers on campus, so the 3G coverage is excellent.  One takeaway from this is that other campuses interested in leveraging mobile devices in the classroom will have to work on network solutions that support this much data traffic.

Another thing I noticed was that participation in the discussion forums waned during the conference.  It took a little while for discussion to get started during the opening keynote, but by the end of it, there were a dozen or so posts in the forum for that keynote.  During the first breakout session I was in (the “pedagogy and praxis” session), there were a lot of interesting comments made on the forum for that session throughout the session.  However, during the later breakout sessions in that track, there was very little backchannel discussion.  I think this was due to the fact that these later sessions consisted almost entirely of small group discussion, whereas the first session was mostly one-to-many presentations (including one by yours truly).  It seems that backchannel works better during lectures / presentations than during group discussions.

That leads me to wonder about the purpose of these kinds of backchannel discussions.  If participants in a session are more likely to contribute to backchannel discussions during a presentation, what does that imply about their interest level in the presentation?  Are they more engaged because they are commenting on and asking questions about the presentation?  Or are their backchannel comments indicative of boredom?  In my experience at the conference, there were times when I fell into the former category and other times in the latter category.  And what effect does this kind of backchannel conversation have on presenters?  A very interesting post by Olivia Mitchell on the Pistachio Consulting blog titled “How to Present While People Are Twittering” argues that not only should presenters encourage backchannel, they should respond to that backchannel if possible.

And what about backchannel during small group discussions?  At one point we had five groups of five to eight participants each very actively engaged in small-group discussion during our session.  No one was backchanneling during this time, probably because they were all engaged in their local discussions.  Could backchannel have been used to cross-pollinate the small-group discussions, however?  If someone had a great idea in one group, a group member might post that to the backchannel so that it might be taken up in one of the other small groups.

The conference organizers also announced a Twitter keyword for the conference, #acuconnected.  I’ve been on Twitter for a year, but it was during the conference that I first tweeted anything.  Because of this, I forgot to include the #acuconnected keyword in a couple of my tweets, meaning they weren’t part of the overall conference stream of tweets.  This also meant that I had to figure a few things out about Twitter, like how to search for tweets with a particular keyword and how to reply to people’s tweets.

The nice thing about using Twitter was that comments weren’t embedded in discussion forums for individual sessions; there was only one stream for the entire conference.  This meant that by following that stream, I could get a sense of what was going on in other sessions at the conference, which was useful.  (Apparently, the podcasting track had some great discussion!)  Another thing I observed was that although backchannel discussion waned in the conference app near the end of the conference, it stayed relatively steady on Twitter.  Perhaps those of us who noticed that discussion in the conference app was waning sought out a larger audience.

Thinking about the uses of mobile devices I’ve mentioned here is important, I think, as we consider ways to have our students use these devices productively.  I welcome any thoughts you have about backchannel conversation, especially.

Back in December, I shared some thoughts on Michael Bugeja’s Chronicle of Higher Education essay titled “Classroom Clickers and the Cost of Technology.”  My post generated a lot of comments.  Bugeja’s essay generated other responses as well, including three letters to the editor published last month in the Chronicle.  All three letters point to educational research conducted on teaching with classroom response systems that Bugeja chose not to mention in his essays, which was my response to the essay, as well.

For instance, Stephanie Chasteen of the University of Colorado-Boulder wrote:

Mr. Bugeja hypothesizes that students would vote against the use of clickers because the costs outweigh the benefits. Research suggests otherwise. In our own large introductory-physics courses, 95 percent of students stated that clickers helped them learn the material. Studies in other disciplines suggest that students are more likely to value clickers when they’re used to promote discussion, rather than to ask simple questions or take attendance.

Doug Duncan, also of the University of Colorado-Boulder, wrote:

Most of the practices [Bugeja] describes are what our research shows to be worst practices. We see them fail, too. When instructors use clickers as part of peer instruction and explain to students that they will attend class more, work harder, learn more, and be rewarded for that, peer instruction and clickers produce learning gains. When instructors ask low-level memorization questions and don’t explain why they are using clickers, students call them dumb and worthless.

I’ll continue to explore and discuss the research on teaching with clickers here on this blog.  Given the clear learning gains that clickers can facilitate as well as the cost of the technology, it’s important to give due consideration to both sides of the cost-benefit discussion.

Further thoughts on Bugeja’s essay or on these responses to it?