Turning Technologies User Conference, Dublin 2013

On Monday 4th November, I headed up to Dublin to attend the Turning Technologies User Conference at Trinity College Dublin. Although we are not a customer of Turning Technologies, we do have some significant experience with the use of Personal Response Systems, or clickers, which I've blogged about before. We have a large number of clickers made by eInstruction, which was recently bought over by Turning Technologies, so I was interested to see where the technology is heading.

Opening Keynote: Eric Mazur

The conference also gave me the opportunity to hear Eric Mazur speak. I've been aware of Mazur's work with clickers and peer learning for some time; his YouTube video Confessions of a Converted Lecturer is a joy to watch. His polished performance yesterday was an excellent demonstration of the techniques he promotes.

Although he was speaking to the converted (he didn't need to convince anybody of the need for more active forms of learning), Mazur kept us engaged and really eager to find out the solution to the thermal expansion problem he set.

Some of the main points I noted:

• A clicker is not just a polling tool, it's an engagement tool. Mazur does not recommend using it for tracking attendance or giving marks for the right answers. This might have the effect of getting the students into the classroom physically, but what you really want is their minds.
• We learn by practicing; we teach by telling. Why are we surprised that this is a problem?
• Lectures focus on information transfer. When lecturing, we tend to focus on what is being taught, not how we are teaching. We tend to simply replicate how we were taught.
• In the average lecture, there is hardly any interaction. What, therefore, do we lose by simply recording the best performers and putting it on the web? In fact, there is much to gain, because the student can hit the pause button, and have time to think.
• Education has to be more than information transfer. To have the ability to transfer what you know from one context to another, is the real essence of education.

I want my students to stand on my shoulders; to solve problems that are not yet solved - @eric_mazur at #ttucdublin
— Sharon Flynn (@sharonlflynn) November 4, 2013

Mazur then went on to describe and demonstrate how he uses clickers to support peer instruction. He described the curse of knowledge - the more expert you are in a subject, the harder it is to explain to a learner. Peer discussion works because the students are explaining a concept they have only just understood, and they know the possible misconceptions.With peer discussion, students become emotionally involved in the process (of working out a problem) and not just the right answer.

Keynote: Mark Taylor

The second keynote was from Mark Taylor, President of Taylor Programs. Dr Taylor is from Arkansas and, as he reminded us during his keynote, is an expert, speaker and consultant on the topic of Generation NeXt. During the next hour, he gave us a condensed version of what is usually a half day workshop on the topic of today's digital learners.

Initially, I wasn't sure if MarkTaylor was for real, or if he was a parody of a motivational speaker. In fact, he turned out to be completely genuine, and an engaging speaker. Some of his arguments were a little over-simplified, but this could be down to the fact that he was trying to get through a lot of material in a short space of time. From initial astonishment, I warmed to his techniques (including his bell of absolute silence).

Dr Taylor's starting point is that "traditional academic practice don't work like it used to", although I'm not sure that the traditional lecture ever did work. Colleges and Universities are getting blamed (by employers) because "we had them last"!

Today's students are from Generation NeXt: the era of the wanted, precious, protected child, who has grown up in a child-centric household. The child who gets trophies for "just showing up". As described in Time magazine, these are the Twixters, children who can't or won't (or aren't let) grow up; the Me Me Me Generation, who are responsibility averse.

And, apparently, this is all down to Reactionary Parenting - parenting in reaction to the way we were parented. Because of our parenting style, today's students have no respect for authority; they have a strong sense of consumerism and entitlement; effort is seen as indicating a lack of talent; they overrate their own skills and are given record high grades for decreasing effort. Because we offer options and choices, our children are less likely to persist, resulting in retention problems.


To be completely honest, I don't think I was the only person in the room who recognised some of these traits in myself, as a parent. And it did make me feel quite uncomfortable.

But, there was a positive message, ultimately.

The idea that attention spans are shorter in the latest generation may be a myth - Mark Taylor #TTUCDublin
— Christopher Wiley (@Chris_Wiley) November 4, 2013

Using the example of Minecraft, Mark Taylor pointed out that children from Generation NeXt frequently watch YouTube videos while playing the game, thus learning new techniques and skills. They are actively seeking out learning on their own. We need to learn from this more about how to engage and motivate our students.

All you need to know about the flipped classroom, according to Mark Taylor #ttucdublin pic.twitter.com/2yTYlg6oX5
— Sharon Flynn (@sharonlflynn) November 4, 2013

At this point, unfortunately, Dr Taylor ran out of time. But he did finish with the advice: Don't talk to the student, talk to the professional you want them to become.

Parallel Sessions

After lunch, I sat in on three sessions given by practitioners.

David Robinson, from Queens University Belfast, described the evolution of clicker use at QUB. From an initial successful pilot with 108 clickers and 2 receivers in 2005, they now have over 200 trained users, with popularity increasing. Schools have bought their own systems, for purposes ranging from interactivity/engagement to module evaluation. Particular issues David discussed include staff training, which they got right, and distribution models, which they initially got very wrong.

Will Moindrot, from Manchester University, spoke about his experience using ResponseWare, an over-web solution allowing students use their own mobile devices to vote. These can be used in tandem with clickers, as a hybrid solution. He found that students liked using their own devices, and ResponseWare was easier to support than clickers, but wifi needs to be ubiquitous and reliable.

Student thoughts on using mobile phone as clicker device #ttucdublin pic.twitter.com/FhAcE8KAL1
— Sharon Flynn (@sharonlflynn) November 4, 2013

The last session I attended was given by Javier Horta, from the University of Massachusetts Lowell. Javier described his use of the NXT device for high stakes exams in large classes, as an alternative to bubble sheets.

Hearing about using NXT devices for e-assessment - possible solution for high stakes exams? #ttucdublin pic.twitter.com/QifyUqRGaU
— Sharon Flynn (@sharonlflynn) November 4, 2013

The advantage of using the NXT device is that it allows more question types than MCQs - multiple answer questions, numerical answers, true/false questions and short text answers.  Students are given a paper question sheet and they submit their answers using the device, which they have to sign-in to. The grading happens immediately, and there is no need for large computer labs. The small size of the screen minimises a student's ability to observe a neighbour's entries.

Javier gave some good examples and advice on creating suitable exams. My impression is that the technology isn't quite there yet - for text answers, for example, there is a bit of manual manipulation of results to account for common mis-spellings. But I can see this approach being very useful for certain groups of students.

Final thoughts

With lots of other work going on here in Galway in the last year, I haven't had my head in the clicker space for a while, so it was good to revisit and find out about new developments. It is interesting to see the different groups looking at mobile apps for clickers, and I'm also watching Blackboard Labs' Polls tool, which is available for free in a beta version at polls.bb (see Steve Bentley's recent review).

As a user conference, I felt that there wasn't much opportunity to meet the user community on the day. There were plenty of scheduled talks, and room for questions, but very little time for chatting with other attendees. There was no list of participants available, which makes it difficult to follow up with people afterwards. I recognised very few people at the conference, which is unusual for an ed tech event in Ireland. So, I'm wondering who all the attendees were.

Following from this, I was surprised that there were no speakers from Irish higher education at all. There was one speaker from QUB, and four from other UK institutions. Why were none of the Irish champions included in the speaker list? Any why, out of 12 speakers, was there only one female?

The tweets from the event, using the hashtag #ttucdublin, have been collected together in this storify.

Related Posts

The clicker experience at NUIG: student feedback
The clicker experience at NUIG: Issues and concerns for staff


Tweet

The clicker experience at NUIG: Issues and concerns for staff

As I mentioned in my last blog post, I've been attending the iClicker Pedagogy Webinar series over the last few weeks, and find them very interesting. On 25th January, Roger Freedman (@RogerFreedman) spoke about Pedagogical Best Practices. On 15th February, Angel Hoekstra spoke about Pedagogical Strategies for Teaching with Clickers in the Social Sciences. This week, on 22nd February, Jennifer Kaplan spoke about her use of clickers in large Statistics classes. It's interesting to hear about the different approaches, and also to see the common themes.

In my post last week, I gave an account of the student feedback on the use of clickers in the College of Science initiative at NUIG. On the whole it was very positive, and it encouraged the College to expand the use of clickers to include 1st and 2nd year undergraduates. But, what about the staff issues?

Three group meetings were arranged during the academic year, when all staff involved in the clicker project were invited to come along and discuss progress, issues, problems, successes and to give suggestions. Initially well attended, the numbers coming to the meetings dropped significantly in the second semester. The issues that came up during the year reflect some of the findings of Roger, Angel and Jennifer.

Mandatory Use
Initially there was concern among some staff members that they were required to use clickers in their teaching. The strong message that came from the Dean's office was that, while nobody was forced to use the system, it was strongly recommended that staff come to training and consider how they might be used. Ultimately, uptake varied in difference disciplines.

Consistency of Use
There was a concern that, if clicker questions were not regularly used across all subjects, students might stop bringing them to class, thereby lessening the effect in those subjects that did use them. Although the use of clickers was not consistent across subjects, the survey of students at the end of the year indicated that 66% of respondents brought their clickers to every class, and a further 26% brought them to most classes.

Attendance Monitoring
There were a number of discussions and concerns around the use of clickers solely for the purpose of monitoring attendance. Everything in the literature indicates that this is not a good use of clickers, and likely to encourage the practice where students bring along 5 or 6 devices for their friends. At the same time, over time the data could be used to monitor trends in student participation, which could be used to identify students at-risk.

Through discussion, it was decided that clickers do not provide a reliable method of tracking attendance and that they were best used as a learning tool. At the same time, the use of clickers, where students find the activity valuable, can have the effect of increasing attendance.

At the end of the first semester, the Physics lecturers polled students (using clickers) about the possibility of awarding marks for participation, based on clicker use in class. The response was overwhelmingly in favour of using clickers and getting marks for participation. As a result, Physics students were awarded 5% of their overall mark for participation in the second semester.

This corresponds to what Roger Freedman described as low stakes clicker credit, in comparison to high stakes credit, where marks are awarded for a correct answer. Roger suggests that the choice of low or high stakes credit for participation can change the dynamics in a class. While there is no difference in the learning gain, high stakes credit can stifle student discussion.

Content and Flow
There was some initial concern that the time used in lectures for clicker questions would result in less time to cover content, and ultimately this did prove to be the case. However, this also raises questions about the responsibility of the student in the learning process, and how much they can be expected to read outside of class. This has the effect of starting to transform the underlying pedagogy to accommodate increased interaction and participation in class.

Use of clicker questions can give very valuable feedback to the lecturer who is concerned about content. A well-designed question can indicate whether a class is following a lecture, for example. As Jennifer Kaplan stated, you may be covering the material, but are they getting it?

Another, related, concern was that the flow of a lecture is broken when a clicker question is asked. Students may begin to chat, and it can be difficult to bring the focus back to the lecture. In fact, a clicker question should be disruptive, encouraging a student to think about what is being asked. In a peer instruction situation, students are encouraged to discuss the question in  pairs or groups.

Roger Freedman suggested that the best clicker questions are challenging, with multiple plausible answers, that reveal student confusion. Conflict leads to drama and gets the discussion going.

Conclusions
In particular, we found that the adoption of the clicker technology is unique to the culture and context of teaching within each discipline, and that this observation had to be factored into the training needs for different groups. The experience of peers is particularly persuasive, and we were lucky to have two academic staff members within the College who already had extensive experience of using clickers in their teaching. They both gave practitioner workshops at the start of the year, and were persuasive in their encouragement to use clickers.

Relating the NUIG experience to that of the experts in the webinar series, many of the issues (particularly around attendance monitoring, credit for participation, and the desire to cover content) are common. Some good tips I've picked up from the speakers are:

• Clickers are best used as a teaching tool, but even the best tools can be misused (Roger Freedman)
• Integrate clicker questions into your lecture, don't treat them as an add-on! You lecture less, and the students think more. (RF)
• Use clickers regularly, and tell students why you are using them.(RF)
• Explain regularly and often why you are using clickers and how the students will benefit. (AH)
• Experiment! See how it works best for you and your students (RF)

 Finally, I think the use of clickers can be an opportunity to change an approach to teaching, but this won't happen overnight. I was heartened by Jennifer Kaplan's answer to a question about the extra time required to prepare lecture materials to incorporate clicker questions. She responded that 5 years ago she didn't change her lecture material, but reworked some slides to incorporate clicker questions. Her approach has evolved and improved over time.

Tweet

The clicker experience at NUIG: student feedback

I've been "attending" some of the webinars in the iClicker Pedagogy Webinar series over the last few weeks. They are organised for 1:00 EST, which translates to 6pm Irish time. If I race home from work, I can just about catch them while watching over the kids.

On 25th January, Roger Freedman (@RogerFreedman), Physics lecturer and clicker enthusiast, talked on the topic of Clickers in the Classroom: Pedagogical Best Practices. Roger gave a lovely presentation, making a compelling case for using clickers and demonstrating that there can be a significant learning gain from integrating them into teaching. The webinar was probably more suited to people getting started with clickers, but it was very interesting to see some of our experience here in NUIG being mirrored in what Roger was saying.

In September 2010, 762 clicker devices were distributed to incoming first year undergraduate Science students and Foundation Medicine students at NUIG. In addition, 40 RF receiver devices were distributed to academic staff involved in the initiative. The aim of the project, led by the NUIG College of Science, was to use clicker devices in lectures to promote student intellectual engagement and also to increase attendance. CELT was involved to provide advice, training and support for academic staff during the year.

In March 2011, we invited student feedback on the use of clickers using an online survey. A total of 272 students responded, giving a 35% response rate.

What did they like?
When asked about what they liked about using clickers, a large number of students (71%) made some reference to active learning ("It made me sit up and think"). Smaller numbers of students mentioned anonymity as a positive, while 2 students mentioned social aspects - that clickers helped them get to know other members in the class.

What did they not like?
When asked what they didn't like, the responses were more diverse. A significant number (19%) said that the clicker questions were disruptive, that other students would start to chatter and it could take some time for the lecturer to regain control. Some students (12%) thought that the questions were not always being used well (questions too easy, questions too hard, trick questions) while about 7% of respondents thought that the clickers weren't being used enough.

None of the students appeared concerned about not covering material in lectures.

Attendance and Participation
In his talk, Roger noted the difference between Empowering and Compelling students to use clickers. They work best as a learning tool, and students prefer the formative feedback they offer, over grading or tracking attendance.

In the second semester of the NUIG Science clicker initiative, students were awarded 5% of the overall mark in Physics for participation in class using clickers. In the survey, when asked what they liked about using clickers, 18 students said they liked the participation marks. However, when asked what they didn't like about using clickers, 19 students said they didn't like the participation marks, because sometimes they forgot to bring the clicker, or it didn't work due to flat batteries. In contrast, when asked what would improve the experience, 22 asked for participation marks in all subjects, 2 asked that they be used to track attendance, while just one students said that there should be no marks for participation.


 Outcomes of the initiative
In the survey, 79% of respondents said that the system should be used with first year students again, with 9% saying no. In fact, the College decided to expand the scheme and in September 2011 all first and second year students were provided with clicker devices.

One of the aims of the project was to promote intellectual engagement. From the student responses to the survey, active engagement is reported as an outcome. This may be due to the use of the clickers themselves, the changed teaching practice as a result of the use of clickers, or just a novelty factor associated with the devices. The student feedback supports findings from more substantive studies in the literature that clickers can play a positive part in student learning.



Tweet

Clickers and Peer Discussion in the Classroom

Previously I blogged about a study of the use of clickers, or personal response systems, in the classroom and how a study had shown that students who used clickers earned exam scores that were 10% higher than students who didn't use them.

A new report, published in Science, by M. K. Smith et al, at the University of Colorado, demonstrates that the effectiveness of clickers can be greatly enhanced by having students discuss the questions, and their answers, in peer groups in class.

The study used a class of 350 undergraduate Science students, who were asked an average of 5 clicker questions per class. The students first had to answer a question individually and were then invited to discuss the question with their peers. Following the discussion, they could re-cast their answer. Unsurprisingly, the number of correct answers increased. However, when asked a second, similar question, the average number of correct, individual answers also increased. It appears that students learned from the peer discussion.

Moreover, students reported that "having someone in the group who knows the correct answer is unnecessary". The opportunity for peer discussion allows the group to explore the concepts and arrive at an understanding constructively.

While this doesn't tell us anything new about the power of peer discussion in the classroom, it does suggest how we can enhance the use of personal response systems.

Clickers in the Classroom

Today's Inside Higher Ed reports from the Blackboard World conference in Las Vegas about the use of audience response systems in the classroom, and whether it is a fad or a useful educational tool. It points to an interesting report on a study at Ohio State University, where clickers are used to engage and involve Physics students. It was found that students who used clickers earned exam scores that were 10% higher than students who didn't use them. Moreover, it seems that female students in particular benefited from the anonymity that clickers give, allowing them to actively participate in lecture sessions while keeping their involvement private.