[c]

An Examination of Clicker Technology Use in Legal Education

Catherine Easton

Senior Lecturer in Law

Manchester Metropolitan University

---

Abstract

Recent technological developments have led to a situation in which the employment of clicker technology in the law school lecture theatre is now a feasible possibility. Influential studies carried out in pure science disciplines (Hake, 1998; Crouch and Mazur, 2001) indicate positive results in both engagement and assessment success attributed to extensive clicker use. There is however a paucity of studies outlining the tailoring of this technology to the specific nature of legal education. This paper presents the findings of a small-scale use of clicker technology within the context of a wider study addressing issues of lecture engagement across a law course. These observations are drawn into the existing debate on clicker use by highlighting key emerging themes and commenting upon their potential impact within the field of legal education. An overview is then presented of the small body of literature on clicker use to teach law. This is then analysed to make observations on the opportunities for legal education presented by clicker technology and the factors affecting its adoption on a law school-wide basis.

---

This is a Refereed Article published on 22^nd December 2009.

Citation: Easton, Catherine, "An Examination of Clicker Technology Use in Legal Education", 2009(3) Journal of Information, Law & Technology (JILT), <http://go.warwick.ac.uk/jilt/2009_3/easton>

---

1 Introduction

1.1 What is a clicker?

Clicker is a term employed in this paper and others to describe audience response systems which enable a lecturer to record student responses within the learning environment. Typically they are developed in the form of a handheld device with buttons resembling a mobile phone which allow for the transmission of numbers and letters. As with much educational technology there are commercial considerations inherent in institutional choice of clicker system. Due to an ongoing institutional trial, the clicker system employed in the MMU LLB study was provided by Promethean and included options for multiple choice, yes/no, true/false, numerical and freetext responses which could then be manipulated on the large screen of the lecture theatre. Lowery (2005) and Barber et al (2007) provide in-depth overviews of the available interactive systems and their relative merits.

1.2 Background: Lectures and interaction

While addressing issues relating to the delivery of lectures this paper does not seek to evaluate the theoretical rationale behind the initial decision to adopt the lecture as a teaching method. A body of work, within which Gibbs (1981) and Bligh (1998) are prominent influences, questions the effectiveness of a large lecture to promote deep learning and suggests that lectures operate as little more than a tool for dissemination of information. Notwithstanding this key debate, this paper operates within the constraints inherent in the operation of a large law school which, due to a variety of factors, and akin to the majority of law schools across the UK, has chosen to employ the lecture as a baseline teaching tool across the vast majority of modules presented. This paper will therefore move away from any discussion of the merits of a large lecture per se and will focus upon initiatives to develop strategies to harness available technology in a bid to encourage student participation and engagement.

The conventional lecture can be depicted as one in which the expert speaker imparts information via a monologue delivered to a passive audience (Goffman, 1991). However, this approach is criticised as representing an outdated paradigm (Johnson et al, 1998). A growing body of literature (eg, Hake, 1998; Northcott, 2001) highlights the ability of an interactive lecturing style, in which student participation is actively encouraged, to foster a deep approach to learning and higher levels of conceptual understanding. These findings have been supported by research in the field of cognitive psychology which finds a direct link between effective learning and active participation which engages multiple centres of the brain (Zull, 2002).

Educational research has developed to address a need to test strategies, approaches and tools which can aid a lecturer to introduce effective interaction between lecturer and student and among the students themselves. At a basic level this can be achieved by soliciting responses to issues posed by the material covered. Meltzer and Manivannan (1996) describe the positive effects of increased student engagement from a large lecture approach which employed a series of coloured flashcards to enable students to communicate responses to questions presented. Technological developments now mean that this approach can be replicated on a much more sophisticated level through the use of interactive systems. Judson and Sawada (2002) indicate that in a variety of guises classroom interactive systems have been in existence and use since the 1960s. Within the last decade however, the available technology has been enhanced and modified to make adoption of these systems a feasible proposition for the modern law school.

In 1999 the United States National Research Council (1999) hailed classroom interactive systems as a key transformative innovation which could revolutionise educational delivery and thereby develop the fully empowered critical learner. Indeed, classroom interactive systems are now available as standard in the large lecture theatres of Harvard, Vanderbilt, Duke, MIT, Yale, Brown, University of Massachusetts-Amherst, and University of Virginia (Ewing, 2007). Similarly, many UK institutions such as Lancaster University (Elliot, 2003) and the University of Glasgow (Draper and Brown, 2004) have at differing levels adopted this technology or are in the process of trialling available systems.

There is therefore a burgeoning body of research into many aspects of clicker use based upon a variety of pedagogical and technological foundations. A good overview of this research can be found in Fies and Marshall (2006) and Simpson and Oliver (2007). A striking theme within this body of work is the high prevalence of research activity in the disciplines of pure science, medicine, engineering and mathematics. As there is no accepted premise that higher education practitioners in the more humanities-based fields are less likely to publish findings related to academic practice, a conclusion can be drawn that adoption of clickers has developed at a slower level across non-science based subjects. Indeed, the available research publications on the use of clickers to teach law are minimal in comparison with those relating to the pure science of physics. A potential explanation for this could be that the level of technical expertise held by staff across disciplines will affect uptake but this may be too blunt a conclusion to draw. Caldwell (2007) however, in her paper "Clickers in the Large Classroom: Current Research and Best-Practice Tips" references research into clicker use across eighteen disciplines including nursing, communication and philosophy but does not refer to any studies specifically focusing on law. Issues relating to this observation will be discussed below.

1.3 The remit of the MMU LLB study

This paper seeks to present the findings from a small-scale trial of interactive technology to teach law. A summary of the work carried out will be presented and this will then be analysed against a number of key themes which emerge from existing literature. Given the relative paucity of literature within the sphere of legal education a separate evaluation will be carried out of the available literature on the use of clickers to teach law.

In the light of a larger study to assess the link between lecture engagement and success the following research was carried out:

• Interactive lectures

• Fulltime LLB: The subjects were first year full-time undergraduate LLB students (n=242) following a module in public law. One two-hour lecture within a series of six was delivered employing clickers as a key part of the presentation. The topic of the lecture was parliamentary sovereignty and the EU. Each student was provided with his or her own device; these devices were distributed at the beginning of the session and collected in afterwards.

• Part-time LLB: The subjects were second year part-time undergraduate LLB students (n=18) following a module in criminal law. One one-hour lecture within a series of 18 was delivered employing clickers as a key part of the presentation. The topic of the lecture was partial defences to murder. Each student was provided with his or her own device; these devices were distributed at the beginning of the session and collected in afterwards.

• Paper-based questionnaires: "Student questionnaire (full-time)" "Student questionnaire (part-time) A paper-based questionnaire was developed following methodological approaches outlined in Oppenheim (2006). This was distributed to the two cohorts. The questionnaire was administered directly after the interactive lectures and a decision was taken to employ a paper format as it was determined that the response would be higher if the students were asked to respond immediately and within the lecture environment. Within the full-time cohort 231 viable questionnaires were returned, 18 were returned by the part-time cohort.

• Web-based questionnaire: "Lecturer questionnaire" A web-based questionnaire designed using SurveyMonkey technology was administered to the 31 lecturers involved in the delivery of large lectures on MMU's LLB. This questionnaire focused on general issues of student engagement and interaction, not specifically clicker technology. This was designed and administered following relevant methodological considerations (e.g. Lazar and Preece, 1999). 19 viable responses were gained.

2. Discussion

Key themes from the literature surrounding the use of interactive technology will now be outlined and findings from the MMU LLB study of clicker use will then be analysed alongside these emerging debates.

2.1 Potential reasons for clicker use:

2.1.1 To assess prior understanding

Herr (1994) comments that before using clicker technology to assess understanding of mathematical and scientific concepts, lectures would proceed based upon assumptions of knowledge gained from prior learning. Using clicker technology can enable the lecturer to evaluate any gaps in knowledge or misconceptions and build responses to these into teaching strategies.

On a wide basis induction and introduction to legal study teaching sessions could employ clickers to determine any misconceptions year one LLB undergraduates hold about key elements of the legal system or legal education, perhaps based upon approaches taken in the media or even prior education. These could then be addressed within the module presentation. On a narrower basis clickers can be employed to assess and refresh understanding from previous sessions. For example, in the session on provocation and crime clickers were used to ascertain knowledge of homicide and manslaughter classification in general. The lecture can then quickly move on and build upon this knowledge with relevant new material.

2.1.2 To test students' engagement with set reading

Burnstein and Lederman (2001) and Knight and Wood (2005) report positive findings from the use of clicker technology to assess student engagement with set tasks and recommended reading. This approach could be used within an LLB presentation to evaluate student understanding of, for example, a case or journal article which they have been required to read before attending a lecture. Clicker questions could focus on basic facts to determine that students had completed the task or include more conceptual questions testing higher level evaluation skills. The lecture could then be developed based upon a lecturer's heightened insight into student understanding of topic.

2.1.3 To provide formative feedback

The positive effects of formative feedback have been determined in a large body of literature (eg Boyd, 1973; Gall, 1984). It has been found that being called upon to answer frequent, structured questions and receive immediate, constructive feedback can significantly enhance student learning (Guthrie, 1971; Kulik and Kulik, 1988). In these studies there was a strong correlation between the immediacy of the feedback given and its effectiveness to promote understanding. This constant feedback within face-to-face sessions was deemed to have a greater impact upon developing cognitive learning processes than the delayed feedback given after an assignment submission. One particularly interesting study undertaken by Frase (1970) found that the assessment scores of less-motivated students improved to a level similar to that of highly-motivated learners when throughout a course they were asked to respond frequently to formative questions.

The findings from this existing body of research carried out in a traditional environment can easily be extracted and applied to create a rationale for clicker use within the learning environment. Once the technology has been mastered, clickers pose an ideal opportunity to develop strategies in which immediate formative feedback is provided to each individual learner within a large lecture environment. A large number of studies on clicker use have reported increased assessment scores linked directly to clicker-facilitated constant formative feedback (d'Inverno et al, 2003; Roschelle et al, 2004).

The opportunity to enable a learner to determine how his or her understanding may be flawed as a lecture progressed was noted by one of the MMU LLB questionnaire respondents:

"The clickers are a great help as you know where you went wrong and you then have the ability to determine the correct answer" Student questionnaire - Full-time

2.1.4 To "break-up" the lecture session

Research such as MacManaway's (1970) indicates that the longest period of time for which students can effectively engage with a lecture is 20-30 minutes. Clickers can be used to provide variety in the lecture presentation which can increase interactivity and therefore engagement. Allen and Tanner (2005) report positive findings through the building of clicker use into lectures to assess understanding periodically. Additionally, Lloyd (1968) stresses the importance of the first five minutes of a lecture which is often the period most effectively recalled by students and which is often disrupted as the students take time to settle down. Employing a clicker question at the very beginning of a session could compel students to engage with the valuable early minutes of a lecture presentation.

Anecdotally, this aspect of clicker use was highlighted by one of MMU LLB part-time student survey respondents:

"I found that I was fully engaged throughout the session. The questions acted as mini intervals - very good."

This finding could be particularly important for the part-time learner who, as outlined in large scale studies such as Bourner et al's (1991), may experience issues which affect engagement based upon personal and financial considerations but also upon the realities of part-time face-to-face learning, which by its nature often sees a working day or week followed by evening or weekend study. Again, anecdotally, the increased engagement associated with clickers and part-time study was succinctly encapsulated in another MMU LLB questionnaire respondent stating:

"They kept my attention through the lecture which is sometime difficult with having lectures late in the evening."

This is an area for further research but the impact of clicker use on the part-time learner's experience is a key area on which the expanding and diversifying law school should focus.

2.1.5 To administer summative assessments

The majority of research on clicker use depicts its employment in order to assess understanding on a formative basis. However, the technology is robust enough to, with careful planning and practical consideration, support summative assessment. Bernstein and Lederman (2001) report the findings of a study in which a small but significant percentage of the final module grade, ranging from 15-25% was awarded based upon clicker responses in lectures during a number of introductory module courses. They recorded increased levels of completed prior-reading, attendance and engagement which increased further the higher the percentage of the overall assessment assigned to clicker responses.

Bernstein (2001) acknowledges that a prior level of technical competence and experience with the clicker system had been attained before the decision was made to employ them to assess summatively. However, notwithstanding practicalities, the technology is now available to assess, for example, a percentage of a large introductory LLB course via responses given in lectures. This, following the literature outlined in this paper, could have positive effects on attendance, engagement and ultimately assessment and retention.

2.1.6 To promote peer learning

Influential studies carried out in the large lecture environment of a physics presentation (Mazur, 1997; Crouch and Mazur, 2001) show a higher level of conceptual understanding and ultimately assessment attainment gained through using clicker technology to promote discussion and debate among a student cohort. Students can be called upon to work together to submit answers which the lecturer then reviews to find any common themes or gaps in understanding. Hake (1998) employed similar techniques and determined a significant improvement in problem-solving skills, again in studies carried out within a large cohort of physics students.

Within the academic study of law problem-solving skills are essential to achieve higher levels of conceptual understanding and sophisticated use of clicker technology could therefore revolutionise teaching strategies. Again, this is an area in which subject-specific research needs to be carried out, this issue is addressed further below.

2.1.7 To promote and monitor attendance

A growing body of research carried out within UK and worldwide establishments (eg Nicholl and Timmins, 2005; Gump, 2006) identifies student non-attendance as a problematic cross-disciplinary issue in higher education. This phenomenon can be evaluated alongside a general acceptance that attendance correlates positively with increased assessment scores, particularly in year one modules (Longhurst, 1999; Sharma 2005).

Woods and Chiu (2003) report an increase in attendance rates due to the adoption of clicker technology; they attribute these findings to increased student participation and ownership. Similar results were outlined in El-Rady's study (2006) in which clicker use was found to increase attendance and ultimately exam scores.

In a large law school strategies can be established to monitor attendance and act on these findings. Small group sessions lend themselves to effective student monitoring but the large, for example 200+, lecture poses problems which can be overcome by the use of technology. Requiring all students to sign a paper register can significantly diminish available lecture time. These problems can be overcome through the use of technology such as swipe cards (Brooke, 2004) but the use of a named clicker would allow effective monitoring in addition to accepted pedagogical benefits. This approach would come from the "kill as many birds as possible with one stone" school of academic practice. However, Zhu (2006) found negative responses when students perceived that clickers were being used primarily to monitor attendance without any tangible teaching and learning rationale. Following this, any strategy which focuses upon clicker use in order, among other aims, to monitor attendance across a year or LLB programme needs to ensure that teaching staff engage with the teaching and learning rationale behind interactive technology and are fully supported in doing this.

2.1.8 To foster a sense of community

Drawing upon key issues found in the integrationist approach to student retention outlined in the work of Tinto (1993), further work (Braxton and Hossler, 1996; Tinto 1996; Yorke and Longden, 2004) has focused upon the positive impact that the development of a community of learners and the fostering of social interaction has on increased rates of student success. These findings are particularly relevant to the first year experience (Beil, 1999) and correspond to responses gained to focus group research carried out among year one LLB learners at MMU (yet to be published).

Responses to these findings have included further support for student societies, informal group meetings and interactive game-playing within induction sessions. Crucially, in addition to the added interaction fostered by taking a peer learning approach, Draper et al (2002) report an increased feeling of community achieved via clicker use. Their study employed clickers within induction and introductory sessions to ask students questions related to themselves and their approaches to learning. The students were then placed into groups based upon the similarities or differences in these responses and encouraged to discuss related issues and report back on these findings. With innovation and creativity these strategies could be adapted to promote a sense of community and a shared learning experience among a large group of LLB students which ultimately could impact positively on retention.

2.2 Key general themes

2.2.1 Student evaluation of clicker use

The vast majority of studies (eg Conoley et al, 2006; Duncan, 2006) report positive student feedback in response to clicker use. Judson and Sawada's (2002) retrospective overview of research into interactive systems found:

"Polls from the 1960s through the late 1990s found that the use of electronic response systems made students more likely to attend class, pressed them to think more, promoted them to listen more intently, and made them feel instructors know more about them as students ."

In a survey of 1,500 clicker users Trees and Jackson (2003) found:

• Most students think that clickers give them valuable feedback.

• Many students enjoy the interaction that clickers provide.

• A group of students dislikes clicker use.

• Explaining how clicker use can benefit students and how it supports your class goals improves students' attitudes toward using clickers.

After the sessions delivered in the MMU LLB study student feedback was elicited via paper questionnaires and recorded as followed:

Statement posed in the student questionnaire Fulltime(FT) Part-time(PT)	Strongly agree	Agree	Neutral	Disagree	Strongly disagree
I feel that the lecture time would have been better used by the lecturer giving a traditional lecture	FT: 0% PT: 0%	3% 0%	7% 5%	15% 15%	75% 80%
Use of the clickers helped me to gauge my own knowledge in relation to the material	FT: 63% PT: 85%	19% 10%	16% 5%	2% 0%	0% 0%
The session with the clickers was well-organised and aided my own independent learning	FT: 68% PT: 80%	17% 20%	13% 0%	2% 0%	0% 0%
The session with the clickers aided my interaction with others in the cohort	FT: 35% PT: 30%	20% 35%	38% 25%	7% 5%	0% 5%

While these results indicate a very positive response to a one-off use of clickers there is, as outlined in work such as Hatch et al's (2005), the very obvious impact of novelty on the evaluation of any newly employed technology. A large scale study within a law school encompassing clicker use throughout the presentation of a module and tied to assessment achievement needs to be undertaken to enhance the results presented here. The responses to the open-ended question: "Please outline any positive or negative aspects of clicker use" were categorised following a basic coding system (Patton, 1990). Of the responses to this question (full-time n=90, part-time n=15) 9 could be deemed to be negative and of these negative responses 3 addressed issues of anonymity, 4 concerns over time available to cover all material, 1 question delivery and 1 a fear surrounding potential overuse, this final comment read:

"Using the clicker is 'fun' but I feel that using them all the time could distract everyone from the actual lecture." Student questionnaire - Full-time

In one of the few clicker studies which presents negative student feedback, Knight and Wood (2005) found that students do not support clicker use if it is perceived to add little to the presentation of a lecture and serve as a disruption to the presentational flow. These findings highlight the need for a sound pedagogical framework surrounding the choice of interactive technology and also, crucially, that meticulous planning is essential to enable effective use of clickers. Researchers such as Elliott (2003) draw attention to the extra time and effort needed to develop materials and strategies to support clicker use. The preparation needed for the clicker sessions outlined in the MMU LLB study took roughly twice as long as that needed for a "traditional" lecture. A law school wishing to embrace clickers would need to address this issue when developing a policy to support staff in the development of interactive learning resources if a decision were made to adopt this technology.

The majority of positive responses to the open-ended question related to increased levels of engagement (64) and increased understanding (28), they included:

Student questionnaire - Full-time:

" Using the clickers to answer questions allows me to know when I am wrong and this has brought awareness of topics where I need to do further background reading ."

"They help to gain a better understanding for kinaesthetic learners like myself."

"They kept me awake!"

Student questionnaire-Part-time:

" We're able to ascertain whether our learning is in-line with others, whether we're struggling with a specific topic area but not communicating this to the lecturer. Concentrated much more ."

2.2.2 Anonymity

The majority of clicker systems offer a choice between requiring the student to input his or her name and be identifiable, or the generation of a random code which allows for anonymity within the lecture environment. An important pedagogical rationale for the adoption of clicker technology in place of an approach based upon a show of hands or flashcards is that less confident students can be encouraged to participate due to the potential for anonymous responses (Burnstein and Leon, 2001; Davis, 2003; Draper and Brown, 2004). Indeed Ward et al (2003) in a study of interactive technology use to teach chemistry write:

"Even in small-enrolment classes, many students are reluctant to respond to faculty questions; the anonymity of responding with a hand-held device guarantees near or total participation by the entire class ."

In lectures where attempts at interactivity can be dominated by the responses of the confident, vocal minority, anonymous interactive devices could provide an ideal way to engage the more retiring student. The dangers inherent in the division between those who enjoy participating in a large groups and those who do not were highlighted by a respondent to the MMU LLB study's lecturer questionnaire stating:

"being too ambitious in trying to engage students in interactive work in the context of very large groups can be dangerous and can end up 'losing' some of the cohort ." (LLB Lecturer questionnaire: Interactivity in lectures)

The potentially anonymous nature of clicker technology presents a unique opportunity to reach hitherto passive students, particularly when covering sensitive subject areas (Abrahamson, 1999). They could therefore be employed in legal education sessions which cover potentially sensitive subjects such as law and morality, gender and law and medical ethics.

Within this small-scale use of clickers however a decision was taken on both occasions to require a student to input his or her name into a device. By manipulation of the relevant technology the named responses of the students could then be displayed upon the big screen. This, along with a familiarity with the cohort, enabled individual students to be called upon to state how they decided upon a correct answer or, for example, whether after further explanation a student understood why a previous answer was incorrect. This approach was employed particularly in the clicker session presented to the large full-time cohort. The debate on clicker anonymity can be encapsulated in certain responses to the open question in the student questionnaires:

"The use of clickers was really effective and should be used more-but people should not be highlighted if they gave wrong answers - some privacy needed ."

"The clickers allowed students to speak up and say whether they understood something or not, whereas the individual would just stay quiet in order not to look stupid ."

"People are usually too afraid to say when they don't fully understand, the use of the clickers enables students to express truthfully their opinions and whether they are struggling "

The named method adopted may appear to run counter to that recommended by existing work and further research would be needed to determine the best approach for the cohort in question. However, at a basic level the response rate was not affected by this decision and response rates stayed at an extremely high level throughout both sessions. This could be due to the nature of the questions posed and the issues covered in the session. A further point in relation to the high level of responses could be that this was influenced by the personal style of the lecturer. Inherent in clicker use is the need for a more energetic, engaged, flexible style of lecturing in order to encourage the students to respond effectively. These pressures indicate that the employment of clickers may not be feasible for lecturers with different pedagogical approaches.

2.2.3 The "teachable" moment

Wood (2004) outlines the concept of a "teachable moment" which arises when the lecturer presents a formative question relating to prior material and receives a very low level of correct responses. Research such as Sabine's (2005) indicates that after students determine that a large group of their cohort do not understand a concept they are much more responsive to any subsequent explanation of this concept. A lecturer prepared to adapt lecture delivery to seize upon a "teachable moment" determined through clicker use can, by revisiting a concept in a different manner, employ this situation to great pedagogical advantage.

Within the MMU LLB study's use of clicker technology to teach public law a "teachable moment" arose when a multiple choice question concerning the nature of EU sovereignty in the light of the UK's doctrine of parliamentary sovereignty received a correct response of just 31%. This led to a change in lecture delivery in which these concepts were revisited in a more detailed and simplified manner. Without the clickers, understanding would have been assumed and the lecture would have continued, building upon weak conceptual understanding. A question on the same issue framed in a different way later in the lecture received a correct response rate of 88%.

The web-based lecturer questionnaire which covered general issues of student engagement in lectures received 87% of responses agreeing or strongly agreeing with the statement: " The delivery of a lecture should be adapted as the lecture progresses if a lack of student engagement is perceived." An effective lecturer can from past experience and relationship-building with a cohort determine strategies to assess engagement but clicker technology can take this a step further and serve as a tangible method of determining conceptual misunderstanding.

2.3 Factors inhibiting clicker uptake

With any teaching and learning innovation there are a number of factors which across all higher institutions affect widespread employment and engagement with new technology. These include a lack of management commitment, a lack of adequate training, past educational experiences and the drive to teach as we have been taught (Timmermann, 2003). Dufresne et al (1996) present these issues as inhibitors to the institution-wide uptake of technology such as clickers and state that further factors include:

2.3.1 Pressure to cover content

The need to cover a wide, set amount of material is a familiar experience of a lecturer in higher education, particularly in a discipline subject to monitoring by an external body, such as law. Blodgett (2006) reports a key worry expressed by staff on the suggestion of extensive clicker adoption was the fear that it would inhibit the ability to "cover the material". The majority of studies (eg Duncan, 2005) admit that clicker use does leave less time to cover material in the time available, this could be due to issues surrounding set-up or simply the time taken posing questions and waiting for answers. This reality is a particular concern for clicker use employed to deliver intensive courses or part-time courses which often have less face-to-face time than full-time equivalents.

Within this small-scale use of clicker technology the need to cover material in the given time was a significant factor and both sessions overran their allotted time slot. This factor was highlighted by student respondents to the questionnaire writing:

"In shorter lectures, it might take up valuable lecture time. Especially as on the part-time we have quite limited contact time at uni". Student questionnaire-Part-time

"Better understanding however can waste info time". Student questionnaire - Full-time

Effective planning and content management is needed in the use of clicker technology, this again needs lecturer and management commitment. Advocates of the system such as Beatty (2004) believe that the benefits of increased participation and engagement outweigh the extra constraints upon time. Indeed, use of the clickers to assess basic factual knowledge based on prior reading could be employed to ensure that students achieve a background understanding of key issues which might usually have been outlined in a lecture, leaving the rest of the lecture free to build upon this knowledge.

2.3.2 Lecture theatre management

Dufresne et al (1996) also state as a possible hindrance to technology adoption lies in a "concern about management of students in a large lecture hall while teaching in an alternative format."

Indeed when asked to answer the open question in the lecturer questionnaire: "What are the factors, general or specific, that you believe can help to improve student engagement in lectures?" a respondent focused upon issues of lecture theatre management stating:

" With groups of 100 or more I think that trying to get a lot of interactivity amongst the students in a cramped lecture theatre would result in chaos, noise and little concrete achievement. I aim for quietness and enthusiasm ." MMU LLB Lecturer questionnaire: Interactivity in lectures

This is an interesting response which outlines a lecturer's personal style and approach to teaching. It needs to be recognised that despite the recorded benefits of clicker use there are many extremely effective pedagogical approaches which foster engagement and encourage interaction using little more than the skills of the lecturer him or herself. The use of clicker technology can be constraining and restrict a lecturer's ability to teach using existing, effective strategies. This paper in no way seeks to state that clicker technology should be employed by all in place of tried and testing methods but aims to highlight the potential advantages such as those discussed in 2.2.2 above regarding its ability to aid in the engagement of interactivity on a large scale.

In the MMU LLB study increased noise levels in comparison with similar traditional lectures were determined, particularly when questions were posed as students conferred with peers to reach an answer. This had to be managed in order to promote engagement and strategies were employed using the available software in order to manipulate the time given for response. This is another issue which indicates a need for training and commitment at institutional level to promote and support the adoption of clicker technology.

2.3.3 Fear of technology

A significant barrier to clicker employment can be grounded in the lecturer's concerns either that he or she lacks the skills to engage with the technology or that it will fail in a highly visible setting. Inherent in any use of technology in a learning environment is the presumption of basic level of competency on the part of both lecturer and student. Duncan (2005) recommends testing the equipment before using it in the lecture theatre and Draper (2002) advocates taking some time to instruct the students on clicker use before tying it to substantive lecture content. In the MMU LLB study a number of questions were devised at the beginning of each session based around facts relating the institution's home city, Manchester. These tested all the available functions of the devices (multiple choice responses, true/false, free text) and the students were then able to use them effectively as the lecture progressed. On the fulltime degree none of the students raised issues of technical complications. However, a on the part-time degree responded:

" I specifically need more experience in using clickers especially when I do not use a mobile but with more experience I see this method of lecturing as supportive in knowledge acquisition and interaction with fellow students ." Student questionnaire-part-time

When teaching with clickers, particularly on their initial use, the lecturer needs to pay attention to any technical difficulties recorded by students and respond accordingly. This may be a more onerous task in extremely large lecture theatres but in the MMU LLB study, perhaps due to a clicker's functions being very similar to that of a mobile phone, no technical problems were observed.

Notwithstanding students' expertise with clickers, their employment obviously involves a level of technical competence on the part of the lecturer not only in the original engagement with the software and question design but also in the lecture itself in order to manipulate effectively the presentation of the answers and respond to any new directions posed by students' answers. In the MMU LLB study the software was tested and trialled in the lecture theatre before its use and it was found to be extremely user-friendly and able to be manipulated to pose further questions based on past answers. Perhaps the most pertinent of a lecturer's technical concerns would be the scenario in which a lecture had been developed interweaving clicker responses into the presentation and the technology, for whatever reason, fails. Clickers not only rely upon the chosen system's software and hardware functioning correctly but also the lecture theatre's audio-visual system performing successfully. Silliman et al (2004) report that the small amount of negative feedback received from the students tended to focus around frustrations created by technology failure. In the MMU LLB study technical support was put in place for the large group session but this would not be feasible on an ongoing basis.

The need for staff development and training and the provision of glitch free technological systems and effective support can only be met with a strong commitment to extensive clicker employment at an institutional level.

3. How can clicker technology be employed to teach law?

3.1 Overview

As outlined above, the majority of studies carried out on the use of clicker technology have been undertaken in science-based disciplines. There is very little work which draws upon these findings and relates them to the specific nature of law teaching. The following is an overview of available clicker research focusing on the findings relating specifically to the teaching of law.

In the United States Paul Caron "one of the first two law school professors to use the interactive system" outlined in his conference speech Teaching with Technology in the 21^st Century Law School Classroom (Caron, 2005) his experiences of clicker use to teach law. His use of clicker technology in the outset related mainly to the employment of multiple choice questions (MCQs) and, when travelling to other law schools to demonstrate clicker use, this reliance on MCQs was criticised for being too restrictive for non-technical legal areas. Caron, however, refutes this by portraying an approach which gives the students, in small groups, a variety of answers and then requires them to discuss the relative merits of each before responding. This approach not only fosters peer learning but highlights that in many areas of law:

"there seldom are "right" answers but only "better" answers in particular contexts." (Caron, 2005)

Caron continues to outline the possibilities of clicker use to enhance discussion of judicial opinions:

"after calling on a student to relay the facts of the case, I ask the students which of the opinions best resolves the legal issue at hand and then call on students to explain their choices. When the discussion is complete, I ask the students to again choose their favorite opinion, and several students typically will change their "vote" in light of the discussion and choose another of the opinions as their favourite [sic]. These approaches, of course, work just as well (if not better) in courses like Constitutional Law than they do in courses like Tax."

Caron's 2005 conference paper built upon previous exploratory work undertaken with clicker technology in collaboration with Rafael Gefy and published in the Journal of Legal Education as "Taking Back the Law School Classroom: Using Technology to Foster Active Student Learning." (Caron and Gefy, 2004). The authors state that clicker technology:

"responds to the failure of law school teaching to encourage active learning by the entire class. Unlike the traditional Socratic method, which engages one student at a time, the CPS extends the dialogue to the entire class by requiring each student to respond to each question ."

Kelley Burton (2004) outlines the findings of a study to assess the effectiveness of clicker technology to teach property law. Teaching materials were devised through which a lecture would be delivered on a specific aspect of property law, the students would then be given:

" four factual scenarios relating to adverse possession time limits. A multiple choice question was devised for each of the four factual scenarios. Each multiple choice question had four responses. One of the responses was correct. Each student was issued with a keypad and they were given approximately five minutes to read and respond to each question ."

Positive findings were reported in relation to student engagement, participation and feedback, mirroring the results of the majority of the literature outlined above. However, a distinct disadvantage of clicker use was the increased workload on the law lecturer and clicker use was not advised on an ongoing basis. The author holds that clicker use could enhance the teaching of any legal subject, not just property law, stating:

"If the area of law is black and white, the law lecturer could create Powerpoint slides that have multiple choice questions, true or false questions or yes or no questions. If the area of law is grey, perhaps the lecturer could create a statement and ask the law students to agree or disagree. The responses to the statement could lead to a discussion within the lecture group ."

Burton highlights that the literature specific to the use of clickers to aid legal learning is limited as perhaps is its uptake, and states:

"Law lecturers should catch up to lecturers in other disciplines and embed an audience response system in their lectures".

This reflects the observations above outlined in section 1.2, that within the discipline of law there has been little development of the pedagogy surrounding clicker use. Perhaps with a greater uptake of the use of this technology a body of work could develop which highlights the potential use of clickers in a discipline-specific manner. It could be argued however that the lack of discipline-specific clicker use is due to the need to build a more conceptual understanding through legal teaching which makes it less suitable for clicker use than the more fact-based aspects of the pure sciences and mathematics. Draper (2002) carried out a study into clicker use across a number of disciplines including psychology and philosophy. The students' evaluation of clicker use was equally positive in these two disciplines as that recorded in relation to biological science and medicine. As part of this research a philosophy lecturer who used clickers to present an introductory logic course stated:

"There have been two noticeable results so far. The first is that, if the students are to answer the questions in a way that will be helpful to them, they have to reflect more on what they have learnt and how they are learning. The second is that my teaching is being directed more by what the students need, or at least, say they need, rather than what I think they need. This means that I am not second-guessing or making unwarranted assumptions about their progress."

This could be taken to indicate that clickers can be used with positive results in more theoretical disciplines such as philosophy and law which hinge upon understanding concepts rather than the repetition of facts.

3.2 Examples of specific teaching strategies employed in the MMU LLB study

Within the MMU LLB study clickers were employed throughout the lecture presentations at roughly ten minute intervals. The following are some very specific examples of how they were used to outline just some of the potential ways in which clickers could be integrated into a law lecture:

Public law (year one fulltime): Parliamentary sovereignty and the EU:

• Assessment of prior learning : Based upon the material covered in the previous lecture the students were asked to type in the number of Member States of the EU. This was followed by a multiple choice question based upon parliamentary sovereignty and implied repeal.

• Peer learning : The students were asked to discuss in small groups the whether or not the following statement was true or false "Parliament cannot amend the Bill of Rights" and submit answers via clickers. The same approach was taken with the statement "Where two Acts of Parliament conflict the courts will apply the earlier one". The students were called upon to discuss how they alighted upon their answer and whether they understood why they were correct/incorrect.

• Community building : A Likert scale question was developed to elicit answers to the statement: "I am confident that I can stand up and pronounce the case name: Internationale Handelsgesellschaft mbH v Einfuhr und Vrratsstelle fur getreide und futtermittel". Only one student responded to this with "Strongly agree", he stood up and pronounced the name beautifully to loud applause from the rest of the cohort.

Crime: (year two part-time): Partial defences to murder

• Critically analysing legal principles . In relation to provocation and the potential impact of a perceived cooling-off period the students were explained the principles held in the cases of R v Duffy ((1949) 1 AER 932) and R v Ibrams and Gregory ((1981) 74 Cr App R 154). The facts of the subsequent case of R. v Baillie ([1995] 2 Cr. App. R. 31) were then outlined and the students were asked to choose the most likely approach taken in this case from a series of options. All of the students chose an incorrect answer and a discussion then arose based upon how this case could be distinguished from previous legal authorities. The students seemed comforted by the fact that the whole group would have expected a different approach and this led to a lively discussion.

3.3 General considerations

• Installation of clicker systems : A very practical drawback of clicker use centres on the provision of the clickers themselves. In the MMU LLB study clickers were given to the students at the beginning of the lectures and retrieved at the end. On an ongoing basis this approach would only be feasible for a group of sixty or under and the large group session could not have been presented without the help of the university's Centre for Learning and Teaching. This can be addressed by installing clickers within the lecture theatres themselves (Cromie, 2006) or asking the students to buy or rent their own clickers throughout the course. In relation to this however, Zhu (2006) states: "With tight instructional technology budgets, students will most likely need to purchase their own clickers… It is important for faculty to be sure that they will really use clickers consistently in class before requiring students to buy them."

• Clicker fatigue : A factor in the positive response of clicker use to teach law as outlined in the MMU LLB study could be influenced by its novelty value in relation to the previous traditional lectures. Following the literature outlined above, clickers can provide an extremely effective aid to teaching and learning but as Simpson and Oliver (2006) state, students react badly to clicker use "just for the sake of it".

These issues both indicate that to support the use of interactive technology policies need to be put in place not only on an institutional but also a course and even year level to identify strategies for effective clicker employment across the modules presented to an entire cohort.

4. Conclusion

Increasingly within higher education a need has been identified to encourage the acquisition of skills which can be transferred to enhance a student's wider experience and activity. This debate has been embraced by legal education with the development of strategies to promote the identification and development of transferable skills (Law Discipline Network,1998). Key studies (eg. Hake, 1998; Knight and Wood, 2005) highlight the ability of effective clicker use to promote high-level conceptual skills and deep understanding. A discipline such as law, not only due to recent focus on transferable skills, but also due to a traditional focus upon analytical problem solving should therefore be at the forefront of clicker use and experimentation. The uptake of this technology, for a number of key reasons outlined above, has been low within legal education and an opportunity is being missed to revolutionise the legal classroom.

Oppenheimer (2003), however, warns against adopting new technology due mainly to its novelty and availability rather than a solid pedagogical rationale for its employment. While the majority of relevant research presents positive findings these have been undertaken by those most likely and able to innovate using clickers, as Kirkwood and Price (2005) state:

"teaching and learning in higher education are unlikely to be improved simply by the application of new technology…the medium itself is not the most important factor in any educational programme - what really matters is how it is creatively exploited and constructively assigned ."

Any extensive use of clicker technology across a law school is reliant upon the belief and commitment of staff members willing to innovate and adapt to benefit fully from the opportunities presented by its use. Burnstein (2001) states:

"The principal barrier to further use and evaluation of keypad merits in our science courses is simply inertia on the part of faculty."

There may be numerous reasons why teaching staff are reluctant to overhaul teaching strategies to embrace technology but any employment of clickers needs to be facilitated by management, not only in the initial decision to make technology available but also in a continuous focus on staff development and technical support provision. Clicker technology presents an unrivalled opportunity to develop an engaged community of conceptually-focused, problem-solving legal learners but this can only be achieved through the engagement and commitment of managers, lecturers and ultimately the students themselves.

This research has been undertaken with the aid of funding from MMU's Fellowship in Academic Practice scheme. The author wishes to thank Dennis Pass of MMU's Centre for Learning and Teaching for his instruction and practical support.

REFERENCES

Abrahamson, A. L. (1999) Teaching with Classroom Communication System -- What it involves and why it works. Paper presented at International Workshop, New Trends in Physics Teaching, Puebla, Mexico May http://www.bedu.com/Publications/PueblaFinal2.html [Accessed 03/01/09]

Allen, D., and Tanner, K. (2005) Infusing active learning into the large-enrollment biology class: seven strategies, from the simple to complex. Cell Biology Education 4 pp262-268

Barber, M. and Njus, D. (2007) Clicker evolution: Seeking intelligent design. CBE-Life Sciences Education 6 (1) pp1-8

Beatty, I. (2004). Transforming student learning with classroom communication systems. EDUCAUSE Research Bulletin (3) pp1-13

Beatty, I., Gerace, W., Leonar, W., and Dufresne, R. (2006) Designing effective questions for classroom response system teaching. American Journal of Physics 74 (1) pp31-39

Beil, C., Reisen, C., Zea, M. and Caplan, R. (1999) A longitudinal study of the effects of academic and social integration and commitment on retention NASPA Journal , 37 pp376-85

Bligh, D. (1998) What's the Use of Lectures? Exeter: Intellect

Blodgett, D. (2006) The Effects of Implementing an Interactive Student Response System in a College Algebra Classroom , University of Maine. MSc: 119

Bourner, T., Reynolds, A., Hamed, M and Barnett, R. (1991) Part-time Students and Their Experience of Higher Education Oxford: OUP

Boyd, W. M. (1973) Repeating questions in prose learning. Journal of Educational Psychology 64 pp31-38

Braxton, J., Vesper, N and Hossler, D. (1996) Expectations for college and student persistence Research in Higher Education 36 (5) pp595-612

Brooke, J. (2004) Make your presence felt, stick in that dibber Times Higher Education Supplement 30^th April http://www.timeshighereducation.co.uk/story.asp?storyCode=188399&sectioncode=26 [Accessed 03/01/09]

Burnstein, R. and Lederman, L. (2001) Using Wireless Keypads in Lecture Classes The Physics Teacher, 39 pp8-11. http://www.replysystems.com/pdfs/benefits/24.pdf [Accessed 03/01/09]

Burton, K. (2004) Interactive Powerpoints: Is There Any Point in Giving Power to Students? Murdoch University Electronic Journal of Law 11 (4)

Caldwell, J. (2007) Clickers in the Large Classroom: Current Research and Best-Practice Tips CBE Life Sciences Education 6 (1) pp 9-20

Caron, P and Gely, R. (2004) Taking back the law school classroom: Using technology to foster active student learning Journal of Legal Education, 54

Caron, P. (2005) Teaching with technology in the 21^st century law school classroom The Future of Law Libraries Symposium FL

Conoley, J., Moore, G., Croom, B. and Flowers, J. (2006) A toy or a teaching tool? The use of audience-response systems in the classroom. Techniques The Journal of the Association for Career and Technical Education 81 (7) pp46-49

Cromie, W. (2006) Harvard launches wireless classroom Harvard Gazette http://www.news.harvard.edu/gazette/2006/02.23/05-eclassroom.html [Accessed 03/01/09]

Crouch, C. and Mazur, E. (2001) Peer instruction: Ten years of experience and results. American Journal of Physics Teachers, 69 (9) pp970-977

Davis, S. (2003) Observations in classrooms using a network of handheld devices. Journal of Computer Assisted Learning 19 pp298-307

Draper, S. W. (2002) Evaluating effective use of PRS: results of the evaluation of the use of PRS in Glasgow University http://www.psy.gla.ac.uk/~steve/ilig/papers/eval.pdf [Accessed 03/01/09]

Draper, S. and Brown, M. (2004) Increasing interactivity in lectures using an electronic voting system Journal of Computer Assisted Learning 20 pp81-94

Draper, S., Cargill, J. and Cutts, Q. (2002) Electronically enhanced classroom interaction Australian Journal of Educational Technology 18 (1) pp13-23 http://www.ascilite.org.au/ajet/ajet18/draper.html [Accessed 03/01/09]

Dufresne, R. and Gerace, W. (1996) Classtalk: A Classroom Communication System for Active Learning. Journal of Computing in Higher Education 7 pp3-47

Duncan, D. (2006) Clickers: A new technology with exceptional promise. Astronomy Education Review 5 (1) pp70-88.

Duncan, D. (2005) Clickers in the Classroom. Addison: San Francisco, CA

Elliot, C. (2003) Using a Personal Response System in Economics Teaching International Review of Economics Education 1 (1) pp80-86

El-Rady, J. (2006) To click or not to click: That's the question. Innovate, 2 (4)

http://www.innovateonline.info/index.php?view=article&id=171 [Accessed 03/01/09]

Ewing (2007) Increasing classroom engagement through the use of technology Maricopa Institute of Learning

Fies, C. and Marshall, J. (2006) Classroom response systems: A review of the literature Journal of Science Education and Technology, 15 (1) pp101-109

Frase, L., Patrick, E., and Schumer, H. (1970) Effect of question position and frequency upon learning from text under different levels of incentive. Journal Educational Psychology 61 pp52-56.

Gall, M. (1984) Synthesis of research on teachers' questioning. Educational Leadership 42 (3) pp40-47.

Gibbs (1981) Twenty terrible reasons for lecturing, SCED Occasional Paper No. 8, Birmingham

Goffman, E. (1981) The lecture In: E. Goffman (ed.), Forms of talk Philadelphia: University of Pennsylvania Press pp162-195

Gump, S. (2006) Guess who's (not) coming to class: students attitudes as indicators of attendance Educational Studies 32 (1) pp39-46.

Guthrie, J. (1971) Feedback and sentence learning. Journal of Verbal Learning and Verbal Behavior 10 pp23-28.

Hake, R. (1998) Interactive Engagement Versus Traditional Methods: a Six-Thousand Student Survey of Mechanics Test Data for Introductory Physics Courses. American Journal of Physics 66 (1) pp64-74.

Herr, R. (1994) Computer Assisted Communication within the Classroom: Interactive Lecturing. Newark: Delaware University

d'Inverno, R., Davis, H., and White, S. (2003) Using a personal response system for promoting student interaction. Teaching Mathematics and its Applications 22 (4) pp163-169

Johnson, D., Johnson, R. and Smith, K. (1998) Active learning: Cooperation in the college classroom Edina, MN: Interaction Book Co.

Judson, E. and Sawada, D. (2002) Learning from Past and Present: Electronic

Response Systems in College Lecture Halls Journal of Computers in Mathematics and Science Teaching 21 (2) pp167-81

Kirkwood, A. and Price, L. (2005) Learners and learning in the twenty first century: What do we know about students' attitudes towards and experiences of information and communication technologies that will help us design courses? Studies in Higher Education, 30 (3) pp257-274

Knight, J. and Wood, W. (2005) Teaching more by lecturing less Cell Biology Education 4 pp298-310.

Kulik, J. and Kulik, C. (1988) Timing of feedback and verbal learning. Review of Educational Research, 58 pp79-97

Lazar, J. and Preece, J. (1999) Designing and Implementing Web-based Surveys The Journal of Computer Information Systems 39 (4) pp63-67

Law Discipline Network (1998) General Transferable Skills http://www.ukcle.ac.uk/resources/ldn/easton_3.html [Accessed 03/01/09]

Lloyd, D. (1968) A concept of improvement of learning response in the taught lesson Visual Education October, pp23-5

Longhurst, R. (1999) Why aren't they here? Student absenteeism in a further education college Journal of Further and Higher Education 23 (1) pp61-80.

Lowery, R. (2005) Teaching and learning with interactive student response systems: A comparison of commercial products in the higher-education market. Paper presented at the Annual Meeting of the Southwestern Social Science Association, New Orleans, LA

MacManaway, L. (1970) Teaching methods in higher education-innovation and research. Universities Quarterly 24 (3) pp321-329

Mazur, E. (1997) Peer instruction: A user's manual. Upper Saddle River, NJ: Prentice Hall.

McLeish, J. (1968) The Lecture Method, Cambridge Monographs in Teaching Methods, no. 1, Cambridge: Cambridge Institute of Education.

Meltzer D. and Manivannan K. (1996) Promoting interactivity in physics lecture classes The Physics Teacher 34, pp72-76.

National Research Council. (1999) How people learn: Brain, mind, experience and school Washington, DC: National Academy Press.

Nicholl, H. and Timmins, F. (2005) Programme-related stressors among part-time undergraduate nursing students Journal of Advanced Nursing 50 (1) pp93-100

Northcott, J. (2001) Towards an ethnography of the MBA classroom: A consideration of the role of interactive lecturing styles within the context of one MBA programme. English for Specific Purposes 20 pp15-37

Oppenheim, A. (2006) Questionnaire Design, Interviewing and Attitude Measurement (11th ed) London: Continuum

Oppenheimer, T. (2003) The flickering mind: The false promise of technology in the classroom, and how learning can be saved. New York: Random House.

Patton, M. (1990) Qualitative Evaluation and Research Methods. Newbury Park, CA: Sage Publications

Quinn, F.M. (2000) Principles and Practice of Nurse Education, (4th ed) London: Stanley Thornes

Roschelle, J., Penuel, W., and Abrahamson, L. (2004) Classroom Response and Communication Systems: Research Review and Theory. Presented at the Annual Meeting of the American Educational Research Association, San Diego, CA

Sabine, N. (2005) The Use of a Student Response System to Improve and Assess Student Learning in Biology Classes http://149.165.1.133/administration/it/documents/technologyaward/sabine-proposal.pdf [Accessed 03/01/09]

Sharma, M., Mendez, A. and O'Byrne, J. (2005) The relationship between attendance in student-centred physics tutorials and performance in university examinations International Journal of Science Education 27 (11) pp1375-1389

Silliman, S., Abbott, K., Clark, G. and McWilliams, L. (2004). Observations on benefits/limitations of an audience response system. Proceedings of the 2004 American Society for Engineering Education Annual Conference & Exposition

Simpson, V. and Oliver, M. (2006) Using electronic voting systems in lectures. http://www.ucl.ac.uk/learningtechnology/assessment/ElectronicVotingSystems.pdf [Accessed 03/01/09]

Simpson, V. and Oliver, M. (2007) Electronic voting systems for lectures then and now: A comparison of research and practice. Australasian Journal of Educational Technology, 23 (2) pp187-208

Timmermann. G. (2003) We Teach as We Are Taught? The Impact of Personal and Professional (Teaching) Experiences on Teacher Educators' Conceptions of Teaching Journal of Interdisciplinary Social Science 3 (5) pp173-178

Tinto, V. (1993) Leaving College: Rethinking the causes and cures of student attrition (2nd ed) Chicago, University of Chicago Press

Tinto, V. (1996) Reconstructing the first year at college Planning for Higher Education , 25 (1) pp.1-6

Tinto, V. (2006) Taking student retention seriously Keynote presentation and paper at Maricopa Community College District, 6 January http://www.mcli.dist.maricopa.edu/fsd/c2006/docs/takingretentionseriously.pdf [Accessed 03/01/09]

Trees, A. and Jackson, M. (2003) The Learning Environment in Clicker Classrooms: Student Processes of Learning and Involvement in Large Courses Using Student Response Systems, Communications Department, University of Colorado.

Ward, C., Reeves, J. and Heath, B. (2003) Encouraging Active Student Participation in Chemistry Classes with a Web-based, Instant Feedback, Student Response System. Presented at CONFCHEM: Conferences on Chemistry, Spring 2003

Wood, W. (2004) Clickers: A Teaching Gimmick That Works. Developmental Cell 7 (6) pp796 -798

Woods, H. and Chiu C. (2003) Wireless Response Technology in College Classrooms. Michigan University http://www.mhhe.com/cps/docs/CPSWP_WoodsChiu.pdf [Accessed 03/01/09]

Yorke, M. and Longden, B. (2004) Retention and Student Success in Higher Education , Maidenhead, UK, Society for Research into Higher Education and Open University Press.

Zhu, E. (2006) What do students appreciate most about clickers? Center for Research on Learning and Teaching The University of Michigan

Zull, J. (2002). The Art of Changing the Brain. Stylus Publishing, LLC: Sterling, Virginia.