Welcome to the second module of the programme.

Before starting the second stage reflect on what we have covered so far. In stage one you considered the nature of action research; data collection tools and reflected on problems of professional practice. You will have looked at consistency and contrast within descriptions of action research. One consistency is in seeing action research as geared to a cycle of design, implementation and evaluation of some kind that is practitioner led and often collaborative. One contrast is between those that see action research as leading to a radical agenda for understanding and articulating the constraints on the education system while others stress it as a method for curriculum implementation. Although descriptions of action research are very similar some tend to emphasise the cycles within cycles and some suggest a more sequential and perhaps over-simplified approach.

You will have gone on to considered your awareness of a problem and how this may have presented itself to you. In developing your inquiry you will have (1) identified a problem, (2) carried out some reconnaissance of the problem and (3) focused your inquiry

In this part of the course we will go on to consider your next steps in your enquiry:

• (4) planning for evaluation
• (5) implementation
• (6) evaluation

A common starting point for many action research projects is to ask if a course of action is better than another.

For example: "Is the class with which I am going to use group work going to achieve better results than the class that does not?"

While a comparison between two or more classes can provide rich sources of data be cautious in overstating their importance. A comparative study, particularly on the scale you are planning, cannot give you a valid indication of whether one course of action is 'better' than the other.

In education

• ...you cannot control for variables sufficiently.
• ...you cannot replay your innovation.
• ...your own involvement in the study is going to influence how pupils respond to change.
• ...you are unlikely to find simple observable outcomes (certainly in terms of attainment).

You need to approach your evaluation of your innovation against the impact of the problem which you were seeking to address. For example, if an innovation is setting out to address class management difficulties you will probably need to make direct observations of pupils' behaviour rather than, for example, draw conclusions from secondary outcomes such as test scores. You need to frame your approach to data collection as 'fit for purpose'. To help you do this we have looked at different tools which you might want to use.

For you to talk about Consider one of these scenarios in which an innovation concerns:

Using an electronic whiteboard to increase levels of participation in whole class teaching. Using peer assessment to help pupils understand the nature of National Curriculum 'levelling'. Developing a performing arts club to increase levels of attendance among disaffected pupils. What kind of evaluation tools would best help you assess the impact of your innovations.

Reflection on the activity All these scenarios raise questions about the context of the innovation, for example is the whiteboard going to be used by someone who is comfortable with using computers , is it being accompanied by a change in question and answer strategies; ...who exactly is running the performing arts club and what support are they getting? ...is there an existing culture of peer assessment in the school? Key tools in the respective innovations (but not the sole methods to use) are: (Performing Arts) Data on attendance - however this would have to be supplemented with interviews with pupils to explore whether any changes might the result of intervening variables which so often affect patterns of engagement and attendance. (Peer Assessment) recordings and notes to pupils work would be very helpful here but evidence of understanding usually found in talking with pupils. (Electronic Whiteboard) simple observation of lessons should shows for example if more pupils are offering answers to questions and how often pupils volunteer to write on the board but again interviews will help provide a fuller picture.

Tools for Data Collection

The previous exercise suggests that there are several tools you might consider using for data collection, some of the most widely used are:

• Interviews and focus groups
• Surveys
• Documentary evidence
• Pupils' work and artefacts
• Observations
• Field Notes and Diaries

Each has it owns strengths and weaknesses, the value of tool lies in its 'fitness for purpose'. For example interviews are not in themselves better than surveys they are more appropriate to gather in depth data in contrast survey can provide an overview of a range of opinions relatively quickly and cheaply.

Top of page

Interviews and focus groups

There are various styles of interview. Structured interviews consist of uniform questions covered in a predetermined sequence in the manner of a script. This has the advantage of making for easier analysis of results, particularly when a large number of interviews are to be conducted or when different interviewers will be conducting them. However structured interviews constrain the discoveries and insights that can be made in a more open format. Semi-structured interviews consist of predetermined topics to be covered but the form and sequence of the questions may vary. The disadvantage is that analysis is more challenging particularly of a large number if interviews are to be conducted.

In an unstructured interview the interviewer seeks to 'break the ice' and allow the interviewee to speak with minimal cues of intervention. The advantages are that there is opportunity for a more even power balance and unanticipated insights and discoveries may be made. However there is the potential pitfall of 'awkward silences' and forced answers, particularly if the interviewer lacks confidence. The analysis can be very complex, particularly when trying to compare two or more interviews.

Focus groups can also be considered as a style of interview. These tend to fall into the category of semi-structured or unstructured interviews in which the aim to is initiate and sustain group rapport. This can also have practical benefits such as reducing the number of interviews undertaken but there are drawbacks such as the difficulties of transcribing a recording in which several voices are talking - not necessarily in turn!

No method of data collection excludes others, for example your purpose for interviewing may be to confirm and check data collected through surveys, observations and so on. In particular it can help you to explore other data in a deeper way, for example by identifying key informants and gatekeepers. In addition it may help you to build rapport with people prior to an observation and give you access to 'insider voices'. One of the main distinctions between interviews and other methods is that they can capture ideas as they evolve in dialogue, providing a dynamic vision, rather than static snapshot, of a situation.

Interviews are typically conducted face to face and attention must be paid to establishing a relationship, use of body language as well as practical matters such as capturing the conversation using audio or video devices, or field notes. With modern technology other methods are possible such as by telephone. Although this is a restricted medium, for example body language is lost and natural pauses constrained, there are some benefits such as recording devices and note-taking becomes less intrusive. Computer technology offers opportunities for video-conferencing or live text-chat, the latter offering the advantage that no transciption needs to be created. Asynchronous methods such as e-mail correspondence, instant messaging, and even mobile phone texting (note that some interviewees may find texting laborious or be uncomfortable with the technology). Asynchronous methods allow interviewees more time, allowing frequent returns, may be less inhibiting and require no transcription.

For you to do Many examples of interview proformas can be found on the internet. For example... ...here is a sample of typical interview questions from the Cooperative Educational Service Agency in Wisconsin ...here are some interview questions used in the 'Digital Beginnings' project funded by BBC Worldwide and the Esm?e Fairbairn Foundation What are the strengths and weaknesses of proformas of this sort? For each question you might want to consider...

Does it unnecessarily indicate the interviewer's attitudes? Does it contain cultural assumptions? Is the vocabulary appropriate? Is it syntactically and grammatically clear? Does it need follow-up questions in order to elicit a satisfactory answer? If so what might these be? Is it sufficiently specific? Is it sufficiently open?

Specific issues are raised when interviewing the pupils you teach. They may find it difficult to see past your role as teacher and aim to provide the answers they think you want to hear. Similarly your may find it difficult to remember you are wearing a 'researcher hat' and must avoid leading questions. As well as these power balance issues there are others such as children's ability to articulate their answers. Ethical considerations need to be considered particularly if you are tackling a topic which is sensitive for your school.

Top of page

Surveys

Surveys and questionnaires offer the researcher a time-efficient method for collecting large amounts of data with the possibility of a high return rate. Standardised questions lend themselves to easy analysis and direct comparison from one response to another. Research subjects are able to provide data anonymously meaning it can be a less intrusive form investigation and may lead to less inhibited or defensive responses than other methods. A possible limitation is that the information provided in responses tends to be more of a superficial and descriptive than a meaningful and explanatory nature.

While surveys and questionnaires offer a time-efficient method to capture and analyse large amounts of data they must be carefully designed, drafted and proofed. This can be time-consuming but imagine sending out 500 copies of a form only to discover they contain a fundamental flaw! Issues of validity and reliability come into play when considering your sample size how representative your respondents are. In a small scale project it might be quite possible to survey, say, all the members of a class who participated in the project.

Careful consideration must be given to the types of question you will use. Open-ended questions offer the respondent the freedom to answer as she wishes and allows for potentially more enlightening data and unanticipated issues to emerge. However it restricts cross-comparison and makes for more difficult and lengthy analysis. Closed-ended questions limit responses often to 'yes-no' or multiple choice responses. Scaled-questions offer a continuum such as the familiar five-point Likert scale of "strongly agree | agree | neutral | disagree | strongly disagree". Rank order questions ask for a list of provided responses to be put in rank order. Non-open-ended type questions restrict respondents but are easier for the researcher to process.

The structure of the questionnaire is important. For example, you may wish to include contingent questions which are relevant only to certain respondents such as those who answered 'yes' to the previous question. A question matrix may be appropriate where a group of questions have the same answer structure, as shown below

Questions should be ordered to give a sense of flow with more taxing questions often placed towards the end. Leading and suggestive questions should be avoided. Validity and reliability come into play once more. Matrix questions may be organised so that respondents have to think carefully about each response - it all to easy to get into a flow of ticking all the way down a single column if all the answers are fitting a general pattern. For example the following matrix...

...might be better laid out as...

For you to do There follows some documents on-line which contain surveys and questionnaires... ...here is a sample survey questionnaire from the Internet Based Masters Degree in Educational Technology (iMet) web site of California State University, Sacramento ...here are couple of published action research projects by students at the Southern Oregon University action research website which contain questionnaires 1 2 ...the Becta report Enabling teachers to make successful use of ICT contains the questionnaire used to gather data ...a report on proceedings from UniServe Science at The University of Sydney includes a sample survey on page 11 What are the strengths and weaknesses of surveys and questionnaires of this sort? For each you might want to consider...

What types of questions have been used and why? Does the survey have a good structure and flow? Is the vocabulary appropriate and unambiguous? Are any questions leading or suggestive? How was the survey administered and how was the sample selected? Any other notable aspects...

New technology enables the possibility of online surveys which have their own set of advantages and drawbacks. You do not need to distribute an online survey but simply provide the web address (url). Data can be collated automatically and even analysed semi-automatically - for example by generating graphs and statistics. It can be cheaper - there is no need for printing and photocopying. Potential drawbacks are that you need the technical skills and even then the survey can be time-consuming to set up. There are dangers that the website may go down at a crucial time or that some participants do not have access to the internet, others may be uncomfortable using computers. Response rates may be lower or slower if people put off or forget to visit your survey - a hardcopy can be a better prompt to get it done.

Top of page

Documentary evidence

Documentary evidence includes historical documents (newspaper reports, television programmes and so on) but also contemporary documents. Within a school setting, this includes registers, minutes of meetings, schemes of work, school development plans, lesson plans and so on.

Documentary evidence is often easily available and free and using it is often a non-intrusive process. The dangers are that it is not a rich source of data - for example the minutes of a meeting do not capture the mood or other subtleties and the meeting may be quite selectively reported. Although factual in essence documents cannot be trusted to be entirely objective and, at worst, may even be deliberately misleading. In the age of targets and league tables they may seek to present the sense of quiet optimism expected by regulating agencies. It is very difficult to validate documentary evidence.

You likely have access to documents in your place of work as well as to historic documents through libraries, museums and institutions such as The National Archives. Ofsted reports are publically available at online at Ofsted's Reports Homepage; example schemes of work can be found at The Standards Site of the Department for Education and Skills; example lesson plans can be found at the Useful Lesson Plans website of TeacherNet; School Development Plans, also known as School Improvement Plans, can be found online using a Google search.

Top of page

Pupils' work and artefacts

Another source of data, similar in some ways to documentary evidence, is pupils' work and artefacts. As well as written work this can include the products of art, design and computer projects and even recordings of performances in music and drama. Again this type of data is likely to be a supplement to other forms rather than a central part of your evidence. As a teacher you are no doubt very familiar with this type of evidence and will be able to use your intuition to help make sense of it. For example, although a piece of work might be dismissed by the lay observer, we can often see the sense the child was making when she produced it.

A large source of pupils' work can be found online at the National Curriculum in Action website. Bear in mind that while the site comprises real work examples by real pupils, it was produced with particular goals in mind. However the contents of the website can help you to think about what pupils' can tell you.

Top of page

Observations

Classroom and other observations can provide rich source of data. As an initial activity observation can be valuable in helping you to define and refine the focus and research questions of your study. If you are carrying out a structured observation of your class you may seewhat is happening in a new light. If you are observing a new class you can quickly become immersed in the culture and context of the setting and to become familiarised with its 'language'. Observation can also provide an extra, richer source of first hand data to triangulate other sources. As with all social science research ethical issues come to the fore when you involve yourself in field observations including consideration to confidentiality and anonymity. Within school settings child protection aspects are vital too.

The notion of the passive, objective "fly-on-the-wall" is just one role the researcher can take. You can take on the role of observer-as-participant or participant-as-observer. Within classroom-based research this can mean being a student, participating in activity; teaching the class (being in charge) or assisting the teacher. You will need to consider how you will introduce yourself to staff or children; where you will position yourself within a lesson; how you will move around and so on. People may be somewhat intimidated if they know you are a teacher wearing a researcher's hat so you may wish to come across as an "acceptable incompetent" to reduce this reaction.

Methodology is important, especially the issue of data collection. You will need to consider how appropriate it is to use field notes, audio recordings, video recordings and still photography. Note the obvious ethical considerations of audio-visual data: you will most certainly need to establish the permission of staff, children and parents. You choice will have an impact on how those you are observing react - a video camera can be quite off-putting in a classroom. It will also have bearing on the richness of the data collected and your method of analysis.

An online example of written up field notes and accompanying photographs from research undertaken at a school in Belize can be found here. Do you think the photographs add anything to the webpage that would not be present if they were absent?

A sample of hand-written filed notes can be found on page 97 of the following dissertation examining computer technology integration. A sample interview with analysis can also be found on page 100.

The following sample movie file of a teaching session, and corresponding transcript which has been prepared using the freeware Transana, gives a sense of the analysis process. (60MB)

Field notes are likely to form a significant part of the data you collect over the duration of your research. Field notes may be taken in situ, particular during observations and unstructured or semi-structured interviews.

Taking field notes has advantages and disadvantages compared with audio and video recordings. They can be less intrusive than a camera or microphone and have the practical benefit when children are your subjects as they lack the legal complications of capturing children on film. Notes allow you to include contextual information and interpretations that cannot be captured by recording devices. In addition not all scenarios are technically viable to record electronically. One possible usage of in situ field notes is a back-up or triangulation for electronic media. A disadvantage is that taking notes can be distracting to the researcher which can lead to omissions. Notes are a far less efficient form of capturing information than recordings and, in addition, unintentionally subjectivity may creep in observations and interviews.

Notes may also be written after the event being described and in this case tend to be referred to as diary entries. Diary entries tend to be of a more reflexive nature where the focus is less on objective-type capture of information than on interpretative recollections of events.