|
Chapter 4 Data Collection Methods
|
|
Introduction
Aims and objectives The aim of this chapter is to outline:
The first objective of this chapter is to create a linkage between the theories, and the progression of the theories of research in the previous chapter with the analysis of the data that is analysed in the following sections of this thesis. The second objective of this chapter is to create a framework of data collection that can be replicated in future research in this field.
The structure of this chapter This chapter continues with the following segments: · The choice of methods used to collect the data · Outline of methods: Interviews · Outline of methods: Participant Observations · Outline of methods: Diary Reports · Outline of methods: Literature Searches · Ethics
The choice of methods
"Experience is the name everyone gives to their mistakes." Oscar Wilde (2001: p.122)
Influences from other authors From previous experience, I decided to use qualitative methodology in both strands of my diachronic and synchronic analyses. Although I trialled both qualitative and quantitative strategies in my previous research, I found quantitative studies particularly unhelpful when investigating concepts that are difficult to classify. Consequently, I felt that it would be paradoxical to use such strategies.
In addition, I felt that qualitative strategies were appropriate as the theories on disability I was testing were influenced by qualitative social theories of shared experiences in a post-positivist framework. These elements led to a more creative interpretation of social observations. I felt it was easier, therefore, to criticise these theories on similar grounds. Furthermore, semiotic frameworks are traditionally applied to qualitative studies.
In order to find precedence for such arguments, I turned to C. Griffin (1985). She argues that qualitative models lead to more tightly observed forms of research, which draw out new forms of understanding. Qualitative research, she argues, is also more often associated with small scale research studies, such as this one. Furthermore, the data collection methods used in small scale qualitative studies are compatible with testing the findings I made in my MEd study, as they too were qualitative.
The choice of complimentary methods There were also two further considerations I made when choosing more specific qualitative methods. These were less to do with creativity and more to do with practicality. The first consideration is that in the synchronic study there were only a small number of people studying art education in schools for the blind.
The small number available for study meant that if I had used the narrow model of a single technique, the scale of my research would have been too small to produce meaningful results. In addition, as a part-time researcher the methodology had to be flexible enough to allow me to study in my own time. I could not choose techniques which took time pursuing, for instance, large scale, data intensive surveys and computational analyses.
My second reason for constructing this framework was the use of hypotheses. K. Punch (1998) and C. Robson (2002) argue that this approach restricts the choices of methodology to highly structured research tools. This is an empiricist’s argument, based on the view that data collection needs to be verifiable within and across studies. As a result, these authors argued that structured and prescriptive research allows others to reproduce all or part of the study later, so that it can be verified. I was familiar with this notion from previous scientific research, and so felt comfortable with its concept.
As a consequence, I decided to merge what seemed to be the complimentary qualitative models in the diachronic and synchronic strands of the research, which gave me a broader spread of the research literature as a guide. Literature in this area ranged from conventional educational ethnography to more conventional and historically minded anthropology. Although this created a different style of research paradigm, it felt necessary as other studies which tested existing areas of research were experimentally based and thus restrictive.
Although this new paradigmatic model is unconventional, Y.S Denzin & N.K Lincoln (1994) have set a precedent for just such an approach in their Handbook of Qualitative Research. In their analysis they criticise the direction of the methodological and theoretical paradigms of current ethnographic research. For instance, they argue that researchers have given up their attempts to create movements of ethnographic theory or methodology, and have stuck to re-researching subjects using re-produced methods.
Lincoln & Denzin continue that this approach by ethnographers led to political models of social scientific research, which focus too often on the single issue of interactions. In doing so they feel ethnographers ignore creative and pioneering descriptions of complex cultural behaviour as it evolves over time. The movement towards my diachronic and synchronic strands, therefore, led away from what Lincoln & Denzin argue is social science’s current obsession with the restrictive “historical moment”; i.e. a restricted, instantly observable period in the time of the social scientific study, in a particular place.
The design of the ethnographic and historical methods I eventually decided to use the tighter, more structured forms of data gathering I had used during previous ethnographic and historical studies. In my new model of analysis, however, all of this data was to be regarded as historical, and my techniques were tightened by the triangulation of the data collection methods, research subjects and the experiences of different cultures. This technique is described by V.J. Janesick (1998) in her points 1 (subjects and cultures) and 4 (methods): 1. data triangulation: the use of a variety of data sources in a study 2. investigator triangulation: the use of several different researchers or evaluators 3. theory triangulation: the use of multiple perspectives to interpret a single set of data 4. methodological triangulation: the use of multiple methods to study a single problem… Triangulation is meant to be a heuristic tool for the researcher. Janesick (1998: p.46-47)
As I began to design my research tools, I formed a wish list of what I wanted to achieve from each of the two analyses. My first decision was to define what I wanted to achieve from the synchronic and diachronic strands, which had to fit my original research questions and hypotheses. I then developed a list of tools for collecting evidence that took account of the topics of my research: the people and the institutions I wanted to study. This was annotated as follows:
Synchronic Study Definition: To discover how art is practiced in the classroom and how it is affected by historical factors. The following issues were related to these topics:
Diachronic Study Definition: To discover how the epistemology of the art education of students who are blind has developed through policy and belief, whether mythological or through advanced scientific discovery. Thus I planned to discover:
From this wish list, I chose the following methods, based on those I had used in my previous study. This allowed a comparison between the evidence of both studies: · Interviews (both the synchronic and diachronic study) · Participant Observations (the synchronic study) · Diary Reports (the synchronic study) · Literature Searches (the diachronic study)
I now describe how the data collection methods were implemented by describing:
Because I found there were overlaps in these periods of research, the institutions and the subjects of both strands of data collection, I will describe these different periods of data collection in chronological order. In addition, I will also add a description of the ethical considerations that I applied.
An outline of the methods: Observation
The theory of observation The observations I conducted for this study were based on participant observation; i.e. a series of observations conducted by actively communicating with the subjects of the research, as well as watching their actions (Lincoln & Denzin, 1994). This method also involves becoming immersed in their community in order to understand how they worked. In this study, the output of the research was not reflexive[1], as is usually associated with participant observation, and described thus by David Cohen & Louis Manion (1994): The purpose of such observation is to probe deeply and to analyse intensively the multifarious phenomena that constitute the life cycle of the unit with a view to establishing generalisations about the wider population to which that unit belongs.” Cohen & Manion (1994: p.106).
I used the techniques of participant observation to gather data within my synchronic study. This meant I had a further wish list of factors to observe through observations, within the context of the classroom and the schools’ environments and cultures. This is not to say, however, that I was using it as a positivist testing strategy. That would itself seem to contradict the framework described in the previous chapter, as well as, it was felt, yielding unproductive data.
Instead I gathered my data looking only, as Cohen and Manion aver, for patterns that appeared to exist within students’ case studies, in order to examine the extent of the validity of Doyle’s existing behaviour model as a valid method of interpretation, and to examine elements of its falsifiability (Popper, 1978).
In the participant observation, I also drew on the experiences of my previous methods (Hayhoe, 1995, 1995b, 2000), which were formed by creating relationships with the research subjects and then note-taking informal and formal conversations. This note-taking employed a mixture of ad hoc recording with tape cassette recorders, and writing pen and paper notes. In addition, and also similar to my previous research, I planned to observe any differences in the patterns of behaviour between the students. I then planned to compare these experiences to their past histories.
There is a precedent for this approach. G.D. Berreman (1968) argues that participant observation is not simply restricted to a small area of an institution’s or person’s life. It is also not only restricted to observations of their behaviour. Therefore, although realising that it was impractical to be involved in all aspects of my subjects’ lives, I tried not to restrict my observations to the art classroom alone.
During my visits to the schools, for instance, I also managed to observe students and staff informally around the whole site. Furthermore, as a result of moving to the town where the main school in the study was based, I also managed to observe how students behaved and were accepted around the town.
There was a problem with this. Traditionally, observation is associated with sociological and anthropological frameworks, so I had to consider how it was going to be used to interpret an epistemological concept. Eventually, I felt that this approach was advantageous, however, as it allowed examination and non-experimental data. It also allowed an investigation of the subjective experiences of students and teachers. This data was eventually analysed alongside their interviews and reports in order to provide a weight of evidence.
This chapter continues by examining the identification of useful administrators who could help identify further students, teachers and areas of research.
The use of participant informers During the observations, I also decided to use participant informers. These are defined by Martin Hammersley (1984) as go-betweens or intermediaries providing a point of contact for the researcher with the culture or institution that they are studying. Hammersley’s own use of informers is described in his study of a Manchester grammar school, in which he describes how he used an important member of staff, in a position of management, in the school to pick his most helpful subjects. - These people he referred to, in common with other ethnographic researchers, as “actors.” I do not intend to use this term in his current description though. -
Hammersley also described how he used his informer to provide a trusted face around school, because he was a researcher with relatively unknown aims. He also had no previous connection with the institution. This meant that he did not have the necessary kudos with the other students that an established teacher already working in the school could provide.
This segment continues with an analysis of the data recording methods.
Data recording methods During my fieldwork, I experimented with specific data collection techniques. The traditional data collection methods I began with were used in studies by for instance Robert Burgess (1987) and Hammersley (1984), and were based on pen and paper participant notes. In addition, I also photographed the students’ work, both in their presence and absence.
Whilst taking photographs, I used a model designed specifically for ethnography by J. Collier (1967). His model specifies that the subjects of photographs should reflect not just the focus of research but their artefacts (by which he meant tools), their positioning in relation to each other and a cross-method comparison with other data[2]. An outline of the methods: Open Interviews & Participant Diaries
An interview ordinarily refers to a face-to-face exchange of information between two people, but their relationship can range from an interviewer’s active interrogation of a passive respondent to an interviewer’s passive attentiveness to an active respondent. Abrahamson (1983: p. 332)
The theory of interviewing Of the main proponents of the interview, J. O’Connell Davidson & D. Layder (1994) present what they see as three main interview approaches: orthodox, qualitative and feminist. In my study, I decided to use the qualitative approach, as it appeared to fit most appropriately with the overall objectives of my methods. In addition, its insistence on verifiability also made it strict enough to be included in my armoury of methods.
I felt qualitative interviews also encouraged open questioning, allowing the research subjects to elaborate on the reasons for their behaviour and their life histories. From previous interviews, I found this led to more relaxed discussions and honest answers. Its exploratory rather than fixed position on definitions of knowledge construction also made it an effective vehicle to explore epistemological meanings. Qualitative research is generally not so much concerned with obtaining accurate replies to closed ended questions, as with obtaining full and sincere responses to relatively open-ended enquiries. O’Connell Davidson & D. Layder (1994: p.121)
The eventual face-to-face interviews I conducted during fieldwork were akin to structured conversations rather than a strict set of closed questions. These structured conversations were designed so each question became a prompt. Burgess (1987) discusses this style of interviewing in a research project, during which he conducted interviews with both staff and students in a British secondary school. He found them more positive, less stilted and more likely, he argued, to sustain honest answers than closed questioning.
The theory of over-familiarity is now discussed in the segment below.
“Going native” in interviews In my interviews I was also aware of the dangers of becoming too familiar with my subjects and the institutions themselves. I felt this was particularly dangerous because, during my previous research, I had built up good relationships with students during informal visits to the schools for the blind and my findings had become subjective. Although the data was to be led by the subjects’ opinions, I had to make every conscious effort to stick to my original research aims as closely as possible to negate this problem.
There were also ethical implications of my familiarity with the subjects. I had to find a way of not being swayed by their opinions because I liked them as people. It was not my intention to “go native”. Such a move could also have led to the subjects worrying about what they had said after the interview.
The problem of such over familiarity in interviews is discussed by several authors, both for its practical and ethical problems (Ram, 1996; Atkinson & Hammersley, 1994; de Laine, 2000; Stenhouse, 1984). Lawrence Stenhouse in particular found this problem dangerous in research which investigated a school’s library access. However, although it caused problems of censorship from the head of the school he found difficult to rectify, his diligence did yield a desirable side-effect. This is presented in his description below: I thought of the classic advice to field-workers’: never accept hospitality and always take your own marmalade.’ In this case there was little or nothing crossed out which bore on my interests, but what I did was to go back and explain why I thought that the head had had to cross out so much and then I interviewed her again and now I have two interviews, one hacked and one complete. Stenhouse (1984: p.222)
In accordance with the advice of these researchers, in this section of research I attempted to maintain a strong distance from the interviewees, although this became a problem. This is discussed further in the following chapter, which examines my implementation. What now follows is a discussion on the nature of the participant diaries.
The theory involved in the creation of participant diaries The purpose of the participant diaries was to maintain the students involved in the participant observations personally involved in the structure of the research by keeping a record of their work in art classes. The method involved was similar to the interviews, although there were slight differences: the emphasis on completing the reports was the responsibility of the subjects and the subjects were also given loose topics rather than structured questions. More particularly, I chose a method of taped reports based on a diary keeping technique of contemporary anthropology (Nachmias & Nachmias, 1981).
These diaries were personal, qualitative records of everyday tasks, and relied on research subjects’ opinions. These were opposed to quantitative log-style diaries found in many other longitudinal studies, which focused on dates and comparisons (Lincoln & Denzin, 1994). The data I eventually collected from subjects’ diaries was only a record of their educational experiences during the fieldwork, collected over one and a half years. I also anticipated that their data would be enough to compile a comparison of the students’ histories. These were designed to show the influences of their backgrounds on their present work (Nachmias & Nachmias, 1981).
This segment continues with a discussion of the context of the diary reports. The contextual problems of the diaries Before beginning the process of collecting diary reports and, to a certain extent, the interviews for the diachronic strand of the research as well, I was also wary of the different contextual qualities the data would contain. I decided that an overly literal interpretation of these diaries and interviews could be seen as dangerous when applied to the narrative, therefore I applied a focus.
The focus of the diaries was everyday tasks in art classes. These relied on the subject's creative development, as opposed to a quantitative log-style diary. Unlike previous large studies over long periods of time, however, the data from my subjects’ diaries were only ever intended to be a record of their recent educational experiences. They were going to be recorded over a period of a week at a time, and then, after I found that this was too often for the students, over a period of a month at a time.
All of the narratives provided by these diaries and reports, to a greater extent, did have an element of the history of both the subjects’ life cycles in the end; this included political and environmental influences. In my previous research and in the literature, however, I found that this technique had pitfalls. Elisabeth Tonkin (1995), for example, warned of such simplistic, non-contextual interpretations. She felt that oral histories are socially constructed in a subjective manner, and, for instance, that social anthropologists cannot see any oral account in cultural isolation. Instead, she suggests that each narrative needs to be compared to their culture, and that of any audiences that hear them.
This segment now continues with a discussion of the forms of data recording.
Recording media of the interviews and diaries My final consideration was the media used to record the interviews and diaries. In previous studies, authors suggested that it is wise to use a combination of tape recorders and pen and paper notes. Whilst it is suggested that tape recorders are used as the main source of data recording, pen and paper notes are used as back up in case the tape recorder does not work.
In some cases, in fact, I felt that it was wise to ignore using the tape recorder all together. This was initially so that I could more easily transcribe any notes I would eventually analyse and use and because previous researchers have suggested that participants are sometimes reluctant to be recorded using such formal methods (Pelto & Pelto, 1978).
The postal and email interviews obviously presented no problem, as there was only a single source of data. However, I initially decided to record the diaries using a tape cassette and the interview using the recorder and pen and paper notes. I figured my notes would give me a basis for a report, and the tape could be used for quotes.
After experience, however, I later only used pen and paper notes in England and the recorder in the less predictable US. Both media also seemed in accordance with the ethics of allowing the students access to copies of their own data. With both the diaries and the interviews, I decided to offer the subjects a taped or written transcript. I also decided to remind them of their right to withdraw their participation and data. In the end though, none of them withdrew.
I felt my approach to the diaries had the advantage of allowing all of the subjects to set much of their own agendas, which helped to maintain their sense of ownership of the data. It was also useful because there was a lack of work done in this area. Consequently, the only hypotheses I had to refer to were my own, although the subjects helped to construct these. The topics I set the students themselves were broadly based on the general theme of their past and present educational experiences. In addition, I also introduced questions about incidences I took from each subject’s observations.
This chapter now continues with the description of the literature search methods.
Outline of methods: Literature searches
The theory of the literature searches Although the literature searches were not the only method of collecting data for the diachronic, historical study, they were to be the most important. They were also originally intended to be the only sources of information in the diachronic study. It was only after I felt I could get access to such a high level of interviewee, however, that I decided to add this method.
My approach to gathering data for this historical study was based initially on the methods that I used in my previous research. However, because the use and analysis of historical data was far more important in this study, I developed a significantly more structured, tighter method of gathering literature. This new method bears a close resemblance to the structured delineation of primary and secondary source data described by Arthur Marwick (2001). The clear and vital distinction between primary and secondary sources is embodied in the rules that all writers of PhD dissertations or of scholarly monographs must obey… the first part… must list all of the archives and other repositories and collections of primary sources used… the second must list [separately] the secondary sources. Marwick (2001: p.158)
There is some controversy about the nature of what is a primary and secondary literature source. Marwick discussed the arguments made by authors such as Jordonova[3], who claims the subjectivity of the production and interpretation of both sources of data made them indivisible. Importantly to this study, in relation to oracy as well as literature, Tonkin (1995) appeared to concur.
However Marwick (2001) himself felt this distinction is necessary, as to him there was a clear delineation. For him, it was how they were read, and not the intrinsic content of these documents that is the important consideration. Therefore in this study it was decided to take Marwick’s model and distinguish primary sources as documents written in the time in which the study is designed to examine, although they are interpretable through the lens of the culture of their era.
Consequently, like Marwick in this study it was felt to be important to begin the literature searches by reading secondary sources in order to distinguish the theories about the era and academic field under study. It was then decided, after creating a frame of reference, to extend this search by applying and examining primary sources of evidence. These were then classified in the bibliography after the main bibliography, and remained unconnected with the secondary source historical literature search.
This chapter now continues with a discussion of the ethical considerations of the implementation of the data collection methods.
Ethics
The choice of the code of ethics In this study ethics were also a major consideration in the choice of qualitative methodology. I particularly wanted to stay away from contentious, experimental laboratory methods. This was important as I was dealing with a subject that appears to currently be at the centre of a great deal of political debate. It was therefore important to include qualitative methods that could be easily explained to my subjects, and allowed their positive participation in as many ways as possible.
There was, however, some contention in the choice of a standard ethical code. In appendix 4, I present a paper I wrote from an interview with Professor Martin Lawn, the then chair of the European Educational Research Association. This suggested that the use of these ethics themselves were culturally subjective. The reality of this situation, though, was that I had to take a cultural standpoint. As a result, I chose the British Educational Research Association’s (BERA) code of ethics, based on their American counterpart’s. These were found on their website, http://www.bera.ac.uk/.
In addition, I referred back to an interview on ethics in research that I conducted in 1994 for my earlier study, with Gordon Dryden, then officer in charge of post-16 education at the RNIB (Hayhoe, 2000b). I felt it was important to include Gordon’s belief’s as the BERA guidelines were not tailored towards blindness and social research, and unfortunately, it appears even now that the only research available on this subject is based on conducting medical research.
This chapter now continues by discussing the importance of consent and gaining the permission of research subjects, which became the central issue of this study’s implementation.
Informed consent and anonymity In the BERA Code, in addition to discussions of the related issues of academic freedom to criticise, the most important issue appeared to be those of informed consent and anonymity. In addition, in Gordon’s interview the most important factor was the recording media. I will now address these issues separately. Firstly, informed consent is now a commonly used practice in social research. It is the process of getting permission from the subject of the research to participate in a project. If they do not consent, they are not included in any form.
In my research I found it good practice to give students a summary of what the project involved, and what I hoped to achieve in the broadest sense – in the literature it was recommended that this should be a “bullet point” list of aims and objectives (Prosser, 2000). In addition, I have previously found it further develops trust if I offered to provide copies of the subject’s notes to him or her if they requested them; I decided to offer the same in this research. I also decided, however, not to make this a formal contract, or a basis for legal consent, as this was beyond the aims of my research. This attitude towards an informal, more trusting approach to ethics is discussed by de Laine (2000).
Even with informed consent, I have found it necessary to make the subjects from schools and colleges anonymous. In some cases anonymity was useless, as institutions for people who are blind are relatively few in number, and I chose some very identifiable members of the community. In this diachronic historical study, I also felt that anonymity defeated the object of the study. Making the important institutions for the blind anonymous would also detract from the issues their policies and curricula raised, and would make the evidence unverifiable.
Moreover, in terms of the interviews with former politicians, it was again important to verify their information in relation to the important positions they held. Furthermore, I felt these politicians had been in a position where their policies, either current or former, were open to public scrutiny, i.e. their importance is of such value as to make scrutiny of their opinions in public necessary. Helen Simons (1989) makes three points relevant to this evaluation, that I felt bore particular relevance to this study: First, the sub-procedures adopted by external researchers of changing the names of places, persons and organisations is simply not possible. Secondly, an important function of self-evaluation is the move towards openness and providing a credible account that can be checked by independent observers. Thirdly, part of the function of.... self-evaluation is to promote intra-professional accountability and this can more appropriately be established where individuals are ‘on the record.’ Simons (1989: p.130-131)
Unlike Simon’s, however, I felt it was unnecessary to name the students and teachers I studied in the synchronic fieldwork. The notes I made from their observations could never be reproduced in exactly the same context. This was not, after all, an experimental situation, neither was it designed to be seen in that context. So Simon’s argument about reproducibility of individual lessons was irrelevant to me. In addition, I meant to aim for a narrative of broad concepts that could be observed across cultures, and so the identification of single people was bound to cause more embarrassment than make a point.
This segment now continues with a discussion of the ethical considerations of using visual recording media with students who were legally blind.
The ethics of the data recording media The final issue is that of data recording media. In my previous study I did not use photography because I felt that it was unethical not to give students an opportunity to see their work as they had no training or had been given no experience of photography (Hayhoe, 1995, 2000b). In addition, I also explained that I felt it was unethical to use media a student could not have feedback from.
I feel that this argument was ethical in the case of the students in the earlier study but it was irrelevant to this study. All of the students and teachers involved in this element of the study could not only understand the concept of photography, they also took and appreciated photographs themselves. However, all of their notes and photographs were, and still remain, open to their scrutiny for the purposes of verification.
What now follows in chapter 5 is a description of the fieldwork’s implementation, the problems I encountered during it and how these were overcome.
[1] Reflexivity is the process of reviewing theories as they arise during observations. This was not used, as the method I employed was designed to interpret specific observations. [2] His work was a traditional anthropological study conducted with First Nations people, hence the reference to them in the text. [3] Unfortunately, Marwick, like many historians, does not practice what he preaches. He does not give a full reference that I can pass on. |
To contact us:
E-mail:
We are based in:
Leicester, UK |