EDU 201 -- Jon Wagner, Winter 1999

Some Guidelines for Designing and Conducting Field Research

 

 

 

This booklet contains very brief guidelines that may help you answer some of the questions that will emerge for you as you try to complete the field work assignments for this course. They may also be useful to you if you chose to develop and conduct field studies of your own design. More complete accounts of these matters appear in the course readings.

 

 

1. Linking questions and data sources

2. Comparing field studies

3. Protecting the anonymity of research subjects

4. Developing interview questions and observation codes

5. Writing field notes and interview transcripts

6. Sample "head note" section for field note or interview

7. Transcribing and logging interviews

8. Organizing, exploring and analyzing qualitative data

9. Labeling data chunks, coding categories and theorizing

 

-------------------------------------------------------------------------------------------------

 

1. Linking questions and data sources

A key challenge in any kind of research is to get a good match between the data you can feasibly collect and the specific research questions you want to investigate. In order for your project to make sense, you need to identify the specific data sources will you use to answer particular research questions you are interested in.

 

Data Sources*

 

Research Questions

Official documents

Unofficial documents

Individual interviews

Group interviews

Surveys

Naturalistic observation

Field Experiment

Accounts of related settings/ phenomena

Group I

 

 

 

 

 

 

 

 

Question #1

 

 

 

 

 

 

 

 

Question #2

 

 

 

 

 

 

 

 

Question #3

 

 

 

 

 

 

 

 

etc.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Group II

 

 

 

 

 

 

 

 

Question #1

 

 

 

 

 

 

 

 

Question #2

 

 

 

 

 

 

 

 

Question #3

 

 

 

 

 

 

 

 

etc.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Group III .

 

 

 

 

 

 

 

 

etc.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

*Note: Data sources need to be specified further within each cell of the matrix. E.G. How many individual interviews with whom will you use to answer Questions x and y? How many with whom for Question z? Observations of what -- how many, how long, when -- will you use to answer Questions x and z? Also: Each data source may be useful in answering more than one research question. Answering individual questions may require more than one data source.

 

-------------------------------------------------------------------------------------------------

 

2. Comparing field studies

Field studies differ greatly in terms of some or all of the following dimensions. Comparing field studies in terms of these dimensions may be useful to you in assessing their value. These same dimensions may be useful to you as a template of questions you need to answer in designing and conducting field studies of your own.

Research design

Research questions

"Boundary" of the study

Data sources and data collection methods

Data organization strategies

Data analysis strategies

Difficulties encountered in conducting the study

Analysis and interpretation

What key concepts are explicated and which are taken for granted?

Which contexts of the phenomena being observed were made explicit and which are taken for granted?

Primary units of analysis

Links between data sources and research questions

Discrimination between observations, analysis and speculation

Theoretical assumptions

Conclusions about what?

Reporting/writing

Organization of the report

Structure of the central argument (if any)

Evidence presented to support or elaborate the argument

Range and distributions of details

Rhetoric and style

Intended audience

Inter-textuality

Significance claims and credibility claims

Author/researcher’s stance and agency

Claims of expertise or authority

Opportunities for identification with the reader

What aspects of the author’s person is revealed, neglected, or hidden?

Other

Relationship of the report to the study

Position of this study relative to other studies

What you liked and did not like about the reports

What you found credible and not-so-credible

 

-------------------------------------------------------------------------------------------------

 

3. Protecting the anonymity of research subjects

 

Please follow the following chart in determining who can know what about the identity of your research subjects and of other people mentioned by your research subjects in interviews and conversation.

 

 

 

Will have access to

Will not have access to

Individual 201 student interviewers

The names of people they interview personally

Identity of other people named in the interview they conduct

Identity of people named as responsible for particular functions

Identity of other students conducting interviews

The names of people interviewed by other students in the class

Identity of other people named in interviews conducted by other students

 

 

 

Other members of the class

Identity of people named in other interviews as responsible for particular functions

Identity of other students conducting interviews

The names of people interviewed by other students in the class

Identity of other people named in interviews conducted by other students

Outsiders:

Including research subjects for student field exercises

The name of the student who interviews them (if they are interviewed)

The name of the course instructor and the name of the course

Identity of other students conducting interviews

The names of other people interviewed by students in the class

The identity of other people named in other interviews conducted by students in the class

Identity of people named in interviews as responsible for particular functions

 

-------------------------------------------------------------------------------------------------

 

4. Writing field notes and interview transcripts

 

Anonymity and Confidentiality: After you have transcribed your interview or entered your field notes. Replace the names of all individuals with pseudonyms. Do the same for the names of particular programs, schools or institutions. Keep a list of the code to turn in with your work, but do not share it with other members of the class, other than those in your immediate work group.

 

Headnotes: At the top of the first page of your interview or field note, you should include the following information:

the date the interview or observation was made,

your name,

the pseudonym of the person you interviewed or observed,

the position of the person you observed or interviewed,

the pseudonym for the setting you observed,

an indication of whether this was an interview or an observation,

an indication of whether the interview was tape recorded, video recorded or recorded through manual note taking.

In addition to providing this information, you should include a paragraph or two that describes the context in which the interview or observation occurred. Was the person a stranger to you or a previous acquaintance? What they understand the purpose of the interview to be? Where did the interview take place and under what circumstances?

 

Text Formatting: To make good use of data-based or qualitative data analysis software, your field notes or interviews will probably need to be prepared for analysis as a straight "ASCII" or "text" file. To facilitate this process try to avoid using any higher order formatting techniques (e.g. bold, underline, strike through, shadow, multiple fonts, etc.) Instead, use a the normal character set and indicate emphasis, transitions, etc. with caps, brackets, quotation marks, slashes, backslashes, colons, stars, and numbers, etc.

 

Your Personal Comments: While you don’t want them to dominate, do not try to exclude all your personal observations and interpretations from either your field notes or interview transcripts. However, you should always set off observations and interpretations of your own by enclosing them in brackets or double brackets. In interviews you may use this same convention to describe activities in the interview situation that are not recorded on the tape (e.g. [Jane closed the book and leaned back in her chair]; [Sam was called out of the room at this point].) In field notes you may use brackets and double brackets to introduce your own commentary or interpretations. I suggest single brackets for direct observations of the setting and double brackets for conjecture, speculation and interpretation. For example, [the more Harry talked about this the more uncomfortable I became]; or [Janice seemed very reluctant to discuss anything about next year]. [[Could this be because of what Fullan had to say about the pace of change?]]

 

Annotations: After you have transcribed your interview or prepared an initial draft of your field notes, read these through to make sure that they are understandable. This is not a simple matter. You may need to include in brackets or double brackets some comments so that you will be able to understand at a later date what someone said or what some activity meant. Also take this opportunity to introduce guiding comments or annotations that could help you find particular observations or comments more easily in the future. These could refer to themes or concepts you are investigating or to different sections of the interview or field notes that correspond to different topics being discussed, different stages of activity, etc.

 

Preliminary Key Words: After you have done all the work noted above go back to the head note for your interview or field note and enter a few themes, concepts or issues that caught your attention as you read through this particular interview or field note. These will not be your final analytical categories, but they will help you start thinking about what those categories might be.

 

Follow Up: After you have completed all the steps noted above, make a list of additional questions you would want to investigate or steps you feel are necessary to move the larger research project forward. These could include checking on dates, documents or other background information, comparing this interview with other interviews, arranging for a subsequent interview, sending something to the person you interviewed, discussing something with a colleague, reading something, preparing a list of questions before the next interview, making some adjustment in your tape recording arrangement, etc.

 

-------------------------------------------------------------------------------------------------

 

5. Transcribing and Logging Interviews

 

Transcribing

In transcribing an interview you will need to develop conventions for how text will be punctuated. In doing so you may need to pay attention to the following: pauses, exclamations, vocalizations that are not words (e.g. laughter, sneezes, crying), hand and arm gestures that you can remember from the interview situation, and so on. You also will need to develop some conventions for linking the identity of speakers to what is said. This may seem simple for a one-on-one interview in which only one person is talking at a time. It is more complex when conversation becomes overlaid or when more than one person is being interviewed. You will also need some conventions for identifying chunks of texts that go together. Sentences are useful as are paragraphs. However, you may also want to identify a particular set of exchanges between two people as a single unit even if it is larger than a conventional paragraph. You may also want to identify topic shifts, though this is tricky if you are interviewing someone whose discourse style does not follow a linear 1, 2, 3 format.

Logging

In logging an interview your goal is to develop a summary and index for the interview that could be valuable in guiding your research, but to do so much more quickly than could be done through transcription itself. To develop an interview summary/log you will need a convention for noting either real time or tape counter time. If you want to supplement tape recording with hand-written notes at the time of the interview, you can prepare ahead of time a real time form to assist you in noting when shifts in topics occur. You can use a similar form when listening to the tape afterwards.

To log a tape accurately you will have to listen to the tape in its entirety. A one-hour interview will take a minimum of one-hour to log. You will also find it very difficult to note topic shifts for some speakers and for some sections of the interview. However, you should enter something for every few minutes on the log. Units of much less than 30 seconds are frequently too fine, though you may want to make even more frequent entries for particular sections of the tape. Units that are much larger than two minutes make it difficult to locate text sections later on.

One very useful strategy is to record a short verbatim phrase at every one or two minute mark of the tape as well as indicate important topic shifts, interesting comments, or longer verbatim transcripts in between and across these "one-minute markers."

In logging a tape, try to keep in mind some of the same themes and categories that you have prepared subsequent to reviewing your field notes or a complete transcript of another interview. Also note in your log any sections of the tape that you think are rich enough as an illustration or example of these themes that they should be transcribed later in their entirety.

 

 

-------------------------------------------------------------------------------------------------

 

6. A Sample "headnote" section for a field note or interview

 

 

Interview/Observation Code Number:

Name of Person(s)/Activities that were Interviewed/Observed:

Position/Title/Organization/ Setting etc.:

 

Interviewer/Observer’s Name:

Interview/Observation Date Time Place

Recording arrangements:

 

--------------------------------------

Next Steps:

 

 

---------------------------------------

Context: A brief description of the context in which the interview/observation was conducted (i.e., a field note about the observation or interview circumstances) that describes the physical setting, purpose, other participants, relation to prior conversations, relationship of person interviewed to you, etc.

 

 

Summary: A brief summary of how the interview/observation went: major themes addressed; patterns of question or problems identified; issues which elicited expressions of strong feeling; things that were difficult for you as an interviewer/observer; etc.

 

-------------------------------------------------------------------------------------------------

 

7. Developing questions for interviewing and observation codes

 

Here are a few distinctions and steps to follow that may help you in developing questions for interviewing or for coding observations in the field

 

Two key distinctions: Keep in mind these two key distinctions

 

Questions you want answered vs. Indicators of what you want to study

Questions you want answered vs. Questions that are good to ask people in interviews

 

For developing interview questions and observation codes

Brainstorm a list of potential questions

Cluster the questions into groups

Identify a general topic for each group

Turn each topic into a master question

Review/edit master questions

To cover full range of topics

To reduce duplication

To simplify and focus

Edit questions within each cluster:

To cover the topic

To reduce duplication

To simplify and focus

And to shift from "what you want to answer" to "what it makes sense to ask someone", to cover a topic, to reduce duplication, etc.

Order the questions within clusters

Order the clusters

Try out the completed interview schedule/script with people and see how well it works

 

 

For developing observation codes

Same as above, EXCEPT, instead of editing towards "what it makes sense to ask" shift to "what it makes sense to observe." This involves the following additional steps:

 

Brainstorm list of things that you could observe or otherwise collect information about

Identify things you can observe or collect that will help answer the master question for each cluster (i.e., What could you look at to learn more about what you are studying? What could you look at that is an indicator of what you focus on in the master question?)

Identify variations in the indicators you have chosen that correspond to individual questions within each cluster (i.e., What specific details will you look for within the general indicators that you are looking at?)

Make a list of the key variations you want to look for within each indicator

Prepare a matrix of all your questions and the indicators and variations you will be looking at

Check the matrix to ensure that you have indicators listed for all the questions you really want to answer (Note: some indicators may be useful in answering more than one question).

Review the overall plan and edit towards simplifying and clarifying and making this project feasible

Develop a coding sheet that lists each variation you will be looking for with each indicator.

Try out the completed coding scheme to see how well it works

 

------------------------------------------------------------------------------------------------

 

8. Organizing, Exploring and Analyzing Qualitative Data

 

You might find it useful to think of qualitative data analysis in terms of three stages: exploring your data, interrogating your data for answers to specific questions, and summarizing these interrogations for others who might be interested in your data, the phenomena you are investigating, or the questions themselves. Organizing and exploring your data can generate new ideas as well as provide a foundation for analyzing ideas systematically in the form of specific questions, propositions, etc.

As a researcher, you do these things with whatever data you need to answer specific questions. For some kinds of research questions -- but not all -- you need information about what people do and what they say and what they think. You might get some of this information in the form of numbers, but some of the most useful information will be in the form of text -- transcripts of interviews or conversations, your own written notes of what you observe, or notes and statements written by the people you want to understand.

The challenges of organizing, exploring and analyzing text data are somewhat different from the challenges of working with numbers. Some people who like numbers a lot try to convert all their text data to quantities and work that way. Others think that what you gain from this kind of translation is more than offset by what you lose, but it really depends on what question you are trying to answer. If you want to answer questions about how people experience and communicate about the worlds in which they live, you’ll probably need to keep their own words in the foreground, and that’s also true if you want to describe what you observe in terms that other people can see. While they may work within different theoretical or political perspectives, almost all "qualitative" researchers struggle with the challenges of organizing, exploring, and analyzing text data. Some qualitative researchers also work with data in the form of video tapes, photographs, and other graphic media, but if they write about these things then they also work with text.

 

Qualitative data and other text data

Qualitative data in the form of "texts" -- transcripts, field notes, texts prepared by research subjects, etc. -- is a lot like another kind of data that you have already have some experience exploring and analyzing, namely the written record of educational researchers, theorists, policy-makers and commentators. Most of the skills you have developed for exploring, reviewing, and analyzing this "literature" can also be applied to exploring, reviewing and analyzing qualitative data.

However, there are two important contrasts between the published research literature and the qualitative data that you have collected/constructed: (1) other people have already developed conventions for organizing, cataloguing and archiving the published research literature, and have already done much of the organizing, cataloguing and archiving for you; and (2) everyone has access to the published literature -- that makes possible a "secondary literature" of reviews, etc. that helps encourage informed and critical readings of the research literature -- but no one else has access to the data you have collected for your study (unless you are working on a team).

In terms of (1), you will need to develop conventions for organizing the data you have collected and constructed for purposes of your study. In terms of (2), you need to find ways to develop and support your own critical reading of the data you have collected. You also need to find ways to report what you have learned about your data. However, these are not that different from those you use in reporting what you’ve learned about the research literature.

 

Organizing your data

A useful rule of thumb is to organize, label, and annotate your data documents so that someone else could understand what they are and where they came from. In practical terms, the someone else may be you at some later point in the study. Some of the things you might want to think about include:

Informative file names: Develop some conventions you can use so that a file name itself tells you at a glance something useful about the file, allows for systematic sorts and searches, keeps related documents related, but still distinct, etc.?

Informative folders: Develop conventions for how you will keep files in folders. What will you organize these by? Data format (e.g., interviews, field notes, comments, etc.)? Research topics? Key settings or key informants? Will you want to use multiple folders and multiple copies?

An inventory of your data documents: Make a list/matrix of all your data documents that will allow you to see quickly where they are located and what their status is (e.g., tape transcribe yet? transcript coded yet? for which variables?)

Coding documents for data descriptors: Think through the logic of your study and identify what some of the key data descriptors might be (e.g., age, gender, position, of informants; observation or interview settings, etc.) and develop a process for monitoring your documents in terms of these descriptors while you are still collecting data.

A data base of your data sources: As an extension of the preceding few suggestions, create a data-base of your data documents that will allow you easily to explore and summarize the scope, focus and substance of your data as you move along.

Back up copies: Keep backup copies of everything you can, and keep them in two different locations (i.e., not just on top of your desk and in your desk drawer). Yes, you can always make multiple copies of computer files and stash some here or there, but a systematic but modest back up plan gives you the best protection.

 

Developing ideas about your data

Developing ideas about your data involves attending to tensions, surprises, and patterns that you notice on your own, but it also involves thinking about how your perceptions, those of your subjects, and those of the audience for your reports may or may not match. Some of the things you might want to think about in developing ideas include:

Personal reactions: Read through your data and take notes on: what you find interesting, surprising, unsurprising, well-stated, confusing, disagreeable, etc.

Key phrases of your informants: Make lists of key phrases that (a) appear frequently, (b) appear rarely if at all; or (c) otherwise seem to help describe key issues and themes.

Common or uncommon references: Make lists of things that are commonly referred to by the subjects of your study, even if they are referred by different names and in different ways.

Conversations with colleagues: Talk about your data with others who have similar interests and ask them directly for suggestions.

Conversations with informants: Keep asking your informants questions that come up for you when you explore your data documents. Do you have it right? Is there something else you need to know that they can tell you? What do they think of this idea, example, etc.? How would they handle this situation?

Comparisons: Take 2-3 documents that seem to present different perspectives on the issue you are interested in, read them closely, read them closely again, and try to identify all the points where they converge or diverge (charts, drawings, tables, etc. are great for this). Add another document to this mix and see where that one fits.

Memoing: Write a note to yourself or a friend about all that you’ve learned from reviewing and exploring your data: try to describe the "kinds" of data you’ve looked at, what data documents were most interesting and why, what additional kinds of data you would like to have, etc.

Presentations: Prepare and/or give a presentation about your study to others. You’ll learn something by organizing your thoughts and moving some considerations to the foreground, others to the background. You may also learn something from response to the presentation.

Reading: Read your data documents and read the research literature together. You are working between these two worlds, and if you let either one fade away, you’ll lose touch with the tensions that allow you to learn something of interest to other researchers.

 

-------------------------------------------------------------------------------------------------

 

9. Labeling, coding, categories and theorizing

Coding is a process of labeling documents and data in terms of their position in a study relative to other documents, data sources, research questions, theoretical concepts, etc. Moving from the mundane to the theoretical, documents can be described, characterized and coded in terms of the following:

Physical format: audio tape, photograph, text transcription, diagram, etc.

Temporal location in the larger study: date, sequence, iteration, etc.

Genre: interview, field observation, native document, researcher commentary, etc.

Sources/settings: specific informants, field sites, events, observers, etc.

"Face value" information re: common sense elements: activities, events, people, organizations, settings, etc.

Representations of subject perspectives: i.e., expressions/accounts used by research subjects to describe their world.

Representations of pre-selected phenomena: classroom interaction, teaching roles, writing prompts, assessment, mixed ability grouping, etc.

Intrinsic interest to the researcher: i.e., "neat stuff."

Representations of policy-practice issues: examples that represent, illustrate or instantiate key issues being debated and considered among policy-makers, practitioners, the public: e.g., school choice, multi-cultural education, performance-based assessment, teachers as researchers, etc.

Representation of analytical concepts: examples that represent, illustrate, or instantiate an analytical concept important to this study: e.g., tensions between individuals and groups, student-teacher role inversion; axes of variation; etc.

Labeling -- or "coding" -- documents and parts of documents according to these different characteristics can help you do the following:

-organize documents so that you can find them when you need them

-assess the quality/adequacy of the data you have for the study you are doing

-examine your data for patterns, relationships and associations

-examine your data for evidence to support/contradict specific propositions

The process of labeling/coding your data will also help you develop or refine the conceptual framework of your study. Depending on the focus and purpose of your study, you might code documents according to all, none or some of these dimensions.