Interviews as Data Collection

By Barry Sweeny, 2003


INDEX:


Interviewing Basics

Every form of data collection has its strengths and limitations and interviews are no exception. Also, as in every method of data collection, it is necessary to choose the tools you will use based on the goals of the process and other circumstances, and then comparing those goals and factors against these strengths and limitations, to decide if interviews are an appropriate tool, or whether other methods are better
suited to the task.

Technically and idealistically, interviews can be structured or unstructured. However, this author strongly suggests that you take a more practical approach and use ONLY Structured methods for interviewing. Given that decision, keep in your mind that YOUR interviews will NOT be just conversations which are written down. If they WERE just a process of capturing whatever people talk about, then they would be easy to do, but you will face a major challenge making the data you collect useful to your purposes. If interviews are purposeful and well done, they are not easy to do.

To be a productive and manageable form of data collection, interviews must be structured, facilitated, focused dialog which serves a specific purpose.


The Four Skills of Good Interviewers

The interviewing process has four parts: Therefore, the core skills are:
1. asking a question 1. forming good questions
2. listening to the answer 2. being a good listener
3. capturing what is said in response. 3. being a good note taker or good in using whatever technology is used to capture
the response.
4. ensuring accurate reception & understanding of the intended message. 4. using paraphrasing and asking clarifying questions

If the interviewers are skilled in the four ways described above, then the interviews will be well done.

 

If the interviewers are not skilled in each of these four ways, they can be trained to develop the skills that are needed, but that is a more involved process. The better alternative is to select interviewers who have the requisite skills already.


The Strengths of Interviews as Data Collection

Interviews are best used when what is needed will require careful listening to the opinions, experiences and perspectives of the subjects, who are stake holders in the process being evaluated. As a result, interviews are a good method for uncovering and clarifying differences in opinions and perspectives among a selected sample from a group of people.

Also, interviewing allows in-depth exploration of issues, perhaps better than any other form of data collection. Therefore, interviews are needed when that is likely to be needed to fully understand any issues.


The Limitations of Interviews as Data Collection

Standardization of the interviewing process is critical. Such standardization requires clarity across all interviewers about the extent to which they will or will not pry, pursue, clarify, etc. to understand what interviewees have said. Whatever they do, they must all do it the same or the comparisons across the various data they collect will be confusing or even impossible. Setting up the conditions for such standardization is time consuming and complex.

Interviews are time-intensive and therefore, not very cost effective. This is true before, during, and after the interviews since:

  • (Before) Developing and field testing the questions takes time, as does standardization of processes, and any training of interviewers.
  • (During) Interviewing takes time
  • (After) Classifying responses, analyzing and disaggregating the data, and finding meaningful patterns to interpret, all take time. While such analyses are not actually part of data collection (this article’s topic) the time for doing these processes is a result of decisions about the method of data collection, and so was deemed relevant here.

SAMPLE SIZE –

Another limitation of interviewing is that you can only give the in-depth time it takes to a few interviews, so necessarily, you can only interview a smaller sample of the total number of stakeholders. The actual size of the sample is dictated by the size of the total group. The primary goals are to hear in sufficient quantity from every segment or potential view point, which means every stake holder or role group, that all perspectives are revealed, and the perspectives can be confirmed as valid because they are held by a sufficient number of the people in the sample.

Sampling is complex. Choosing a totally random sample has some statistical benefits, but may miss hearing from all stake holder groups, and the smaller the sample, the more likely that is. That’s why this author prefers what he calls a “balanced sampling” approach, which uses a mix of:

  • interviewees who are carefully selected for their ROLES, so AT LEAST 3 groups specific view points are more likely to be revealed (i.e. mentors, protégés & managers)
  • but the specific individuals from each role group are randomly selected.

In that way you are ensured of hearing whatever diverse viewpoints exist, but you have not biased or structured WHAT the view points are that you will hear. This approach is primarily a quality control issue, so it is important to takes such steps. That way YOU and those who receive your findings later on, will be more likely to accept the findings as credible.


Helpful Interviewing Hints

  1. Be thorough, systematic, and standardized, or don’t do interviews. The results will not be worth it, and could even steer you wrong. Carefully follow the above advice regarding use of goals for deciding approach, consideration of the strengths and limitations of interviewing, and decisions about the size and make up of the sample of interviewees.
  2. Use one unstructured open-ended question or opportunity for interviewee comments at the end of each set of questions on a topic. That increases the likelihood that you are hearing what the interviewees want to tell you, not just the information for which you have asked.
  3. Decide when the findings are needed and then work backwards to structure a time line for the whole process.
  4. Field test your questions with at least one person from each role group in your sample. Then refine your questions before using them with the whole sample.
  5. Before doing the interviews, think about what data you will have as a result, what the data will look like, and how you will organize it for analysis. Doing this may change how you decide to record the interview, even how you may organize the data for analysis. This step can actually save you many hours of tiresome labor later on. Don’t skip it.