There are two ways in which to organize the discussion of the techniques that are included in the RRA toolkit and the situations in which they are applied. One is according to the purpose for which they are to be used. The other is in terms of specific techniques. In this section of the discussion, the decision has been to focus on the specific techniques, describing them briefly and indicating some of the types of circumstances in which they are applied. Where there are strongly controversial issues related to the use of particular techniques, such as is the case regarding the value and applicability of formal sampling to RRA exercises, these aspects of the toolkit are discussed subsequently at greater length as detailed issues.
The general methods that are employed in RRA to address the time constraint on data collection and analysis are: use of cross-checking or triangulation, extensive reliance on the available secondary data, use of detailed but open-ended interview guides to ensure pertinent issues are covered, and extensive team interaction to maintain a multi-disciplinary perspective. This paper will not go into detail about these general methods. The box below lists a number of excellent sources that describe the general principles of the RRA toolkit.
The most important principle is open-mindedness. Most errors in RRA occur because the team stopped questioning their hypotheses and accepted their findings at a too early stage. A good team always questions the findings that are emerging and chooses tools and methods that provide new evidence and raise new issues. Simple using RRA to confirm one's assumptions is the biggest pitfall in applying such an approach.
Triangulation or cross-checking is not unique to RRA methods. It means quite simply gathering information about a particular topic from a variety of different sources, using a variety of data-gathering methods. If, for instance, a farmer informs the team that eucalyptus will not grow along his field boundaries without greatly reducing his yields, the team should "check" this information by asking other farmers with similar fields, by discussing this issue in group interviews, by checking what project reports have said on this subject, or by measuring the impact of existing trees on crops in other fields. If the same information is heard repeatedly, it is likely to be correct. The trick in using cross-checking is to be sure the information is actually coming from a different source. Teams with tight time constraints often make the mistake of asking questions only in a limited geographic area and only read reports regarding that limited area, and come away generalizing about a very site-specific or class-specific phenomenon. For example, one FSR/E expert talks about a diagnostic survey team which concluded labor was a severe constraint to farmers in the harvest season. In fact, this information only pertained to the small set of farmers they had interviewed, who lived relatively close to a sugar-cane growing area and who migrated at harvest time to capitalize on the high wages paid by the sugar-growers. Their cross-checks had not included a broad enough geographic radius.
This topic is well-discussed in the sources mentioned above. The proper of pre-existing and secondary data is always problematic in the short RRA exercise, since there is seldom adequate time allotted to the team for background literature review and document-reading. Some research team leaders a conscious effort to budget field time to include substantial gaps of time during which team members can read documents, particularly those only discovered the field visit area. For natural resource management planning, maps and to graphs of many different kinds are a key set of pre-existing information idly increasing in use. Practitioners also go back to raw data collected in surveys and re-analyze key topics before going to the field. One issue for the proper use of aerial photographs and remote-sensing imagery is the need for individuals on the team or available to the team who are adequately trained to interpret these documents. Often this requires someone who knows the area in the map or photograph well, along with an expert in map or photo interpretation.
Two of the findings of this review have been that: 1) the systematization of interview guides for land-based, natural resource management planning is increasing, and 2) this is a methodological area which would benefit greatly from more work on compilation of the interview guides being developed by practitioners for different topics. Interview guides differ by subject matter and by the geographic area for which they are being used. More guidelines are being developed for specific topics and specific areas. As mentioned above, John Bruce (1989) has developed a series of checklists on land-tenure issues. Barbara Grandin (see Annex 1 for address) has developed a ranking exercise to reveal wealth criteria. A number of host-country institutes are working on developing sets of socio-economic proxy indicators and sets of checklist topics that are applicable to short-term, field work. Articles are beginning to appear in the academic literature regarding the applicability of specific indicators to different field situations, for example see Schwartz (1988). The RRA Notes which are regularly published as a newsletter by IIED, London, has tried to compile the available information on the progress made on this topic.
One particular topic that is only beginning to receive detailed attention in interview guides is institutional and staff capability and constraints. While teams tend to pay close attention to bio-physical issues, socio-economic issues, or human resource issues related to the beneficiary/client population, a weak area has been sufficient questioning about the dynamics of local-level functioning of an agency or organization in terms of staff incentives, supervisor/worker relationships, status of staff vis-a-vis their clients, decision-making strategies, and organizational priorities.
A commonly-recommended system of allocating team resources during a RRA exercise is the sondeo. Sondeo is a Spanish term applied to a system of team interaction developed in the CGIARs, initially by Peter Hildebrand. Under this system, the team members split up in pairs each day of the field visits, rotating the composition of the pairs, so that participants of different disciplines all benefit from interacting with each other on a one-to-one basis. The team members in this system meet regularly as a whole to redefine their objectives, discuss emerging hypotheses, and reallocate their time for the remainder of the RRA exercise. Most RRA exercises advocate some version of this principle of mixing and matching disciplines in an optimal way. In reality, this often remains more of a principle than applied reality, because of perceived time constraints or the difficulty of imposing this degree of order on the team members' interactions.
Interaction with a multi-disciplinary team becomes a central issue for the social science-trained members of an RRA team. Often these team members are the most sensitive to the pitfalls of field survey techniques and are most conscientious about finding ways to avoid the pitfalls. They are frustrated in field interviews when other team members are not aware that lines of questioning or survey approaches are generating misleading information. Yet it is a difficult decision as to whether these specialists are more useful when they spend substantial amounts of time apart from the main team or not. Particularly when RRA is used for a broad-purpose, exploratory exercise, team members have multiple objectives: collect good data about the local situation, provide multi-disciplinary perspectives on the issues, interact closely with local people, and sensitize planners and decision-makers present in the field to particular, sticky, problem areas. Jamieson (1987) has a good article about the interactive RRA paradigm, and why it provides a very different perspective to the process of rural development project planning than do other forms of appraisal.
In interviews conducted for this review, particularly with social scientists who use RRA techniques on a regular basis, an interesting mix of opinions emerged regarding the best ways to develop a good team interaction and still allow the social scientist to maximize the quality of information collected from local people. Some argued strongly that the role of the social scientist in a team setting was threefold: to collect relevant socio-economic information for recommending a course of action, to introduce a social perspective to team members from technical disciplines and project officials, and to provide training for officials and extension workers in fruitful methods of interviewing beneficiaries through the RRA exercise itself. Leaving the team for several days of independent interviews, perhaps with a translator or junior project staff member, was seldom a wise decision unless a special target group was likely to be excluded from the sample unless they went alone to interview them. What was important to these social scientists was that other team members and officials have confidence in their observations -- something only possible if other team members and planning officials experienced the interviews for themselves. This group concurred with the article by Jamieson, arguing that good RRA is revolutionary for project planning because it leads planners to genuinely talk to target participants -- both host-country planners as well as expatriates.
How the team split in the field for this opinion group was a question of logistics -- available vehicles and how high profile the visit had become. With more than one vehicle in high profile situations, one vehicle could take the 'official" route and the other an informal/considered route. With proper team preparation, sub-teams could ask questions for each other and compare notes in the evening. (Though with big teams and big agendas, this seldom happens -- with no time to complete the sets of issues in team's checklist most pertinent to your own area of expertise, individual team members tend to ignore topics of interest to other members.)
The training function of field interviewing was also considered central. Spending time interviewing with extension staff provided these staff with invaluable training for carrying out subsequent sensitive interviews and for becoming aware of the importance of previously unnoticed socio-economic factors. It also reinforced the donor agency's concern that a project be "participatory" to field staff, a must in plantation or soil and moisture conservation projects that usually tend to over-emphasize physical-achievement targeting.
A second group argued for adequate time apart from the rest of the team and high-profile officials. Only then could they have enough confidence in their findings to argue strongly for a particular course of action when the report was being written. They understood the trade-off expressed by the other group, but felt reliable and comprehensive information-gathering on socioeconomic issues was of more importance. Constraints on working exclusively as a team were that other team members often short-cut necessary introductory warm-up questions/dialogue in an interview and made it impossible to get a grip on subtle social dynamics or areas villagers were reluctant to discuss. Most of such individuals spent half or more of their field time on their own. One practice of team leaders to meet both objectives was to hire the team social scientist and maybe another technician concerned with local conditions for a longer time period so they could spend a week or two prior to the rest of the team's arrival in the field sites gathering field information.
Some examples of the kinds of information that such practitioners felt difficult to collect included: the range of fuel scarcity and use patterns expressed by men and women in a region, gender-linked attitudes in general, decision-making criteria for agro-forestry practices in a complex ecological and market setting, and sensitive issues on which people were apt to stonewall teams, such as land tenure and common property management. Resettlement specialists mentioned that high-profile visits tended to reveal a more positive picture of the new settlers' situation (an Indonesia specialist noted that if people interviewed said the situation was simply okay rather than wonderful, she translated that to mean that there were some very serious problems).
RRA approaches work best when the team of field researchers or appraisers is small (2-6 members). When larger donor agency or host-country planning teams of 7-14 or more try to apply systematic RRA tools to their work, many logistic problems arise. For the long-term, development agencies could do well to reassess their current strategy of deploying large planning teams to the field en masse. More effective for using RRA tools and approaches would be to deploy two smaller teams at separate times and producing a single report subsequent to their visits. With large teams, the productive and structured dynamics of RRA team interaction during interviewing and problem analysis are simply impossible to achieve. While this would require a major change in bureaucratic protocol, it could greatly increase the quality of planning at little additional cost.
The following are some guidelines to follow when teams are too large to allow for an effective interaction among the team members when they travel in the field as a single group.
Are other vehicles available? - if YES can split upteam
Can the team in general avoid a high profile? - ifNO must split into at least two groups, so one is high profile and other isnot
Is the social scientist needed to elucidate keysocial issues to team and project staff in course of field visits?
- if YES then social scientist spends more time withteam
- if NO then social scientist may spend more timeapart from team with a local assistant/translator
Are the main questions essentiallymulti-disciplinary, requiring particular disciplines to be represented ateach interview? -- if YES send divide team appropriately (forester-socialscientist, economist-ecologist, etc.)
All RRA applications make use of individual, household and key informant interviews to gather information about the local situation. Some general principles of the toolkit are ways to reduce bias in the questioning process. Rhoades (1982) on the "Art of the Informal Survey" is a good guideline along with Warwick (1976) The Sample Survey: Theory and Practice. Acquiring expertise in sensitive and non-biased interviewing is a slow process requiring considerable training and long-term field experience. Khon Kaen practitioners advocate developing this capability by constructing RRA teams that always include only a minute proportion of untrained or semi-trained members on the premise that other team members can in this way devote adequate time to their training in the course of the RRA. Humility is the key. The good, seasoned interviewers are humble when evaluating the effectiveness of their own techniques. When one technique proves biased, they substitute another.
Interview techniques are a core element of the toolkit. Since there are a host of applicable techniques described in conventional and RRA literature such as those listed in 3.2 and 3.2.5. following, I have not devoted much space to this topic here. A few of the more central tools for proper interviewing are listed below:
avoiding questions that are phrased so as to lead the respondent to provide a certain answer (non-leading questioning)
use of the six helpers (what, where, who, when, how, why) for all topics to ensure that the interviewer is really understanding the situation and not drawing conclusions on the basis of partial information
probing, or not stopping when a respondent replies, but continuing to elicit more detailed information about that response
using local names for socio-economic characteristics, bio-physical characteristics, lands, customs, time intervals and measures
eliciting the local systems of classifying those things relevant to the project (trees, types of arable and non-arable land, seasonal variations, constraints to productivity, forest products) (Indigenous technical knowledge) as well as understanding these from the researcher's perspective
collecting detailed information about the history of resource use, for a particular piece of land or water resource. Individuals can provide information about specific cases of resource conflict (when a court case was filed about the encroachment of a wealthy herder on a common property resource, when a local woman was prosecuted for extracting wood illegally from a public forest, etc.)
When interviewing households, rather than individuals, it is important not to make the assumption that one member of the household can speak for all the rest. Women will have different knowledge and opinions than men and older individuals will see things differently from younger individuals. Older individuals have a different perspective and different set of priorities than younger married and unmarried members. Children's input into the farming system of local resource management is often missed, since adults, both men and women, generally omit this kind of information in interviews. Gaps in the knowledge base of different household members can be a key to differences in responsibilities or differences in their perceived interests or needs.
There has been a common assumption in RRA manuals that women are necessary members of teams to elicit information and opinions from women in field visits and interviews. While adding a woman's perspective to a field exercise and offering her often greater knowledge of women and development issues is important, such a rule should not provide an excuse for male team members to avoid talking to or eliciting information about women. A seasoned FSR practitioner with strong expertise in women and development was very adamant in an interview carried out as a part of this review on the point that men can gather exceptional amounts of information from and about women if they make an effort to do so. One of this practitioner's self-assigned roles in RRA exercises was to encourage male team members -- both researchers and local extension agents -- to collect information about women and not leave this task to her. This and a growing number of other female practitioners are concluding that even in highly segregated societies, information on land-use patterns and resource management issues can be acquired from and about women by male researchers and male extension agents, if they are given the confidence and sensitivity to do so.
There are certain interview practices that work much better with women than men. In general, women are more familiar with local, cultural categories, time intervals, size classes, and measurements than are men. An Africanist cites an instance in which lands had begun to be measured in hectares rather than in acres. Less educated men and most women did not "convert" their land size when this change occurred, but for a given size farm land began to tell interviewers that they had 2 hectares rather than 2 acres, even though the plot had not changed size. Women are less used to formal interviews, and questions must be phrased in a straightforward manner that does not assume the respondent is clear as to why the question is being asked or what the researcher means to find out.
When properly interviewed, however, women may have very detailed information about harvest quantities, processing values, storage losses, consumption patterns, etc. The interviewer must know how to collect such information. A classic example from African experience is the great underestimation of root crop production, because these crops are harvested as needed from garden plots, rather than harvested all at once (Hill, 1986). Questions must creatively elicit estimates of total production from consumption patterns, rather than relying upon "guesstimates" of total harvesting.
Women also tend to know a very different set of information than do men about resource availability, resource use, and resource processing. Interviewing women about a particular village resource or farming system will invariably elicit a different and sometimes conflicting set of information about use and optimal management practices. Women may benefit very differently from an activity than do men, and therefore have a different opinion about the value of particular interventions. The need to appraise women as well as men is part of a general rule of effective RRA. The team should be careful to find out who is engaged in what activity or sector and interview that person rather than whoever happens to be easiest to interview. Information can be as easily biased by trying to interview a landlord about the work done by his or her employee, or a farmer about the constraints faced by a herder, as by interviewing men about the values held by a woman.
In terms of resource management, women may have strategies that are not commonly understood or of interest to men in the same village. Women may have fuel-harvesting techniques designed to maximize forest regrowth, or they may meet informally to decide on forest protection techniques that are not discussed in detail with men, since they do not regularly harvest forest products from that resource. Women may also have very different suggestions about project activity plans than will the men of the same locality.
Key informants are a major source of information for those conducting in depth research or for those interviewing under time constraints as is the case with RRA applications. Key informants, simply defined, are individuals with a special knowledge of the topic of interest (respected leaders, irrigation system maintenance personnel, chairperson of the forest committee, etc.). RRA manuals pay particular attention to describing how and when to interview key informants and how to combine such interviews with other information. There are a number of manuals for RRA methods that provide insights into interviewing such individuals and about how to weigh their answers. One excellent book on this topic is Eliot J. Feldman's A Practical Guide to the Conduct of Field Research in the Social Sciences (Westview Press, Boulder, Colorado, 1981). One issue for use of key informants is that a particular respondent can serve both as a key informant and as an individual interviewee at different points in a single interview. A village headman can be a key informant at one moment, when advising a team on village-wide patterns of land use, and be an interviewee in the next, when describing his personal land-use practices to the team. Team members must be able to keep these roles separate when taking notes about what this individual says and when evaluating the information acquired.
|
One male interviewer who works often on community forestry has found that he can much more easily interview women in Asia by starting' with questions about their stoves and cooking habits. This leads him into their kitchens where they feet at home and in command, and in this setting they are forthcoming in expressing opinions attitudes and felt needs about a whole range of forestry and local economic issues. |
There are a large number of documents regarding proper ways to conduct an interview. These include Warwick (1976), Rhoades (1982), Chambers (1985), Khon Kaen (1987), and Odell, Odell and Franzel (1986). These include providing the proper warm-up to an interview, ordering questions in an open-ended and unthreatening way, interviewing on-site without unduly disrupting the individual's work (many farmers will appreciate the interviewer's offer of help with an on-going task before breaking off from work to answer questions), interviewing individuals in a comfortable setting (women generally speak more freely in their kitchen than outside the house), and phrasing questions in a way that is easily understood by people interviewed.
One practitioner recommends adding a last question to the interview schedule which is, "Are there any questions that you would like to ask us?" This allows the interviewer to get information that might have been missed, puts the respondent(s) more at ease since the interview is not totally one-sided, and also provides a cross-check on whether the respondent and interviewer understood what each was getting at. If the question is out of the blue, there is a good chance that the respondent didn't really understand what the interview was about and the interviewer is unlikely to have elicited an accurate picture of the respondent's behavior or attitudes.
The interactive tools (ranking games, sketch map-making, discussion charts) that will be discussed in a separate section are other tools to reduce bias and ensure that researcher and respondent understand what each is saying. Rational selection of interviewees and sampling methods are another way to reduce bias.
Opinions differ on the best way to use interpreters/translators during RRA. Nearly all practitioners agree that being able to converse directly with interviewees is a decided advantage of using interpreters or translators, and that promoting RRA skills in local institutions should be a high priority. Nevertheless, there are many occasions when RRA work must be done when RRA practitioners lack native language competence. Nearly everyone agrees that, whenever possible, native language speakers should also be included in the team and that this will be of great help. However, during interviews themselves, using a fellow team member as a translator can be problematical. Each team member has his own skills and interests, and it is very trying for him or her to also act as a translator during interviews. At best, statements tend to get "interpreted" or summarized, rather than translated.
On the other hand, using professional (ideally, simultaneous) translators, while preferable during interviews, may also be difficult. Professionals in this field, for the two languages needed at the time, may be hard to find, and many have had little or no experience working in rural areas, much less in the kind of highly sensitive modes necessary for translation work during RRA. There are many practical hints that can be used to help: control your translator"; have the translator sit behind you, and slightly to the side; maintain direct contact with the interviewee yourself, talk directly to him, not to the interpreter, pay attention to him when he responds, even though you do not understand what he is saying, etc. Perhaps most important, practitioners emphasize the necessity of taking the time to make sure you understand exactly what was said and what it means, and that the interviewee understands what your mean. Interviewing with translators is, of necessity, an even slower, more difficult and more sensitive process than would otherwise be the case during RRA work.
Group interviewing is an important element of the RRA toolkit because of a) its usefulness in collecting information from a wide range of individuals relatively quickly, b) its usefulness in generating discussion of relevant problems, issues, and optional solutions, and c) its usefulness in the process of village-level planning of activities. Written information about group interviewing techniques focuses much more often on the first two objectives than on the third. In addition, group interviewing is often unavoidable, since household interviews often become group interviews when passersby flock to see the team at work and must ultimately be included in the interview. Manuals also include techniques for discouraging new arrivals on the scene of the interview from dominating the discussion and for capitalizing on their presence by changing the nature of the interview.
There are a number of excellent guidelines on group interviewing apart from the guidelines mentioned in Box 4. These include Kumar (1987), Shaner, Philipp and Schmehl (1982), Chambers (1985), Khon Kaen (1987), and McCracken, Pretty and Conway (1988). Honadle (1982) and Hendricks (1987) include useful charts outlining the trade-offs between different types of group interview techniques.
Like individual interviews, group interviews are a tricky proposition. Gow (1987) makes the important observation that community interviews can often provide a team with a distorted picture of the local situation because local leaders dominate the discussion, because respondents discuss an 'ideal reality' rather than actual practices, because respondents may take control of topics out of the hands of the moderator, and because interviews with large teams often degenerate into a chance for respondents to air grievances and petitions. Kumar (1987) suggests some strategies for circumventing these problems:
carefully worded, leading questions to circumventavoidance of real issues 3;
interviewing a group of village leaders beforehandand introducing the meeting with their remarks to encourage others tospeak;
sub-dividing the group into smaller, morehomogenous, working groups;
varying topics to spark the interest of a widergroup; and
using humor when pointing out the limitedparticipation of certain subgroups.
A practitioner drawing upon his work in Africa suggests that group discussions can more fruitfully lead to village consensus about their desired course of action if the research team discusses issues of importance informally with a traditionally-respected community leader and then allows that leader to generate the discussion, thereby gaining the confidence of the group of villagers present that this topic deserves their interest and attention. In discussions of the optimal ways to achieve effective common property resource management over a particular piece of communal land, for instance, villagers are more likely to seriously consider new options and provide a research team with an honest opinion if the discussion is initiated by a local person who commands their respect on this topic, than if the team members present hypothetical options from their outside view of the situation.
There is often an assumption that host-country team members and local project staff will be able to generate a balanced, useful discussion with local groups of people. While it may be the case that local professionals will better understand local attitudes and conditions and there by generate a more honest and, relevant discussion, this is not an absolute if the individuals are not skilled in interviewing and use of the toolkit. Host-country team members often have as many -- though different -- pre-conceptions about "what the villagers think or do " as the outsider. Or they may have a very clear understanding about a certain group or class within the village, but fail to recognize the diversity of families' situations. Host-country team members must be careful to filter their own assumptions when choosing questions, phrasing questions, or leading a discussion.
Group interviews can be useful in the following situations to provide complementary data to that available in individual interviews.
1) TO find out village-wide information, or to define an initial range of situations that can be refined in household or individual interviews (such as, how many cattle people own, whether grazing areas are adequate, what is the effect of time spent in gathering fuelwood on the overall workload of a typical arm household, where people go for outside employment and how prevalent this source of income is for different types of households);
2) WHEN smaller, homogenous groups of people are assembled for a specific purpose when the team enters the village -- women, landless, smallholders, irrigated-land owners -- group interviews generate in-depth information about differing perspectives of different types of villagers);
3) TO gain information from a group of 'professionals' who are less likely to be as frank as individuals about their problems -- government extension workers, local medical personnel, local foresters -- to identify the range of their views on a subject;
4) WHEN information from individual interviews does not make sense -- groups may clarify the information and explain it more comprehensively in light of additional information or they may react emotionally to the information, thereby revealing a key issue needing further discussion; or
5) TO elicit information about local knowledge categories, so that questions asked in an individual or household setting will be more fruitful (techniques promoted in the toolkit for this objective are the ranking games, such as those described by Chambers in Box 14 of this report, or questioning that uncovers how local people make decisions about collecting or harvesting specific products, such as that used by Eric Rustin [Research in Progress], in his research on fuel and fodder in the Nepalese agroforestry system).
A specialized group interview technique that is becoming more popular for natural resource management-related surveys is the focus-group interview. Focus-group interviews are adapted from social marketing methods in private industry and involve interviewing relatively homogenous groups of local people or government extension agents, such as women seeking medical care, smallholders growing fodder trees, or forest guards. This technique is appropriate when smaller, homogenous groups of people are naturally assembled during the field visits -- all women, all landless, all smallholders, all irrigated-land owners -- to gain in-depth information about particular issues. It is also used to get a consensus of opinions from a group of 'professionals' -- local foresters, local medical extension personnel; etc.
Questions in these interviews are highly focused on a few, key issues. It is generally not useful to have such interviews at a very exploratory stage, before the team understands the general parameters of the problem. It is useful to evaluate the range of circumstances of the group and to provide a setting in which individuals are comfortable about giving their honest feelings or opinions.
One of the most critical and most controversial areas in sound use of the RRA toolkit is selection of respondents and sampling. There is a wide range of opinion on this subject -- some practitioners try for an intentional selection process (purposive sampling is one name for this), interviewing people and groups of different classes, ethnicity, age, gender, resource base, and adding new respondents to round out the gaps in information that are emerging. Other practitioners find this process unacceptably biased and either combine this method with some random sampling or make more conscious choices based on formal sampling principles.
There have been studies comparing the findings of informal and formal, statistical surveys in a particular geographic location for a particular topic (Franzel [1987], Ngamsomsuke, et al [1987]). There appears to be a strong lack of consensus as to whether these studies have more general application.
The key controversy is not whether the purpose of RRA is to formally sample the population -- a clearly impossible task -- but whether some kind of application of formal, sampling principles in the context of an RRA exercise is a useful means of correcting bias. The proponents of this stance do not feel that all interviews should be selected in this way, but feel this should be the manner of selecting some of the people. The practitioners with this viewpoint feel strongly that even when teams work hard to reduce bias (following some of Carruthers and Chambers' [1981] pointers)4, they will fail unless they include some interviewing with a more formal method of respondent selection. They find it especially important that individuals who are not well-seasoned field interviewers not try to generate a balanced, purposive sample.
Some practitioners argue for more follow-on of RRA exercises with small, formal surveys to substantiate or undercut conclusions emerging from the RRA. One practitioner at the World Bank has developed a computer package for such surveys carried out as part of project monitoring and evaluation for use by the non-statistician. It assumes that the users will have partial information about the question for which study is needed. The sample size will be determined by what is already known on the question and therefore the size of the sample will be generally much smaller (yet equally accurate and less costly) than that usually devised for a large-scale, formal survey. (See Annex 2 for a description of this program.)
What can realistically be adapted from formal sampling theory in selecting people to talk with during RRA-style field visits? What formal sampling achieves in statistical surveys is to reduce the chances that investigators will pick a certain set of individuals over another, thereby coming out of the field with a skewed impression of the local situation or problem. A well-selected statistical sample will have what is known as low, random sampling error. Any survey, however statistical, has a chance of other sources of bias (skewed results), which are lumped together in survey terminology as non-sampling errors. One criticism that is often levied against formal surveys is that, while the random sampling errors are very small, the non-sampling errors resulting from poor wording of the questions, poor choice of question order, lack of sufficient attention to the context in which the question is asked, and poor choice of a time of day to hold the interview, can be much more damaging than sampling errors. The sample for the survey may be picture-perfect, but the data that results from the survey erroneous and useless.
In-depth and open-ended interviews attempt to reduce the non-sampling errors by paying close attention to putting the person at ease, asking questions in a number of different ways to reduce the chance that the question was misunderstood, eliciting longer answers from the person to ensure the researcher understands what is being said, and a host of other such techniques. However good the interview, it remains difficult to be sure whether the interviewee or household is typical, unusual, unique, or expresses a universal condition without an adequate sampling of the rest of the population. In a situation in which RRA tools are being used, there is seldom time to search out individuals selected through a complete, random sample or to code and analyze the information collected. Use of key informants who have good village-wide knowledge is one check on the representativeness of individual interviews carried out. It is possible, however, to in addition to interviewing key informants, apply some of the principles underlying random sample selection to help reduce the biases from a completely purposive sample.
One such principle is stratification. In RRA, this is considered a form of triangulation (cross-checking). This is a technique used in constructing a formal sample that ensures that certain groups in the population are included, despite the limited size of the sample. A stratified sample is constructed by dividing the population into the groups of importance. One can either take a percentage of the sample from those groups in proportion to their representation in the population -- i.e., 50% women because men and women are more or less equally present in the population, or 20% landless if 20% of rural dwellers are known to be landless. Or, the sample can take equal numbers of each strata --30 poor, 30 landless, 30 wealthy farmers, 30 smallholder farmers. This sample principle can guide informal, purposive sampling as well, to ensure certain groups are adequately included.
|
Box no.12 - Proving the Null Hypothesis While quick turn-around surveys like RRA do not give the interviewers a chance to talk to enough people throughout the population to draw firm conclusions about hypotheses, there are ways to add more validity to the data collected. one way is proving the null hypothesis. All statistical analysis is based on the use of mathematical tests that establish whether a particular' correlation of patterns means something or whether it occurred by chance sampling of unusual individuals. if a survey of 80 households finds that 70% of the trees planted had a survival rate of 80%, the mathematical tests serve to show that it is highly unlikely that this survival rate would be measured in the sample population unless it was representative of a general pattern in the population as a whole. If the sample population had 70% survival of trees, says the statistician, it is most likely that the rest of the population did as well. These tests disconfirm the null hypothesis, which argues that the sample is not representative, but a skewed subset of the population. |
One application of this principle to quick' surveys that do not use random sampling is, instead of sampling the` population to prove a 70% survival rate is prevalent in the following chart summarizes some of these options.
Figure 3: SAMPLING TECHNIQUES APPLIED IN THE RRA TOOLKIT
| Problem | Possible Solutions |
| 1. How to ensure that the views of less visible target groups are not under-represented in interviews | - Stratify the sample of interviews that will be carried out during field visits to include specific proportions of various groups (resource-rich/resource-poor, men and women, old/young, landless/landed, different ethnic groups) |
| 2. How to combat the common problem that more remote agroecological zones e poorly represented in RRA field sits | - Pre-select sites that include these zones when the team plans their program |
| 3. How to include some random sampling that is not too time consuming to compose a sample and find people in a limited time period to generate information that is convincing to planners who require more statistical or quantitative evidence of a team's findings | - Use the Null Hypothesis (interview a limited number [4-8] of individuals to disprove rather than prove something which has been a working assumption or finding of earlier monitoring surveys (i.e., survival of homestead plants has been found to be lower than survival of plants people put on field boundaries in project formal reporting). If most of your "sample" does not conform to the rule, you can be sure your working hypothesis is questionable, since the statistical likelihood of encountering so many exceptions to the rule is verylow. |
| - a. Use the quick and dirty sampling program developed by Ronald Ng, (See Annex 2) to obtain a sample for a complementary, sample survey to the RRA itself. | |
| - b. Use existing lists to draw a quick sample (health registers, nursery registers, voting lists) with a random numbers table in the field. | |
| - c. Interview a small sub-sample from a previous, formal survey sample (such as one conducted by the ME unit of the project earlier on, or from a baseline survey). | |
| 4. What to do to test hypotheses that are only formulated halfway through the RRA exercise | - Try a random sample halfway through the field visits to test emerging hypotheses. Plan a second visit and make the RRA an iterative process. |
| 5. What to do when households are reluctant to give accurate information on sensitive issues | - Combine individual interviews with: - group interviews - participant observation o direct measurement - secondary data review o unstructured, casual conversations - ranking or planning games o key informants - change households |
There are two points that are relevant in deciding whether or not some randomizing adds to the validity of the information collected. First, random sampling gains the researcher nothing if the interviews of people selected through that random process are poorly conducted. Second, however much use of RRA techniques may be seen as cost-effective, it still costs time and money. It may be worth more in terms of scarce resources to introduce a little more rigor into an RRA-style exercise and make it a slightly longer process, than to have to come back to the field with the same range of team members to collect the same sort of information later on. This is an issue that will no doubt receive more attention by those project staff adapting RRA more extensively to forestry and natural resource management programs in different project stages.
Two opinions as to why sampling might not be appropriate which were expressed in the course of this review are as follows:
1) Formal sampling has no place in an RRA, which is a "scan" of the range of situations in the project area. The objective of an RRA exercise is generally to find out the scope of a problem or of several issues. The team does best by keeping their "noses in the wind." There is neither time to carry out formal sampling, nor is the objective of gathering information about general issues/hypotheses commensurate with a need for formal sampling.
2) The biggest constraint on formal sampling in an RRA is time. While it could contribute substantially to validity of information, there just is not time to devote to constructing a random sample and interviewing those in it.
One set of social scientists who were interviewed focused not on the trade-offs of random versus purposive/opportunistic sampling, but on the need for RRA practitioners to become more conscious of the value (and limitations) of qualitative data collection methods that are traditionally applied in a longer-time frame, but that can be adapted to the RRA setting. One such methodology is "situational analysis".
In ethnographic work, situational analysis is used as an alternative to talking to a range of individuals. Instead, the interviewer tries to gather information as completely as possible about a single "situation" or a set of situations of importance to the project (grazing conflict, allocation of common lands, introducing contour plowing on a farm, irrigation water allocation). The set of groups and individuals interviewed is dictated by the "situation" rather than by a methodological decision to interview a selected "range of individual types" (small/large farmer, upstream/downstream, and so on). What emerges is an in-depth understanding of a situation that the team sees as a salient pattern in the project area. Those interviewed represent a range of opinions/views of the situation, and interviews give the team some key ideas about local decision-making processes, a key to land-based resource management. Recommendations made on the basis of such an approach (either alone or in combination with other methods) are grounded upon in-depth knowledge, but the approach relies upon the prior training of the interviewer in social and cultural theories of norms and behaviors, at least those relevant to the geographic area where work is being carried out.
A corollary to this is the importance of time-based information about resource use. Conway's Agroecosystems Analysis (1986) and Raintree's Diagnosis and Design (1986) analyses both include questions about use of land for trees and crops over time. In addition to checklists about what crops were grown over time, however, is the importance of information on the history of conflicts over land-use and management of these potential or actual conflicts. Villagers should be asked about changes in the composition of management groups, the history of their formation, changes in leadership, use of watchmen or people assigned to maintenance tasks. In conflict history, it is important to establish how the social or economic role of the individuals involved influenced the decision. If a woman had grazed her sheep on the land, would there have been a case or was it because it was a man with a large cattle herd? For cropping patterns and trees, Conway's approach includes collection of information for a 10-year period for both crops and prices, a very useful body of information when evaluating the economics of an intervention from the farmer's perspective.
The qualitative approach generally requires a social scientist on the team, who has a first-hand understanding of qualitative research methodology, so that they can correctly evaluate what values and norms underlie a particular person's expressed opinion or action. For participatory planning applications of the RRA toolkit, there may not need to be a social scientist, but local staff should have solid training before undertaking such an exercise on their own as well a first-hand understanding of the communities with which they plan to work, not just a common national identity.
|
Box no.13 - Situational Analysis Rangeland and: use patterns: can be highly complex and herders are generally reluctant to offer information to outsiders about their hard sizes; movements, or ownership rights. This makes us of RRA tools highly difficult in village-level planning or research efforts related to this topic. One good source of information is: the history of conflicts over rights. Different parties will provide different perspectives on events in the conflict and their resolution which can be very helpful in ferreting out sets of rules and responsibilities for pasture use and access. One practitioner advocated: hypothetical questions arising from discussions of actual cases, such a, "If women had let her cattle loose in this pasture rather than a man, would the conflict have been different?" |
