Community Forestry Field Manual 1

Community Forestry Field Manual 1: GUIDELINES FOR PLANNING, MONITORING AND EVALUATING COOKSTOVE PROGRAMMES
prepared by Stephen Joseph, edited and designed by Carla R.S. Koppell
FAO, Rome 1990


Preface

This publication has developed over time, with inputs from a large number of people. In October 1983, under the leadership of Marc de Montalembert, a meeting of experts who had ongoing stove programmes was organized in FAO, Rome. Participants were: Mrs. J. Ki-Zerbo (Upper Volta), Mr C.E. Estrada (Guatemala), Mrs. H. Navarathna (Sri Lanka), Mr. M. Kinyanjui (Kenya), Mr. A Sudjarwo (Indonesia), Mr. T. N. Bhattarai (Nepal), Mr. G. Madon (France), Mr. S. Joseph (Australia), Mr. F. Manibog (World Bank) and Mr. C. Nieuwvelt (The Netherlands).

They discussed the issues which needed to be confronted when monitoring and evaluating stove programmes. This material was synthesized in a draft set of guidelines. This draft manual was distributed widely and was the basis for over 20 reported project evaluations. The same participants, together with other researchers and rural development specialists also attended The First International Meeting for Woodstove Dissemination held in Wolfheze, The Netherlandslater in 1983.

The debates were lively but it was evident that they were often based more on sentiment and commitment than on tested results. Participants at the meeting emphasized the need for guidelines that could give comparable information while also helping stove project personnel manage their own programmes with more confidence.

In 1987 the Foundation for Woodstove Dissemination (FWD), which had developed out of the Wolfheze meeting, organized a meeting in Guatemala entitled "Stoves for the People". The staff of many of the projects represented at the first meeting participated as did those of several large, new improved woodfuel stove efforts. The atmosphere was significantly different at this meeting. Eight of the programmes had done extensive evaluations and had solid results to present and discuss. This material has been edited and can be obtained from FWD. However, though the results of the evaluations were encouraging, there was also discussion of the need to simplify and make the guidelines more participatory in line with the new approaches projects are using.

Dr. Steven Joseph, familiar with each of the field projects and has a global, first-hand knowledge of woodstove programmes was selected to prepare the work. He wrote the first-draft document and was further commissioned to streamline and modify the guidelines based on the experiences of those who attempted to use the draft guidelines. This field guide was edited and designed by Carla R.S. Koppell. This work was supported by the FAO/SIDA Trust Fund Forests, Trees and People Programme.

This publication is felt to reflect the concerns and interests of many reviewers. However, it needs to be field tested. FAO would appreciate having feedback directly to the Community Forestry Unit, Forestry Department, Via delle Terme di Caracalla, Rome, Italy 00100.

Marilyn W. Hoskins
Senior Forestry Officer
(Community Forestry)
Food and Agricultural Organization


BACK TOP TABLE OF CONTENTS BOTTOM NEXT

Introduction

This publication was produced in response to requests for a mehtodology which encouraged more local participation than conventional programmes and was based on actual field experiences.

The objective of the manual is:

to provide a concise set of participatory guidelines for plannning, monitoring and evaluating both pilot and national stove programmes.

These guidelines can help people prepare their own monitoring and evaluation manuals. The manual provides the following information in the section that is indicated:

  1. a concise definition of monitoring and evaluation and specifically, the criteria and indicators to be used (Section 1);
  2. a methodology that involves users, producers, extension workers, managers, and research workers in the monitoring and evaluation of all phases of stove programmes (Section 2);
  3. a method for planning and managing monitoring and evaluation programmes (Section 2);
  4. a simple system for the collection, analysis and use of monitoring data that can be used and understood by extension workers (Section 3);
  5. guidelines for training monitors who will collect data and communicate it to users and producers (Section 4);
  6. the minimum information needed to monitor and evaluate all of the different phases of a larger stove programme (Section 5);
  7. a number of simple methods for collecting data in both urban and rural areas (Section 6);
  8. simple methods for analysing data and presenting it to users, producers, stove designers, extension staff, managers and donors (Section 7).

The manual is designed so that it can be used by professionals at all levels, in all types and sizes of programmes. The text of the manual:

  1. clarifies the objectives of M & E;
  2. develops a framework for planning and managing stove programmes;
  3. develops an effective monitoring and evaluation training method;
  4. develops a simple, low cost participatory method for collecting, analysing and using information to improve the acceptability/performance of the stoves;
  5. outlines one method that can be used to disseminate information about the stove.

This method has been developed largely from the experience of the Nepal HMG/World Bank/FAO Community Forestry Programme and the Sarvodaya Stoves Programme in Sri Lanka. It emphasizes participation and works to assure that stoves are developed that appeal to and address the problems of users, producers, distributors, sellers and stove programme personnel by maintaining a constant multidirectional flow of information.

The Appendices allow you to tailor the monitoring and evaluation system to meet your needs and capabilities. They contain information that can serve as background or can be used to design and implement monitoring and evaluation programmes in small distribution programmes or large national programmes. There is detailed information on the techniques that can be used to collect and analyse information. It should be noted that the simpler techniques can be used in larger programmes but more information may be desired from local stove groups and key informants if a detailed evaluation is to be carried out. Likewise if smaller programmes wish to undertake a detailed impact evaluation they may want to use these thorough methods.


BACK TOP TABLE OF CONTENTS BOTTOM NEXT

Case examples: different ways to implement pilot monitoring programmes

The three case studies that are summarized in this section, are representative of some of the approaches that can be used. The experiences of these three programmes are further expanded in the text as they provide examples of approaches and techniques that can be used.

Guatemala

Initially implemented by ICADA, a voluntary organisation involved in community development and appropriate technology in the highlands, this Guatemalan stove programme's objectives were to introduce stoves that reduced fuel consumption, and improved the kitchen environment. The implementors also saw the programme as a way of increasing local community involvement in the development process.

Canadian and British volunteers on a small experimental station in the highlands created the initial model, a large mass mud stove (called the Lorena) based on an Indian design. It developed from a dialogue with eight local women over a period of six months. It was tested in both the designers' and the initial users' households. The designers spent considerable time with the users refining the stove's design. The designers relied on users' perception to gauge fuel savings. No detailed measure of fuel consumption was used.

Following the stove's development, other volunteers and then local extension workers were trained to build the stove. The households who had received the stove were visited by the builders to rectify minor problems. No record of the monitoring programme was kept. One year later a second voluntary agency, CEMAT (the Center for Mesoamerican Studies on Appropriate Technology), became involved in the dissemination of the stove. People from villages were trained to build the stoves with support from technicians/instructors from CEMAT. Various individuals assisted establishment of commercial stove production enterprises. Much of the follow-up was undertaken by these trained stove builders. The staff monitored the complaints, and recorded the innovations users suggested, while both the staff and the builders modified the stove. There was no formal system for collecting, storing and processing the data though a limited evaluation of the initial ICADA project was carried out by a volunteer in 1979. The evaluation indicated that the extension strategy and the stove's design needed to be improved. Both the monitoring and the evaluation led to implementation of a more comprehensive training and follow-up programme.

Funding was obtained to undertake a much larger national programme and a number of organisations joined in the dissemination effort. An evaluation of all the programmes was undertaken in 1984/85 to determine the factors that affected adoption of the Lorena stove. Further needs identification surveys were used to determine the other groups of people that needed improved stoves and how stove designs could be modified to meet the needs of these new groups. CEMAT has been helping other organisations disseminate the stove throughout Guatemala as part of a larger national cookstove programme.

Kenya

The Urban Stove Programme in Kenya was initially implemented by the Ministry of Energy using grant aid from the US Government. Energy Development International hired a Kenyan consultant to train ministry staff, and provide advice on implementation of the programme. ITDG and Approvecho provided other technical assistance. Research and testing was, and continues to be, carried out by Kenyatta University College with dissemination by local voluntary organisations, small businesses and individual artisans. The Kenyan Energy and Environmental Non-Government Organisation (KENGO) has at this point, taken on the role of coordinating all cookstove activities.

The principal objective of the Programme is to reduce the household consumption of charcoal by improving the efficiency of existing metal charcoal stoves. A secondary objective is to reduce the amount of toxic pollutants emitted.

Initial baseline data on patterns of fuel consumption were available when the project started, but further information on existing production and distribution patterns had to be obtained. The data were used to design the first prototypes and develop a comprehensive pilot programme strategy. Initial laboratory testing and user trials were carried out by the University. The new design added a ceramic liner to an existing metal stove.


Kenyan improved charcoal stove

Once experimentation indicated that the stove was both more fuel efficient and acceptable to women who had done cooking tests, a voluntary organisation was contracted to conduct field trials with the new stove. Five hundred stoves were introduced into areas in Nairobi and Mombasa. Households were surveyed before the stove was introduced and two and six months after their introduction. Changes in fuel consumption and factors affecting usage were recorded. During the field test period, some of the households were also informally visited.

A great deal of information was obtained from formal surveys, but due to the lack of resources, detailed analysis of the results took over a year to complete. Monitor impressions and a visual analysis of the survey data indicated that the stove saved fuel and was acceptable to most people. Informal surveys and users' analyses formed the basis for the design changes that were then made. A programme to develop production units and distribution outlets was initiated.

The resources for a formal expansion phase monitoring system were not available, although project staff kept records of visits to different production units. At the end of one year, a consultant evaluated the progress of the different production units and determined who was selling the stoves and how they were obtaining them. Based on this evaluation and an analysis of the monitoring records, a more comprehensive extension strategy was developed. This new strategy included a research and development segment devoted to improving production techniques.

Following this evaluation, a monitoring programme was set up by KENGO. In 1986 an evaluation, funded by FWD, was carried out to determine who was using the stoves, why they were using the stove and what the impacts of stove use were. The results of this evaluation have been used to improve the promotion programme.

Nepal

In Nepal, a stove programme was implemented by the Division of Community Forestry. The primary objective of the programme was to reduce present fuel consumption levels by introducing 15,000 stoves over a four year period. The secondary objective of the programme was to improve health and hygiene in the kitchen. The total budget allocation for the project was approximately $200,000. ITDG was contracted to provide an ongoing consultancy input. The initial design and development work was carried out by a research institute (RECAST) in the national university. The extension, monitoring and evaluation were done by the Division of Community Forestry.

Before the project started, an initial fuel use survey was carried out by the Monitoring and Evaluation Unit within the Community Forestry Division. At the beginning of the project more specific data relating to the local kitchen systems was collected. Based on this information, ceramic stove designs that had been developed in Indonesia, Sri Lanka and India were modified and laboratory tests were carried out. Local potters produced design ideas; 100 of each of the 3 most promising designs were produced and field tested in three villages. Stove performance measurements and monitoring of acceptability were carried out over a six-month period.


Use of the stove built for Nepal

Initial field results showed that one of the designs was both fuel efficient and acceptable to most people. The Division of Community Forestry installed another 600 stoves in other villages for more extensive field tests. Local people were trained to install the stoves. Initially, no formal monitoring programmes were implemented. Information was collected by field and headquarters staff and RECAST personnel. Consultants helped improve designs to overcome problems as they became apparent.

After approximately six months a monitoring survey was carried out. The results of this evaluation indicated that the basic design was acceptable to most people, but refinement was necessary. After improving the stoves, training programmes for installers and stove promoters were developed and a programme to introduce the stove to other areas and other ethnic groups was undertaken.

One year after the start of the expansion phase, monitoring/evaluation results indicated that improvements in the training of the extension workers and potters were necessary and that a formal monitoring system was required to record rates of production and installation, estimates of production quality and the number of follow-up visits. The programme has now entered the dissemination stage. Regular monitoring and evaluation is carried out once a year and the extension strategy and/or the design are adapted to meet changing needs.


BACK TOP TABLE OF CONTENTS BOTTOM NEXT