Systematic reviews of academic literature for evaluating the effectiveness of farm advisory services: Preliminary findings based on a case study about farm advice and occupational health.

Résumé 0

This report addresses the question of the evaluation of interventions based on farm advisory services. The aim is to provide guidelines for producing evidence about the effectiveness of such interventions. These guidelines have been tested on a specific case: the effectiveness of farm advice for reducing occupational exposure to pesticides of farm labour. In the first section, we discuss the distinction between the different objectives which an evaluation of farm advisory programmes can endorse. We propose to distinguish three major goals for evaluation, by adopting the typology of Berriet et al. (2014): i) to measure the impact of farm advisory programmes; ii) to understand the causal path that generates changes through farm advisory programmes; and iii) to support learning processes for the stakeholders involved in a programme to in turn enhance its implementation. A key point of this typology is that it allows for reflection on the methodology which is needed to gather the best possible evidence according to the goal pursued by the evaluation. The ‘best’ possibleevidence should be: a) socially relevant to those concerned (external validity); b) based on types of evidence (evidence of presence, of difference-making, of mechanism) which is adequate for the purpose of the evaluation; c) reliable (produced using rigorous methods, to ensure the highest degree of probative force, i.e. the highest level of evidence). Numerous studies provide frameworks for describing the impact schemes (or causal paths) of farm advisory service interventions, however there is a clear knowledge gap regarding the actual of their impact, especially in Europe. The second section is focused on this specific question of impact assessment of interventions based on the support given to farm advisory services. It highlights the fact that many methods (experimental or quasi-experimental) are now available to implement impact assessments. It also gives some critical points for evaluating the internal quality of studies based on such methods and for identifying their limits of validity for practice. A key dimension is that the results of impact assessments of farm advice interventions might not be extrapolated beyond the very context where the study was implemented. This statement calls for a need for reviews that combine results from impact assessments implemented in various contexts. The third section poposes methodological corner stones for the implementation of such systematic reviews about the effectiveness of farm advice. It describes the different steps for implementing this methodology, and proposes some examples of analytical tools such as: i) algorithms and diagrams to search and sort the papers; ii) tables to describe and evaluate the quality of papers; iii) criteria to combine evidence from different researches. We tested this methodology on a specific case: the effectiveness of farm advice regarding the reduction of occupational exposure to pesticides of farm labour. The case study is presented extensively in section 6. The results highlight a specific knowledge gap in the European context. There are almost no studies on the impact of farm advice on health issues in European agriculture.Nevertheless, the systematic review reveals that some positive effects of interventions (training, advice...) on farmers and workers have been demonstrated in North Ameca (Canada and United States of America).