Narrative Methods of Project/Program Evaluation

Have you been involved in a project/program in which a rigorous project/program evaluation was not possible or simply not wanted? For such times there are other methods to conduct evaluations, especially those with few pre-determined indicators, difficulties in implementing rigorous studies, or with the possibiliity of many unforseen or unitended results/outcomes.

One broad approach to evaluating projects/programs is called the "narrative" method. The narrative method is described by Charles McClintock (Dean of the Fielding Graduate Institute’s School of Human and Organization Development, in his article "Using narrative methods to link program evaluation and organization development" published in The Evaluation Exchange Volume 9, Number 4, Winter 2003/ 2004 by the Harvard Family Research Project).

The narrative method is fundamentally storytelling and is related to participatory change processes because it relies on people themselves to make sense of their own experiences as it relates to the project/program. And, the participant and beneficiary's stories can be systematically gathered and claims verified from independent sources or methods.

The narrative method can be divided into three basic types, depending on the purpose of the evaluation.

1. Success stories
2. Positive and negative outcomes
3. Emerging themes

1. Success Stories: One of the most prominant narrative methods for the purpose of success stories related to intermediate outcomes and impact is Most Significant Change, or MSC (Davies and Dart 2003, see MSC document list on the right tab). This method is highly structured and designed to engage stakedholders at all levels. Davies and Dart recommend MSC when a project or program is:

• complex and produce diverse and emergent outcomes 
• large with numerous organisational layers 
• focused on social change 
• emphasizes participation 
• designed with repeated contact between field staff and participants 
• struggling with conventional monitoring systems 
• highly customised services to a small number of beneficiaries (such as family counselling). 

2. Positive and negative outcomes: this narrative method is called the Success Case method (Brinkerhoff, 2003). This Success Case method has two phases: a) a short questionnaire sent to all project/program participants to identify those for whom the project/program has made a difference and those for whom it did not make a difference; b) next, a number of extreme cases are selected from those two ends of the success continuum (i.e., did and did not make a difference) and respondents are asked to tell stories about both the features of the project/program that were or were not helpful as well as other factors that facilitated or impeded success. Based on the logic of journalism and legal inquiry, independent evidence is sought during these storytelling interviews that would corroborate the success claims.

These stories serve both to document outcomes, but also to guide management about needed change in project/program interventions that will accomplish higher level outcomes and impacts.

3. Emerging Themes: This narrative methods is basically qualitative case studies (Costantino & Greene, 2003). Here, stories are used to understand context, culture, and participants’ experiences in relation to program activities and outcomes. As with most case studies, this method can require site visits, review of documents, participant observation, and personal and telephone interviews. Stories can include verbatim transcripts, some of which contained interwoven mini stories. Selection of a few cases, studied in-depth, are able to develop many more "themes" that are involved in a project/program and of relationships among participants and staff.