Outcome Harvesting and Ripple Effects Mapping for Measuring Big Societal Changes

by | May 3, 2023 | On the Blog

How to measure big societal change?

We love learning about new methods! And we recently had the opportunity to do just that. Last year, Catalyst was hired by a private foundation who is funding professional organizations to make big societal changes. Our role was to join the client’s team to create an external evaluation review process that will happen about 5 years after the organizations are funded. No one expects the groups to have made their full impact in that time, and so how will the foundation know if they are on the right track? What may be the indicators that the groups are on their way to having their intended impacts? What if the groups are having unintended impacts – how can we design an evaluation that will pick those up? 

We had a few ideas, but decided to reach out to our evaluation networks to ask for recommendations. People shared several suggestions with us including Appreciative Inquiry, Journey Mapping, Outcome Harvesting, Ripple Effects Mapping, and Value Creation Stories. Ultimately, the two that were most relevant for us that we decided to do deep dives on were outcome harvesting and ripple effects mapping. We share a little more about each plus our recommended resources below.

What is Outcome Harvesting?

Outcome Harvesting flips more traditional evaluation approaches on their heads. Often we define our intended outcomes and design instruments that will measure progress toward those outcomes. Instead, outcome harvesting starts with the outcomes – it looks for what has changed and then works backwards to determine the likely causes. Outcomes and their causes are substantiated and verified by people who are independent but knowledgeable. As you might suspect, this substantiation makes Outcome Harvesting a time- and resource-intensive method! This approach is particularly well suited for complex projects that are likely to have unpredictable (even negative) outcomes. For more information about outcome harvesting, we recommend reading Wilson-Grau & Britt (2014) available here.

What is Ripple Effects Mapping?

Ripple Effects Mapping brings together stakeholders to share stories of the project’s impact and visually map out the changes or “ripples.” The process begins by applying an Appreciative Inquiry* approach, where participants interview each other in pairs about the best parts of the project – the elements that they would want to carry forward and build upon. Participants then come together as a group for a facilitated discussion and reflection to identify the key effects from the interviews. Then the group works together to create the map to depict the causal chains between all the effects. After the mapping session, facilitators may need to further reorganize the map and substantiate the data through additional interviews. For more information about outcome harvesting, we recommend reading A Field Guide to Ripple Effects Mapping, available here.

What we picked and why.

While there were many strengths of Outcome Harvesting, we decided that Ripple Effects Mapping was the better fit for this evaluation plan because it uses both a participatory approach and utilizes elements of appreciative inquiry. The participatory approach aligned very well with the foundation’s values – it was important to them that all stakeholders have a voice and share their stories. We also saw great potential for the appreciative inquiry questions to help the funded organizations identify successes for them to build upon.

*Appreciative inquiry is an organizational change approach that goes well beyond the questions used in Outcome Harvesting. The method begins with the same types of questions used in Outcome Harvesting (e.g., What is working well? What gives the project life?) in the inquire phase. In the subsequent three phases, participants build upon the answers generated to imagine what might be, innovate what should be, and implement who will do what (see Preskill & Grindle, 2015 available here).

Get in touch!

Want to learn more about how Outcome Harvesting or Ripple Effects Mapping could be used to show the impact of your work? We’d love to talk with you!

Meet the Catalyst Team

We're glad you're here! We are evaluators, researchers, and strategists who study and support STEM education programs. We specialize in the in-between spaces where we can use data to get context, tell important stories, and help our clients take strategic action in their programs.

Recent Blog Posts

More Blog Posts to Read:

Tengamos una plática (Let’s have a talk)

Tengamos una plática (Let’s have a talk)

The country's Hispanic population is increasing daily, leading to the incorporation of new languages, traditions, customs, and cultures. While it is important to recognize that the Hispanic population is not monolithic, it is also true that many Hispanic populations...

read more
Reflections on academic publishing

Reflections on academic publishing

At Catalyst Consulting Group, we love a good, data-driven story. We recently published an article in the Journal of Geoscience Education that is just that. While Catalyst and our larger group of co-authors for this paper have a history of publishing this kind of work,...

read more
Reflecting on Evaluation in Service of Racial Equity

Reflecting on Evaluation in Service of Racial Equity

At Catalyst Consulting, we enjoy honing our skills, growing, and deepening our understanding of evaluation and research methods. We come together once a month for a lively discussion on a variety of topics. Over the past year, we spent three of these meetings...

read more