At Catalyst Consulting, we enjoy honing our skills, growing, and deepening our understanding of evaluation and research methods. We come together once a month for a lively discussion on a variety of topics. Over the past year, we spent three of these meetings discussing a series of webinars titled Doing Evaluation in Service of Racial Equity, commissioned by the W.K. Kellogg Foundation, and authored by Community Science.

Let me begin by telling you that I was beyond excited to delve into this series with my colleagues.  My career began in youth community services where I traveled throughout San Diego’s many neighborhoods striving to support youth, families, schools, and communities. What I learned during those years, including societal inequities, structural barriers, systemic racism, and their impact on historically minoritized populations, shaped who I am, generated the questions I’ve had throughout my graduate work and life, and inspired me to focus my evaluation work on programs seeking to address racial inequity.

Session Reflections  

Our first discussion centered around the suggestions given in Session One: Theories of Change and Logic Models: Use a Systems Lens and Check Your Biases. For Catalyst Consulting the use of a systems lens and learning to check our biases is ongoing. Having projects based throughout the U.S. and its territories, for our work to be beneficial to each of these programs, not only do we need to understand the program, its aims  and goals, who participants are, program services, and how change occurs, we also need to understand the contexts in which each program is operating out of. A systems lens approach broadens and enriches our understanding which, in turn, enhances our ability to support a program’s evaluation goals. By using the tree metaphor (see figure below) we understand that visible inequities are rooted in systemic issues. Importantly, as we are using this approach, we as evaluators, need to be open and willing to examine and identify our own biases. I’ll be the first to admit that identifying our biases is not the easiest thing to do. That said, when I’ve been introspective, new perspectives and understanding flourishes and I’m a humbler, improved evaluator, and person. One way that Catalyst explores biases is through our discussions. We’re a relatively diverse team (e.g., racial, ethnic, geographic location, academic discipline, work experience, age, disability, background, etc.) who takes the time to thoughtfully consider one another’s viewpoints, welcome challenges and face biases whenever they surface.

The next webinar, titled Doing Evaluation in Service of Racial Equity: Deepen Community Engagement, focuses on community which they define as “people and relationships based on some sort of connection.” It highlights the dynamic aspect of communities, that they’re often filled with people who belong to many communities (e.g., language, sexual orientation, culture, age, race, ethnicity, place, etc.) and ways to get to know the community or communities that the program sits in. Importantly, a community is not defined by outsiders, it is defined by its members. At Catalyst, we’ve spent several hours talking about and listing community details to learn about. In practice, it’s a genuinely awesome opportunity that allows us to be curious, to grow our awareness and to build relationships.

The last of the series, focused on The Traps of Myths and Mental Models, and why evaluators should rethink and debunk these notions. Three myths are discussed with the first myth being of particular salience to me. This myth states that “Rigorous science is objective and value free. Evaluators who are committed to racial equity are no longer objective and value free.” I’ve lived with this myth since my first undergraduate social science research methods course, into my graduate work and beyond. Where I saw my commitment to racial equity as a driver for learning and using my skills to address inequity, others saw my commitment as a risk. Although, I’ve remained committed to racial equity, a nagging critic would periodically pop in my thoughts and interactions with colleagues. It was empowering to listen in as the presenter highlighted how a commitment to racial equity and science principles propels evaluators to dig below the surface, examine the complex nature of racial equity and try to understand and evaluate it.  

Moving Forward

This series lived up to my expectations. It taught me new strategies and new ways of thinking. As expected, we had energetic conversations about it and most importantly are always striving to practice what we learned. Meanwhile, we’ll continue to pursue greater understanding, include community members in all aspects of our evaluation work, find resources that will push our skills, shed light on those nasty biases and mental models, and broaden ourselves and one another as we continue our evaluation work.  

Check out the full series here:

https://communityscience.com/webinar/organizational-effectiveness/evaluation-in-service-of-equity/