Over 40 Michigan organizations join collaboratory to center equity in evaluation practices

This article is part of State of Health, a series about how Michigan communities are rising to address health challenges. It is made possible with funding from the Michigan Health Endowment Fund.

Like many nonprofits, Ann Arbor-based Food Gatherers uses evaluations to measure the efficacy of its hunger relief programs and make a case for donors to continue their support. But the organization has joined other Michigan nonprofits in confronting the ways those evaluations may contribute to further inequity for those they're supposed to serve.

Food Gatherers recently joined the Michigan Health Endowment Fund's 2020 Equitable Evaluation Initiative Collaboratory, which aims to engage nonprofits in reframing evaluation work through the eyes of the people it serves. More than 40 entities from across the state are taking part in the collaboratory, including Michigan State University, the University of Michigan School of Social Work, Henry Ford Health System, Habitat for Humanity of Kent County, and Greater Midland.

"It’s important for us to look at not what’s important to us or our funders, but what is important to the community we work with,” says Markell Miller, Food Gatherers’ director of community food programs.

Markell Miller, Food Gatherers' director of community food programs.

The Michigan Health Endowment Fund is one of seven foundations across the country involved in the Equitable Evaluation Initiative (EEI). A five-year endeavor launched in 2019, the EEI engages participating foundations through collaboratories that explore and advance EEI's Equitable Evaluation Framework. Equitable evaluation creates a process that is accessible and comfortable for participants, and asks program participants open-ended questions that get at the true root of a program’s impacts. It also emphasizes awareness of the ways that systemic drivers of inequity are tangled up in the strategy being evaluated.

“It focuses primarily on the philanthropic ecosystem at this moment in time because there is a willingness and readiness to think differently about the effects of systemic racism on all things, including evaluative practice,” says Jara Dean-Coffey, director of the EEI. “Over the remaining three and a half years, we hope to deepen engagement with institutional philanthropy and expand to other actors in the ecosystem such as consultants and evaluators. Ideally, we'll begin to touch academia and maybe nonprofits. Our charge is to seed a field.”

Miller notes that Food Gatherers has been examining how to best evaluate its programs for some time. She and her colleagues want to be able to prove that they are making a real impact for the people they work with — not just compile data that looks good on an annual report. Through EEI's collaboratory, they met and engaged Dr. Ebony Reddock, owner and principal consultant of Ypsilanti-based Bumblebee Design and Evaluation.

“There’s all kinds of ways of approaching evaluation. When I was trained as an evaluator, [it was] about giving a client what they want in terms of what they need. If the evaluation responds to a funder’s questions, then that was what you were doing,” Reddock says. “Equitable evaluation leads to digging into processes that organizations implement internally.”

Ebony Reddock.

To help Food Gatherers conduct an equitable evaluation, Reddock suggested hosting a series of community meals throughout Washtenaw County as a way of gathering input from the people the organization served.

“These would not only serve as a way of learning from community members and engaging them in what Food Gatherers was doing. They also have the goal of promoting community cohesion,” Reddock says. “With COVID-19, we are needing to reexamine design elements, but ultimately we’re looking at making sure that the evaluation work we are doing is not taking away from community but partnering with them, that we’re being led by what we’re hearing from the community.”

When Reddock began doing evaluation work, her background in feminist studies and systems of privilege made it easy for her to view evaluation strategies through the lens of equity. During an evaluation of a health care system that sought to reduce health disparities, she says some in the organization had difficulty understanding why she was interviewing nurses and people within the organization.

“I knew from being a health educator and being engaged from a practice perspective that marginalization sometimes comes from within the healthcare system and not from outside of it. It’s not always poverty. It’s the cultural norms which place groups at a disadvantage,” Reddock says. “Equitable evaluation requires a certain willingness from the evaluator’s part to take a stand, to speak truth to power, which can be scary, but also it’s a responsibility.”

She adds, "The nice thing is we are in a moment where people are more willing to look at themselves and the work they are doing with a critical eye. Now they are grappling with that history and they are willing to do it.”

Another nonprofit participant in the collaboratory, Wayne Metropolitan Community Action Agency (Wayne Metro), works to eliminate poverty through homeownership workshops, counseling, utility assistance, educational programs, and more.

“Our goal is to serve all members of our community that need help. Our goal is poverty elimination in all segments of society,” says Nadeem Siddiqi, Ph.D., the nonprofit's director of data strategy. “If we do not have equitable evaluations, we are going to miss critical signs of who needs help and the types of help that is needed. Having equitable evaluation gives us this critical knowledge about who, when, and how they need help.”

Nadeem Siddiqi.

While Wayne Metro still uses standard evaluation processes of collecting demographic and program-specific data, the nonprofit is trying to better understand participants and modify evaluation practices to be more equitable.

“Bias in data collection is there," Siddiqi says. "You have to be very careful about it, no matter what tool you use. You can design biases to prop up a conclusion by the way you design interview questions."

Instead, Siddiqi says, now Wayne Metro's first step in developing an evaluation is asking: "Are we being equitable?"

“To me, equitable evaluation is simply about being honest and true to yourself,” Siddiqi says. “You need to get out of your own comfort zone and meet the person where they’re at.”

Reddock notes that, even before COVID-19 and the recent resurgence of the Black Lives Matter movement, many philanthropic foundations funding nonprofit work had shifted their mindset to pursuing equity. She lists the Michigan Health Endowment Fund, Kresge Foundation, W.K. Kellogg Foundation, and Robert Wood Johnson Foundation as foundations acknowledging that, historically, they had received money from racism and racist practices. They are now taking steps to dismantle racism from within. Equitable evaluation is one tool for accomplishing that goal.

“Evaluation as a discipline was built on racist beliefs and a racist way of thinking," Reddock says. "… These ways of evaluation cannot get to the heart of why program after program does not eliminate inequity. Ultimately, evaluation was serving inequity. As an evaluator, my responsibility is to change that and really look at the ways I contribute to inequity without meaning to."

In choosing to implement equitable evaluation, organizations are acknowledging that their success is tied to community members’ success. As equitable evaluation practices help organizations see that they have likely been a part of the problem — and that they do not have all the answers — they can better build programs that truly help them achieve their missions.

“Equitable evaluation is a way of letting go of that control and saying [to the people being served], ‘You are the expert. Please guide my work.’ If that’s done on a large scale, think how powerful that can be,” Reddock says. “Organizations that serve marginalized communities will actually belong to marginalized communities instead of treating them as a petri dish or a conduit that helps them stay in operation.”