“What would it take to end homelessness in Boston?” “What would it take to reduce childhood obesity by 50% in Germantown, MA?” “What would it take for students to eliminate Styrofoam from Boston Public Schools?” These are the types of challenging questions that emerged recently at an AGM workshop for nonprofit and foundation staff on outcomes measurement; and they are essential to guiding the work of any organization. But how do we know if our day-to-day activities are leading us to answer these big, mission-related questions?
Collecting and learning from data on results achieved by an organization can help us comprehend how the mission of an organization is achieved. The hard part is implementing an effective outcomes measurement process.
Outcomes measurement refers to the “so what?” or hard results of an organization’s work. Many training programs teach the skills needed to measure outcomes. For instance, staff at a youth development program can learn how to track the socio-emotional growth of participants. But learning the skills is not enough. Organizations must also have the capacity to implement evaluation practices, interpret the results, and react purposefully.
In a fast-paced, interactive workshop held last week at Associated Grant Makers in Boston, GMA Foundations staff Prentice Zinn and I led nonprofit and foundation staff through a number of exercises aimed at 1) demystifying outcomes measurement and the value it can bring to an organization and 2) providing some basic tools to begin preparing for outcomes measurement. The exercises also challenged the group to identify key obstacles to implementing outcomes measurement.
We tested a diagnostic tool developed by Informing Change to help participants determine their organization’s readiness to implement evaluation. Prior to the workshop, participants completed the “Evaluation Capacity Diagnostic Tool”. A majority - 8 out of 12 organizations surveyed - said they would also use the tool within their own organizations.
When asked about what they learned from the diagnostic tool, participants agreed that implementing outcomes measurement involves overcoming the following key challenges:
- Developing shared tools throughout an organization
- Recording data consistently across programs
- Increasing staffing capacity and time to evaluate and learn from data
- Translating existing outcome indicators more clearly and creatively
- Overcoming internal resistance to beginning an evaluation process
- “Losing sight of the forest for the trees” or focusing so much on data that the long term vision gets lost
It can be helpful to think of outcomes measurement as just one part of an evaluation approach that can lead to learning and organizational change. After digging deeper into the obstacles, the group agreed that certain steps can help prepare an organization to implement outcomes measurement effectively:
- Understand where the organization currently stands – here’s where we are, here’s what our challenges are, here’s what our key indicators currently are – to identify the possibilities for growth.
- Own a particular set of indicators to tell the organization’s story in the most authentic way.
- Institutionalize a process for learning from outcomes that is a team effort.
According to participants, presenting outcomes measurement as a positive learning opportunity can be a critical way to engage the team. Learning-as-motivation gives staff ownership over the data, makes the evaluation process a team effort, creates “buy-in” from individuals, and helps shift organizational culture. From this perspective, understanding an organization’s results in a data-driven, concrete way can be used to think critically about the difference we are making in the world.
Funders also have an opportunity to learn from the challenges grantees face in carrying out outcomes measurement and evaluation. Capacity-building grants and unrestricted funds for general operating support can help nonprofits track and understand outcomes, but as we have seen, an organization must be ready to do this work. Funders can start conversations with nonprofits about how data helps them achieve their goals and can encourage organizations to focus on outcomes that demonstrate impact. Ultimately, data will be most valuable when it teaches us something about how an organization is accomplishing its mission, and asking the right questions can help us get there.