Sharpening Your Lens: Defining Your Evaluation Focus
Written by Hannah Wohltjen
So, you're embarking on an evaluation – a crucial step for understanding impact and driving improvement within a program, your organization, or maybe for a collaborative effort. But before you dive into data collection and analysis, there's a foundational step that's often overlooked but incredibly vital: defining your evaluation focus.
What do we mean by defining your evaluation focus? (Spoiler: You don’t have to measure everything!)
At its core, defining your evaluation focus means pinpointing the specific questions your evaluation aims to answer. It's about narrowing down the vast landscape of your program's (or your organization’s) activities and outcomes to a manageable, meaningful scope. Instead of trying to evaluate everything, you're strategically identifying what aspects are most critical to examine given your resources, staff capacity, timeline, and the needs of your stakeholders.
Think of it like focusing a camera lens. A wide-angle shot might give you a general overview, but to truly understand a specific element, you need to zoom in and bring it into sharp focus. Similarly, a well-defined evaluation focus ensures your efforts are concentrated on generating actionable insights about the most relevant aspects of your work.
Why it’s important
A clear evaluation focus is helpful in several ways:
Efficiency and Resource Allocation: Without a defined focus, you risk wasting time and resources collecting data you won’t actually use. A clear focus guides your data collection strategy, ensuring you gather only what you know you’ll need to answer your key questions. It also reduces the burden on the folks you collect data from and helps you communicate why their input is needed and what action or decision making it will inform.
Actionable Insights: When your evaluation is focused, the findings are more likely to be specific and actionable. You'll be able to provide concrete recommendations for program improvement and organizational decision-making.
Stakeholder Alignment: A shared understanding of the evaluation's focus ensures everyone involved – program staff, partners, funders, beneficiaries – is on the same page about what will be evaluated and why. This fosters buy-in, enhances data quality, and increases the likelihood that the evaluation findings will be used.
Credibility and Validity: A well-defined focus enhances the credibility of your evaluation. It demonstrates a thoughtful approach and helps to ensure that your findings are relevant and robust.
3 Ways to Define Your Evaluation Focus and When to Use Them
There are several tools to help you define your evaluation focus, each suited for different contexts and purposes.
1. Logic Models
What they are: Ah. Trusty logic models. As most of you probably know, a logic model is a visual representation of how a program is supposed to work. It illustrates the causal chain from inputs (resources), to activities (what the program does), to outputs (direct products of activities), to outcomes (short-term, medium-term, and long-term changes).
When to use them: Logic models are excellent for:
Individual program evaluations: To assess whether program activities are being implemented as intended and understand if desired changes are occurring and at what stages. While some folks use these at the organizational level too, we find that they can get overly complex for larger organizations and require a high tolerance for detail.
New or developing programs: They help clarify assumptions about how the program will achieve its goals.
Programs with clear, sequential steps: Where there's a well-understood cause-and-effect relationship between activities and outcomes.
How they help define focus: By mapping out the program's intended flow, you can identify specific points along the chain to focus your evaluation. For example, are you most interested in whether participants are receiving the intended services (outputs)? Or are you more concerned with the actual changes in their knowledge or attitudes (short-term outcomes)? At Elevate, we generally recommend that folks prioritize measuring their immediate or short-term outcomes. While a logic model will clearly outline the contribution a program may have on longer-term outcomes and community impact, these can be more time and resource intensive to measure.
2. Theories of Change
What they are: Similar to logic models, a Theory of Change (ToC) describes how and why a desired change is expected to happen in a particular context. However, ToCs tend to be more high-level summaries of a program, organization, or collaborative’s hypothesis about how the work they do will address a community need and outline the "why" behind the "what."
When to use them: Theories of Change are particularly useful for:
Collaborative shared measurement: To provide an easy to understand road map of the goal the collaborative hopes to work toward and the strategies they will use to get there.
Programs addressing systemic issues: Where understanding the broader context and influencing factors is critical.
Advocacy or policy-oriented programs: To articulate the intended impact on broader systems.
How they help define focus: ToCs encourage deeper reflection on the mechanisms of change or root causes behind the community need a program, organization, or collaborative hopes to address. This allows you to define evaluation questions that explore the validity of your assumptions around the scope and scale of the community issue itself and the effectiveness of interventions that may be less defined and more complex.
3. Strategic Impact Maps
What they are: Strategic impact maps are visual tools that help organizations articulate their long-term vision and how their programs contribute to achieving that vision. They often connect program-level outcomes to broader organizational or societal goals, emphasizing the strategic contribution of individual initiatives. They might incorporate elements of both logic models and theories of change but are typically presented in a way that directly connects outcomes to mission and vision.
When to use them: Strategic impact maps are most valuable for:
Organizational-level evaluations: To understand how multiple programs contribute to overarching strategic objectives.
Portfolio evaluations: To assess the collective impact of a group of related programs or funding streams.
Communicating impact to high-level stakeholders: As they provide a clear, concise overview of strategic alignment.
Decision-making about program prioritization and resource allocation: By showing which programs are most aligned with strategic goals.
How they help define focus: Strategic impact maps help you define an evaluation focus that aligns with your organization's highest-level goals. They can help you prioritize evaluations of programs that are most critical to achieving strategic impact, or to evaluate the cumulative impact of several programs working towards a shared vision.
Defining your evaluation focus is not just a preliminary step; it's a strategic imperative. By investing time upfront to clearly articulate what you want to learn, you set your evaluation up for success – ensuring it's efficient, insightful, and ultimately, a powerful tool for program and organizational improvement and informed decision-making. Whether you choose a logic model, a theory of change, or a strategic impact map, the key is to sharpen your lens and focus on what truly matters.