Specify Outcomes and Have Indicators and measures in Place

To develop a strategic map—a key to partnership success—you must address three areas. First, you need to understand and apply five essential evaluation elements. Second, you have to recognize the potential pitfalls in setting up evaluation. Third, with the terminology in hand and the potential pitfalls in mind, you have to create the strategic map itself.

Five Essential Evaluation Elements

To effectively focus on evaluation and create a strategic map that leads to managing growth and demonstrating performance, you have to understand and apply five terms: impacts, outcomes, outputs, inputs and indicators.

Impacts—Overall Benefits to the Community and/or Population

The impact statement provides a picture of significant social, economic, environmental and/or political change from the situation desired in the community and/or in a population. The results statement focuses on benefits and is comprehensive and long-term.

Impacts answer the question: “What are we trying to achieve overall?”

Impacts have no direct measurements. They are a broad umbrella under which everything else is gathered.

Outcomes—Changes in a Particular Population or Segment of the Community.

Outcome statements describe the specific changes desired in the population or situation. Each outcome statement is directly related to the desired impact, is more specific, and represents one of many possible benefits that would help achieve the desired impact over time.

Outcomes answer the question: “What difference will we make?”

Outcomes are measured by changes in knowledge, attitudes, skills, behaviors, conditions or status. Changes measured can be short-term or long-term, and are directly accomplished or clearly influenced by the effort.

Outputs—Programs, Services, Initiatives or Products developed, Enhanced or Coordinated

Output statements describe the programs, services, initiatives and/or products that will be implemented or modified to accomplish the outcomes. Outputs may affect more than one of the outcomes.

Outputs answer the question: “What will we do to make a difference?”

Outputs are measured by changes in the number of, participation in and quality of programs, services, initiatives or products.

Inputs—Improvement in Partnership Infrastructure.

Input statements describe the resources needed to produce the outputs. Inputs include staff, volunteers, skills, time, money, materials, equipment, facilities, planning, activities, meetings, trainings, policies and reports.

Inputs answer the question: “What resources do we need?”

Inputs are measured by factors such as the people involved, time given, materials assembled, plans completed and money donated.

INDICATORS—Measures and Descriptions

Indicator statements declare specific changes to be measured or described that show that the outcomes are being accomplished. Indicators can also be used to measure or describe changes in outputs and inputs.

Indicators answer the question: “How will we know we are making a difference?”

Indicators vary. Short-term indicators are within the control of the partnership. Long-term indicators, however, may not be in direct control, but should be able to be influenced by the partnership. To establish indicators, a baseline of current conditions is needed through measurements or descriptions.

Common Evaluation Pitfalls

Here are common evaluation pitfalls to keep in mind as you frame your decisions through outcomes, indicators, outputs and inputs:

• neglecting to establish baseline data to know if change occurred before or after action is taken;
• defaulting to readily available information instead of seeking targeted and timely baseline and progress data;
• ignoring that results are dependent on the commitment of individuals and their organizations to make a difference and fulfill self-interests;
• failing to include the perspectives of diverse stakeholders; 
• planning activities so that integrating evaluation becomes a separate activity to be added later, if at all;
• accepting unclear relationships between desired outcomes and the activities chosen to achieve those results;
• selecting measurements that are not relevant to the activities and desired results;
• choosing indicators that are so abstract they are not measurable or easily documented;
• believing that the ability to bring about results lies within the partnership alone and doesn’t involve other key constituencies;
• attempting to attribute actions taken to results achieved, despite many causal factors; 
• demanding to be comprehensive when not everything can be measured; and
• using measures to find fault and blame rather than to learn and self-correct.

Creating the Strategic Map

Once you are familiar with the five essential elements of evaluation, and the pitfalls, you need to create a strategic map that specifies outcomes and has indicators and measures in place. Consider the following steps in developing your map:

Step 1. Identify the best people to determine the desired outcomes. Consider including those who have diverse points of view about why the change is important and those from the stakeholder groups most affected.

Step 2. Determine each person’s most effective role: creator of the outcomes and indicators; provider of advice or secondary input; approver of the final product. Ensure that all participants understand the scope of the desired changes and the organizational, technical, cultural and personal requirements to make them happen.

Step 3. Gather baseline data about older adults in the community. Decide where trends are headed and where you want them to go. Learn what is already in place to deal with these conditions.

Step 4: Create drafts of possible outcomes, stated in terms of making a difference in the lives of older adults and/or their caregivers and/or the community. Have your desired impact statement, even in draft form, in front of you all the time.

Step 5: Review each possible outcome for feasibility:

• Desire: The outcome might be good for older adults, but what do your members have a desire to do? Passion is a great determinate of what will be accomplished.
• Infrastructure: Are the partner’s behaviors and mindsets supportive of your collaboration’s policies and procedures?
• Support: Is there support outside your immediate partnership?
• Timeliness: Addressing a strong need in the community will make it much easier to make a difference.
• Resources: The more resources you have readily available, the easier it will be to achieve the outcome.
• History: Have outcomes similar to this been achieved locally and/or does your partnership or its member agencies have a history of success in the community? Success breeds success.
• Politics: Are the policymakers, funders and other key stakeholders in the community likely to be supportive?

Step 6: Review each possible outcome for potential results:

• Achievement of the desired impact statement: A successful outcome could be seen as a contributing factor to the overall impact.
• Significant demonstration of change: The greater the change in the lives of older adults and/or their caregivers and/or the larger community, the more likely partner energy will be sustained and outside supporters will contribute.
• Communication power: Does the outcome statement have intrinsic merit; does it speak for itself without a lot of explanation. This will make it easier to understand, achieve results and secure support.
• Return on investment: Will the partners see payoff? Will the self-interests within their own organizations or constituencies be met in the form of dollars, recognition and political influence? 
• Success or failure: Beyond making a difference for older adults and the community, how will undertaking this change effort strengthen (or weaken) your partnership? What would happen if you did not achieve your outcomes?

Step 7. Divide the possible outcomes into what you can achieve in the short-term and what is more long-term. Initially you should focus on what you can accomplish the fastest with the least resources. Smaller, shorter-term outcomes allow you to demonstrate the efficacy needed to build longer-term support.

Step 8. From the information in steps 3-7, identify the desired outcomes. Then draft your strategic map: the overall impact statement, one or more outcomes, and outputs, inputs and indicators for each outcome. Use positive descriptions and ensure they all directly relate to your impact statement.

Step 9. Determine how your strategic map will be used and by whom. Test the map and its statements with various stakeholders and refine it as best you can.

Step 10. Determine how you will continue to refine the outcomes, outputs, inputs and indicators as you proceed and who will be responsible for updating the strategic map over time.

To create the map, plan from the impact back to needed inputs. Start with desired impact and then create outcomes, outputs and inputs in that order. Include indicators. That’s how the road is built. To make a difference in the lives of older adults, work forward: develop the inputs (resources) that allow you to implement the outputs (programs, services and initiatives) that make the differences in lives (outcomes). All the while, you track growth and performance. That’s how the road builds us.