Stage II: Identify Outcomes using a Logic Model

There are three parts to explain this stage of developing a performance measurement framework:

• introduction of the concept of outcomes and why they are important
• introduction of the logic model, the importance of relationships and the connections between inputs, activities, outputs and outcomes
• attribution and its effects on outcomes in a logic model

For examples of logic models used in economic development, please refer to Appendix 3: Sample Logic Models/Performance Measures.

Outcomes

What are outcomes?

Outcomes are the actual effects of activities and outputs. They are the reasons why organizations invest in and carry out initiatives. They are the desired changes on which organizations allocate resources, design activities and identify target audiences. Outcomes are written as declarative statements (i.e. jobs created) and are usually focused on the target audiences and/or external stakeholders.

Before you continue, you must first identify what success will look like. Having done this, you can then put measures on an outcome. Outcomes are the changes, benefits, or other effects that happen as a result of work done. They can include changes in:

• changes in knowledge
• changes in skills and attitudes
• improvements to the economic environment of industries or regions

As outcomes reflect the changes expected to occur as a result of the initiative or project, they should be measurable. Identify a few key performance measures to track outcomes and determine whether or not they have been reached, in whole or in part.

The level of outcome can be described based on short-term, medium-term and long-term results that could be expected from the activities and outputs of an initiative or project. As the table illustrates, the long-range outcomes are dependent on the short term, and without realizing the short-term outcomes, the resulting medium-term and long-term outcomes are not realistic.

An example of an outcome change for research-related activities might be:

• Short-term: New technology developed and shared with members
• Medium-term: Technology implemented by members
• Long-term: Increased productivity

What your organization does and what it produces can influence the outcome. A short-term outcome could be that the members gain additional knowledge as a result of the new technology being shared. As a result, in the medium-term some members who gained the knowledge chose to implement the technology. Long- term outcomes are usually the same as the goals of an initiative. For example, a long-term outcome for a technology adaptation is that members implemented the technology and as a result have shown increased productivity in their efforts.

Direct and indirect outcomes

Usually outcomes are shown to be improvements or changes for those served directly by an organization. Sometimes the work of an organization may lead to indirect outcomes for its membership, clients or the public. For example, the adoption of ideas by several producers who attended a farm innovation workshop could lead to additional change when they explain the new technologies to their neighbours, who see the benefits on the farms around them and adopt the technologies; this would be considered as an indirect outcome - one that you likely did not anticipate.

Writing outcomes

An outcome is the actual effect of activities and outputs; two questions to ask when writing outcome statements are:

• Who is impacted by the program, project or initiative?
• What will change or what will be new or different for your target audience/s?

This may include new knowledge or new practices.

The answers to these questions may help to define what your organization wants to  say. Here are some other things to consider when writing and reviewing outcome statements:

• Define what specific terms mean for the organization. For example, if you will be using the term outcome interchangeably with benefit, impact, results, or consequence, note it up front so everyone knows what it means.
• Explain exactly what is meant. Elaborate if using descriptive words such as enhanced, innovative, sustainable, efficient, effective, excellent, high quality, stronger partnership, etc. Think if, and how these outcomes can be measured’
• Write outcomes as simple declarative statements of what is expected to be seen on the ground as a result of an initiative. “Farmers adopt the newly improved varieties of wheat," and "farm gate income is increased.” are two examples of simple, declarative outcome statements.
• Use plain language and be consistent with the terms used.
• Take time to clarify each outcome statement and what it means. Failure to do this could cost time debating performance measures and create measures that do not provide the information needed. A well-crafted outcome statement should make it easier to identify a performance measure.
• When finished with writing outcomes, put them in order of priority.

The logic model

What is a logic model?

A logic model is a visual representation of the connections and relationships between inputs, activities, outputs and outcomes. It serves as a road map for an initiative. It illustrates what activities will be undertaken and the intended results using graphics and text.

The logic model is flexible, meaning that if external factors come unexpectedly into play, the logic model can be adjusted. Logic models come in different formats, but the basic information is the same as the information in the diagram below called Logic Model.

A logic model maps out the following:

• resources/inputs
• activities
• outputs
• outcomes (short, medium and long-term)

A logic model will help focus on the causal relationship between inputs, activities, outputs and outcomes. It will also help determine whether the short-term, medium-term and long-term outcomes of an initiative or project are being achieved.

This logic model shows the important role this tool can play in helping us describe the relationship between what is being done and the changes to be influenced.

The benefits and uses of a logic model

The process of developing a logic model can help you:

• Visualize what is expected of an initiative or project
• Identify the target audience of an initiative or project
• Determine what is realistic and reasonable for the resources available
• Set priorities
• Identify appropriate performance measures
• Make it easier to communicate results
• Provide a clear summary of what an initiative or project is working towards
• Determine the links between inputs, activities, outputs and outcomes

Key points to remember when creating a logic model

A logic model is not a product that one person produces on behalf of an organization. It is a multi-staged process that involves many people thinking and working together. It is important to involve those whose work is represented in the model and those who have a stake in the outcomes.

Creating a logic model is a dynamic process that may involve many revisions as ideas are tested. The information entered in the logic model at the early stages may need to be revised as new information surfaces.

A logic model is a simple one-page document. It can be used as a communication tool and should be self-explanatory. Include enough detail so it is easy to understand; further details or rationale could be placed as appendices.

Sometimes more activities and outcomes are created than what can be achieved with the resources available. Have a team meeting to identify what is most important and most likely to be achieved.

A template for creating a logic model can be found in Appendix 4: Logic Model Template.

Two approaches to creating a logic model

1. Start with the end in mind – move from outcomes to inputs

a. Decide on the outcomes to be achieved

Begin by brainstorming with the team about the outcomes to be achieved.

Write them down and then refine them so that they are clear, concise and declarative statements.

When thinking about long-term outcomes, and to some extent medium-term outcomes, make sure they are within the scope of the initiative’s influence. Although it may not be possible to draw a straight line from one to the other, you should be able to show that your initiative contributed to your long-term outcomes.

Group decision making

If you have multiple outcomes, narrow the field and measure what is important rather than measure every possible outcome. One way to do this is to hold a team meeting to determine which measures are most likely to produce the outcomes looked for.

b. Identify outputs

Once you have identified outcomes, determine what outputs are needed in order to achieve the outcomes. Remember, outputs are the products or services provided such as brochures, guides, training sessions or public meetings. Staff may refer to outputs as "deliverables".

c. Decide on activities

With outputs identified, decide what activities are needed to deliver them. Remember, activities are the behind the scenes things that need to be done to produce outputs such as designing a brochure or developing a training session.

d. Determine what resources/inputs you will need

After listing all the activities required for an initiative, identify what resources will be needed to put them in place. Resources can include volunteers, staff members, consultants, equipment and money, etc.

Questions to help your team develop a logic model

2. Ask the why (or the if…then…) questions

Another way of working through the components of a logic model (inputs, activities, etc.) is to identify the chain of events that lead to the long-term outcomes.

Do this by asking questions that begin with the word why, such as the following:

• Why invest in training?
• Why is training important?
• Why is new knowledge important? Or ask if… then what happens questions.

For example:

• If a workshop is provided, then more food processors will have access to training that is specific to their needs (outputs).
• If the workshop covers the best practices of personal hygiene for food processors, then the participants will increase their knowledge about what constitutes good hygiene and create personal hygiene plans for themselves (short-term outcome).
• If the participants create their own personal hygiene plans, then they will be more likely to put them into practice (medium-term outcomes).
• If more food processors put their personal hygiene plans into practice, then the risks associated with personal hygiene in food processing organizations will be reduced (long-term outcomes).

Building a logic model

Once your organization has written the outcome statements, listed your inputs, activities and outputs, you are ready to develop the logic model. As the logic model is being developed, outcome statements may be refined as inputs, activities and outputs are assessed.

As mentioned earlier there are different logic model formats. For example, a logic model may be a table or a diagram. A logic model may read from the bottom to the top or from left to right, as in the diagram below.

No matter the format, a logic model should clearly show the relationship between the inputs, activities, outputs and outcomes. Ensure that for every activity identified, there is an output that connects to an outcome. There may be multiple connections and your outcomes may form a chain of short-term, medium-term and long-term outcomes.

Inputs, Activities, Outputs and Outcomes - shows the relationships among inputs (resources), activities, outputs (products) and outcomes

This logic model shows the important role it can play in helping us to describe the relationship between what is being done and the changes seeking to be influenced.

Test the logic model to make sure it works

When testing the logic model consider the following criteria:

1. Does the logic model include the key components – inputs, activities, outputs and outcomes?
2. Is it relevant and representative – does it reflect the purpose of the initiative?
3. Are the connections logical - is there an “if ... then” or cause and effect relationship? 4. Are the activities, outputs and outcomes realistic, given the resources/inputs available for them?

Attribution

Attribution can be described as without the involvement of the organization(s) the outcome would most likely not have occurred as quickly, effectively, efficiently or have been as positive for the client.

There are always numerous partners involved in any initiative. It can be a significant challenge to develop performance measures that acknowledge the real contributions of all the stakeholders involved in a project.

In addition to acknowledging that “no person is an island” and that it takes a community to develop the local economy, performance measures must also take into account the fact that there are external forces over which local or regional players have no control. Things such as economic cycles, exchange rates, trade policies and global supply chains will influence the health of local economies. Organizations need to look at the level of attribution of influence when assessing the value of their different economic development activities.

Attribution and Control - highlights how organizations have a higher level of control and greater ability to link the activities of their strategic plans with the outputs and the short-term outcomes than with the long-term outcomes. It is more difficult to attribute the longer-term outcomes to their efforts because of external forces and events which are beyond the control of the organizations.

Degree of attribution

The degree of attribution or influence that an organization has had on the outcome of a plan can be classified using the simple scale of high/direct, medium/indirect, and low/little influence.

High or direct — means that the organization played a key role in the project such as providing funding or staffing support.

For example, an economic development officer made a company aware of a funding program and coached the staff in the writing of the application. The company received the funding and was able to add a new product line worth $1 million. In this case, the economic development officer could attribute 100 per cent of the $1 million investment as an outcome, as it probably would not have occurred without the officer’s help.

Medium or indirect — means that the organization played a supporting role in the project such as providing information or contacts that resulted in a project with a positive outcome for the region.

For example, an economic development officer met several times with a small business that was looking for the financing to expand. The officer connected it with the local Community Futures Development Corporation. After several meetings with the funding organization, the small business was approved for a loan and purchased a $50,000 piece of equipment. In this case, the economic development organization could attribute part of the new investment as an outcome, usually set between 33 per cent and 66 per cent. Without the officer’s connection and guidance, the business might have spent a great deal of time looking for financing and may have been unsuccessful.

Low or little influence — means that the economic development organization is aware of the project but had limited involvement with the client organization as it moved forward with its project.

For example, a company requests a copy of the market analysis that the economic development organization conducted for the downtown area. The company had no other direct interaction with the economic development organization as it was able to find the information required on the organization’s website. In this case the economic development officer could attribute a low level of influence usually set from zero to 33 per cent. Without the market analysis, the company might not have invested in the downtown area.

Another example of low level of attribution could be a company that decided to invest in the community but needed help from the economic development organization with planning approvals. In this case, the clients experience may not have been as positive without the involvement of the economic development organization.

A good rule of thumb with the level of attribution is to think about what the staff members of the company or institution that was assisted would say if they were asked directly. If unsure, take a lower level of attribution and be conservative rather than over reporting.

The logic model and attribution

In the following diagram, the inputs, activities and outputs on the left-hand side are the components that are commonly associated with some level of control.

The outcomes on the right-hand side of the diagram are the components of the logic model over which control gradually lessens but some degree of influence remains, even for the very long-term outcomes. These outcomes are the ones that are more likely to be influenced or affected by external factors over which an initiative or project has no control. External factors might include the influence of competing or supporting programs, incentives and changes in policy or government.

When developing a logic model, ask whether an organization’s outcomes are within their control or are influenced, at least in part, by the activities and outputs of the initiative. Another way of thinking about attribution is this: would the outcomes have occurred as quickly or effectively without the initiative or project?

A logic model should include long-term outcomes to capture the meaningful changes for beneficiaries and to reflect the full extent of the benefits of your initiative. Do not select outcomes beyond the influence of the initiative or project so that it is hard to measure and prove. For example, achieving change within your clientele is more realistic than claiming change nationally.

Linking outcomes to attribution

At this point, we have covered outcomes, attribution and performance measurement - how do they fit together? During the lifespan of an initiative, observe the links between the inputs, activities and outputs and its short-term and medium- term outcomes.

However, it may not be possible to easily attribute the long-term outcomes directly to the initiative or project. That does not mean that it had no effect at all. It may have had some influence on the long-term outcomes or contributed to them.

One way to make that link is by looking at the context in which the initiative or project operated and by acknowledging the other factors that contributed to the results. Long-term outcomes were probably realized, at least in part, because of the hard work of many people working with other organizations on different programs with similar outcomes.

Becoming comfortable working within a performance measurement framework will lead to discovering ways of linking or attributing the activities and outputs to the long-term outcomes. Be creative. Use narrative success stories and anecdotal evidence to make the connection between the lower-level or direct outcomes achieved and a higher-level outcome. For example, if there are people who have said that the initiative was one of the reasons, they changed a long-time agricultural practice and, if they agree to be used as an example, use their testimonial in a report.

The following questions can help quickly identify and explain to others where you are at and what you are doing in your performance measurement process. If you answer “yes” to all these questions, you are prepared to move on to the next stage.

Stage II: diagnostic questions

• Can the inputs, activities, outputs, and outcomes be clearly differentiated?
• Have clear outcomes been established and agreed upon?
• Have the key activities and resources been defined?
• Has a logic model been created to link inputs, activities, outputs, and outcomes?
• Has the logic model been explained to and agreed upon by stakeholders?

A full list of diagnostic questions from all four stages can be found in Appendix 2: diagnostic questions.