Appendix IV: Needs assessment analysis procedure

Analyzing Needs Assessment Data

AFC needs assessment data collected at a city-wide scale can ignore the needs of individual households, streets or even neighbourhoods. Determining an appropriate ‘scale’ of analysis is critically important. Choosing an analysis scale will always involve some uncertainty, but never make a final choice arbitrarily or for reasons of mere practicality. Data collected through a needs assessment is susceptible to bias (for example, what makes up ‘a neighbourhood’?), but there is no reason that you cannot analyze the supplementary data sources in Appendix I at multiple scales. You can create a socio-demographic community profile for the city as a whole and for individual neighbourhoods using census tract data from Statistics Canada. By examining this data at both the larger and smaller scales, your analysis can:

1. Create a ‘city-scale’ profile: Directly relate the data from your needs assessment to an entire city. This allows you to assess the socio-demographic information on certain issues. It also allows you to predict a trend for these issues resulting from the future demographics of your community. Your needs assessment may find, for example, that current social programming is not meeting the expectations of older adults, but you may find that in light of a changing population, planning and funding future programming should factor in cultural diversity.
2. Create a ‘neighbourhood-scale’ profile: Neighbourhood-specific data can help you identify high-priority areas within your community and place broader issues in context. Your needs assessment may find, for example, that older adults are looking for information about home modification programs, but the broader ‘city-scale’ socio-demographic data suggests that the distribution of information packages requires a unique approach, because older homeowners are clustered in only a few neighbourhoods.

Most AFC and QoL instruments require you to use a specific set of questions and responses, allowing every community to collect and analyze data following a more or less standard procedure. In some cases, these instruments require a third party to analyze the data following a proprietary procedure that often involves some cost to the end user (for example, AdvantAge). These proprietary tools do provide certain benefits to the user: standardization and less direct analytical work. They have limitations, however. While getting answers at the end of an evaluation is important, the evaluation process itself can often be just as useful. This guide takes the approach that the partnerships, insights and shared understanding you develop through a bottom-up assessment procedure are indispensable drivers for change. You can only realize these partnerships, insights and understandings when local stakeholders explore local issues.

This guide allows you to create a customized instrument. As a result, it cannot also provide a standardized process for analyzing your data. However, data you collect under the p-e fit framework can identify gaps in your community infrastructure with a simple descriptive analysis. Below you will find a list of detailed recommendations to help you organize, generalize, synthesize and interpret the information you have collected. In analyzing your AFC data, you will discover the ‘issue agenda’ that will be the basis for your AFC action plan — keep this goal in mind as you work through your analysis. Although it is too early to know exactly what your action plan will look like, you should develop a general sense of what it will contain and how to structure it. Knowing these basic aspects will ensure your analysis approach does not hinder you from completing your plan.

Organizing the Data

1. Create a spreadsheet for your survey data and transfer each question and its responses into it. To determine the appropriate structure for your database, which will make finding useful information easier, know the general structure of your action plan. If you are basing your action plan around the AFC dimensions used in your needs assessment, we recommend you store each question as a row in your spreadsheet. This makes each column a different survey respondent. (Note: Because it is more common to structure an action plan around AFC dimensions, the remaining analysis recommendations assume this approach). If you are basing your action plan around specific groups of the population, you will want to do the exact opposite.
2. To quickly sort the information you gathered into the AFC dimensions, add a new column and give each question a numeric code based on the dimension into which it fits. Open the spreadsheet with your original questions set (see Section 5) and transfer the code for each question into your new spreadsheet.
3. Using the new ‘Dimension Code’ column, you can now quickly sort your data based on the AFC dimensions. When doing this, keep all p-e question pairs together, as the goal is to find gaps between what older adults need and prefer and what the environment currently supplies. Because each dimension represents a broad topic area, it will also be helpful to group questions within each dimension into smaller clusters representing more specific concepts. This more detailed grouping will make exploring your findings more logical and much more efficient. As an example, within the Mobility dimension, group all the public transportation questions together. Please refer to the instructions on how to use the Questions Database in the video available on the University of Waterloo website.

Generalizing the Data

4. In a new column, summarize the responses for each question using the most logical metric. In many cases this will probably be the average (or ‘mean’) response for a question, or the frequency of specific responses (for example, Yes=63; No=84). Keep in mind that the methodological decisions you have made, such as the sampling method you used, affect the metric you use to summarize your data. For example, if you collected data from a non-random sample of individuals at one location, like a mall, that could skew the responses you received in one direction. This would make using a mean score questionable, as it could misrepresent what the data is actually saying. In this case, using the median may be more appropriate. Consider using the following references to help determine which metric is best suited for your data.
   • Dictionary of Statistics & Methodology: A Nontechnical Guide for the Social Sciences (Authors: W. P. Vogt & R. B. Johnson)
   • Discovering Statistics Using SAS (Authors: A. Field & J. Miles)
   • The Basics of Social Research (Author: E. Babbie)
5. Most people have a much easier time understanding data in a visual format, instead of an assortment of numbers in a spreadsheet. Translate your summarized data into a set of graphs. Because we want to compare the fit of each p-e pair, a clustered bar graph is the most logical way to visualize the data. To organize the data, try to include several p-e question pairs in each graph. Use the groupings of specific concepts above to determine what each graph should include. Please refer to the instructions on how to use the Questions Database in the video available on the University of Waterloo website.
6. Determine whether there are any subpopulations within your survey sample — for example, different age cohorts — that you want to compare, and create the appropriate graphs.
7. Analyze the responses to any open-ended questions that you included by repeatedly grouping the responses into a set of themes. Each theme should represent a holistic idea, but not share a significant conceptual overlap with any other themes. The general name for this process is ‘content analysis’ and the following resources offer a great overview of its purpose and specific techniques to follow in its use.
   • The Content Analysis Guidebook (Author: K.A. Neuendorf)^{footnote 21[21]}
   • Qualitative Data Analysis: An Introduction (Author: C. Grbich)^{footnote 22[22]}

Synthesizing the Data

8. Now that you have effectively summarized and presented data in a convenient format, explore the results. What you are looking for are gaps between older adults’ needs and what the environment supplies. If you followed the recommendations for organizing the response scales in your needs assessment, your graph will indicate a mismatch between the need and supply of a given resource. The table below summarizes the implications of the four results that could potentially occur for any p-e pair.
9. Identify the existing resources and resource deficiencies in your community and use the other data sources you have at your disposal (e.g., socio-demographic data, focus group data, open-ended questions) to provide context for the reason the gap exists and the potential consequences it might have on older adults’ QoL.
10. Use the socio-demographic information you have collected about your community to examine whether areas where a satisfactory supply (such as the right p-e fit) will remain stable as the community evolves, or whether you’ll have to act strategically to maintain this balance due to the changing context of the community.

Prioritizing the Issues

After identifying the gaps in your community’s physical and social infrastructure that present challenges to older adults’ QoL, establish which areas should be given priority in your community’s action plan. You can establish this ‘issue agenda’ in several ways — for example, asking a group of AFC experts to rank them. However, several decades of research on participatory planning indicate that a plan has a much higher likelihood of succeeding in its vision if the local population perceives it to be a true representation of its voice.

Talk to your AFC stakeholders again, addressing the principles, goals and priority dimensions that you established during focus group discussions. To help with this task, create a matrix similar to the one on the next page and use this four-step process:

1. Create a short, meaningful label for each gap you identified and add it to the first column in the matrix.
2. Use the dimension rankings from your focus groups to sort the eight dimensions into the matrix in the order of importance that you established.
3. Identify which dimension each of the gaps belongs to, for example by shading in the appropriate cell in the matrix.
4. Using the scale below, consider each gap along with the goals of your AFC initiative and determine how much you agree with the following statement: ‘Addressing this particular issue is critical to achieving one or more of the goals for this AFC initiative.’ Then add the appropriate rating to your matrix.

1 = Weakly Agree 2 = Agree 3 = Strongly Agree

Once you have completed this exercise, you will have a much better sense of what the priorities for your action plan should be. The further along the dimension scale a gap appears, and the higher you rated that issue’s importance, the more central it should be to your action plan.