Cottage Health Evaluation Toolkit
Welcome to the Cottage Health Evaluation Toolkit
This toolkit takes you through the steps of evaluating your health program. At each step, you can download worksheets to complete the step, and refer to the Case Study for an example.
Engage Stakeholders: Involve people who are connected to or have an interest in your program.
Logic Model: Get everyone on the same page about the goals of your program and what it will take to achieve them.
Evaluation Plan: Determine what to evaluate, identify what information you need, determine the best way to collect that information.
Collect Data: Gather data to answer your evaluation questions.
Analyze & Interpret: Make sense of your data to learn and improve.
Use and Share Findings: Share findings with stakeholders and use findings to improve your program.
Framework for Program Evaluation
The toolkit adapts the CDC's Framework for Program Evaluation in Public Health as a guide for following the six steps to conduct evaluations across a wide range of health-related activities and programs. These steps are generally followed in order, but users should feel free to jump to content that is most relevant for your organization’s current needs.
Why Evaluate?
When Cottage Health began this Population Health journey in 2015, we asked ourselves, how will we know if we are succeeding? We recognize that improving the health of a population requires many players beyond Cottage and many sectors beyond health.
Evaluation as a Tool
Cottage Health is focused on evaluation as a tool for improving, not only proving. Evaluation can be used throughout the life of your program to better understand your progress in achieving goals, make needed course corrections, and communicate about your program’s impact to key stakeholders, including funders.
The health challenges we face are complex, and no single organization can be expected to have all the answers. To track and measure the success of the health system’s efforts alone would only account for a small part of what makes a community healthy.
After conducting a communitywide Listening Tour in 2016, we made a commitment to work together with our community-based partners to define what success would look like, and measure it using the same tools and language.
We also recognize that most nonprofit organizations have limited time and resources and typically staff are not evaluation experts. That’s why we created this toolkit—to help local nonprofits answer the difficult questions you have about your programs and services, such as --
- Are we reaching the most vulnerable populations?
- How can we improve?
- What difference are we making?
- How can we collaborate with other organizations to be more effective?
On this website, we provide the tools you need to put evaluation to work for your program, organization, partnership, or coalition.
Why is Evaluating Population Health Important?
What does a healthy Santa Barbara County look like and how do we get there? In our 2016 Listening Tour, we heard from community leaders that a healthy community is one where:
- Everyone has what they need to survive and have a meaningful life, including access to affordable health care, housing, and food
- Services are holistic and take into account people’s mind, body, and soul
- Organizations create networks and work across sectors to support those in crisis and ensure that no one is left behind
We have a lot of assets in Santa Barbara County, including a strong and active nonprofit community. Cottage Health believes that we can achieve this vision of a healthy community by supporting nonprofit organizations as they assess community needs, strengthen programs, adopt promising practices, and work in closer alignment with other organizations to achieve common goals.
Can My Organization Do an Evaluation on Our Own?
Individual organizations and coalitions can successfully evaluate their programs and use that knowledge to improve and demonstrate their impact. The Cottage Health Evaluation Toolkit is designed to take the mystery out of evaluation by providing a roadmap and key tools necessary to be successful.
Even with toolkit resources, you may want to hire an external evaluator. It is not realistic or even appropriate to expect nonprofit program staff to become evaluation experts. Your timeline and resources, the complexity of your program, and the type of evaluation you want to conduct will all be factors in deciding whether you have the expertise in-house or whether you need outside assistance. An external evaluator can be used on any aspect of an evaluation from start to finish.
What Principles Are Important in Evaluation?
Sometimes it can be difficult to tell if you are on the right track in designing and conducting your evaluation. The evaluation standards at the center of the framework ensure your evaluation is:
- Useful. Who will use the results? Will the evaluation provide them with the information they need in a timely manner?
- Feasible. Is the evaluation practical and realistic given the time, resources, and expertise available?
- Ethical. Have we made sure we are protecting the rights of participants and those who might be affected by the evaluation results? Is the evaluation respectful, legal, and appropriate?
- Accurate. Will the results be valid and reliable, and tell the story accurately
Learning Through a Case Study
Case Study Example
Follow the link below to download the full case study, including the completed worksheets for each evaluation step.
Download the Case Study
One of the best ways to learn how to evaluate is to see how it works through an example. We’ve created a case study to bring each of the 6 steps in evaluation to life. The fictional Health Connect program is facing the issues, challenges and conditions that are typical in community-based nonprofits.
The full case study is presented here, including the completed worksheets for each evaluation step.
Description
Three community partners—the food bank, the community center, and a community clinic—have pooled their resources to set up Health Connect, which embeds Community Health Workers (CHW) at the community center to connect low-income diabetic residents to healthy food, health care and opportunities to be physically active.
The Health Connect program leverages CHWs, trusted community members who are specially trained to be frontline health personnel. Often CHWs do not have advanced medical or healthcare degrees. Their expertise comes from their understanding of the community. By building trusting relationships and bridging cultural barriers, CHWs facilitate access to health and social services.
Meet the Program Team
The three community partners have formed the Health Connect Program Team, which will work together to plan and implement the evaluation. This team includes champions from each organization that will be implementing the program as well as the CHW that is integral to the program’s success.
What Will the CHW Do?
The Health Connect CHW will be trained in the areas of diabetes and nutrition as well as outreach, community education, informal counseling, social support, and advocacy. The CHW will meet with clients and help them to understand and manage their diabetes. They will also facilitate referrals to the community partner organizations.
Is the Program Working? Evaluating Health Connect
The community clinic was charged with evaluating the program’s impact to understand whether or not it is meeting expectations, and to implement a continuous improvement process.
