Youth should be involved throughout the entire process in a collaborative relationship with adults. They can engage by helping think of research questions, issues, focus groups, and observation. It is key to make sure youth aren’t just saying what they do and don’t like but actually making decisions. Adults can help in this process by building their youth engagement skills through exposure to youth culture.
When entering into a new setting the best way to get results is to start with respect, and engagement in the evaluation can be a good way to do this. Key people who know the community can be good people to ask who should be part of the evaluation. People that live and have a commitment to the community know the ways something can be done better than a research team, and they want to feel empowered and part of the process to develop a meaningful outcome and understand its impact.
When a program is evidence-based, it has been through an experimental design, shown to work, and evaluated at a rigorous level. A research-based program means the incorporated content of a program is supported by current research. Model programs are listed on a website called Blueprints, shown with the best evidence of effectiveness, but qualifications can vary by website. In order to implement an evidence-based program, content must be followed to reach the same outcomes, keeping in mind that some things may need to be thoughtfully adapted to fit the audience.
Evaluation is key to knowing, especially for participants, that differences are being made in terms of program quality and outcomes. Funders want to make sure they are putting their money to good use in a sustainable program. Evidence of change being made also helps the program tell its story and get its message across.
When reporting results identify the audience and what they need to know in order to make a decision by using bullet points and putting findings within the context of similar findings. Usually a less biased person that is an advocate for the program that the audience will listen to is a better choice than the researcher who conducted the study. Some general rules when conveying results: simple is always better, pictures help clarify, and use multiple perspectives.
The goal of qualitative methods is to make meaning out of experiences and make use of that information. The types of qualitative methods should be decided by looking at unit of analysis: is it the individual, collective discourse, or a concept? Methods like an interview, focus group, or ethnography can make meaning from different levels of perspective.
Outcomes can occur at different levels; activities, participants, and system wide outcomes. Process evaluation looks at what went on in the program that is helping achieve the outcomes. Outcome evaluation looks at what is expected to change in the participants when going through the program and why.
Program quality is not just outcomes but also a point of service quality, or what’s happening within interactions. Components of good program quality can include youth engagement, supportive relationships, critical thinking, and physical and emotional safety. All of these components can serve as intermediate developmental outcomes.
These models are a blueprint for what you’re doing as a program in order to evaluate if a goal was achieved and why. They can include needs, outcomes, results, indicators, activities, and resources, which should be aligned with both short term and long term goals of the program. Logic models also help provide a framework for everyone involved to help focus on results and get the desired outcome.
This video explains the importance of a community or school review board in order to protect the people involved in research from harm and experimenters from any legal issues. The review board must know what kind of data will be collected, how information will remain confidential, and how consent will be obtained.