This evaluation report describes the findings from a multi-year evaluation of the Youth Service Professionals’ Knowledge, Skills, and Abilities Professional Development Demonstration and Evaluation Project (YSP/KSA Demo). NCWD/Youth conducted the YSP/KSA Demo to demonstrate the value of and further grow a professional development system for youth service professionals to improve program quality and service delivery for all youth, including youth with disabilities.
The objectives of the YSP/KSA Demo were:
Following a competitive nationwide application process, NCWD/Youth selected three sites to participate in the YSP/KSA Demo. The three sites are Baltimore City Mayor’s Office of Employment Development (MOED), Youth Services Division – Baltimore, MD; City of Los Angeles Economic and Workforce Development Department – Los Angeles, CA; and New York State Department of Labor – Albany, NY. Each cross-system site consisted of at least three youth serving agencies that were working together through the demonstration project to build staff capacity and improve youth outcomes within their geographic region. As part of the demonstration, each site had to:
NCWD/Youth contracted an independent third party evaluator, the Systems Improvement Group, Institute on Community Integration at the University of Minnesota, to conduct an external evaluation of the YSP/KSA Demo. The evaluation was designed to examine the effects of professional development on youth service professionals’ practices, organizational culture, and outcomes for youth. Based on Kirkpatrick’s Four Level Evaluation Model, the evaluators collected and analyzed data for each of four levels – 1) Reaction; 2) Learning; 3) Behavior; and 4) Results. Participants’ reaction and learning (Kirkpatrick Levels one and two) were evaluated during, and at the conclusion of, the training delivery (a short recap of these results is included in this executive summary, complete results can be found in the interim evaluation report, “Pre-Post Survey Results from Site Trainings”). At six-month and 12-month intervals post training delivery, data was collected from training participants and site administrators regarding the behavior and results (Kirkpatrick Levels three and four). In broad terms, Levels one and two examine participants’ acquisition of skills, while Levels three and four examine participants’ application of those skills.