This evaluation report describes the findings from a multi-year evaluation of the Youth Service Professionals’ Knowledge, Skills, and Abilities Professional Development Demonstration and Evaluation Project (YSP/KSA Demo). NCWD/Youth conducted the YSP/KSA Demo to demonstrate the value of and further grow a professional development system for youth service professionals to improve program quality and service delivery for all youth, including youth with disabilities.

The objectives of the YSP/KSA Demo were:

• To demonstrate the components of high quality professional development in the workforce development system to provide lessons that state and local systems can replicate.
• To deliver high quality professional development training and technical assistance to youth service professionals (YSPs) across a variety of systems.
• To evaluate the effect of professional development on YSPs’ competencies and practices, as well as the organization’s policies and professional development culture.
• To increase alignment of professional development competencies of YSPs across youth systems.
• To inform federal, state, and local policy development to promote an effective foundation and infrastructure for the professional development of YSPs.

Following a competitive nationwide application process, NCWD/Youth selected three sites to participate in the YSP/KSA Demo. The three sites are Baltimore City Mayor’s Office of Employment Development (MOED), Youth Services Division – Baltimore, MD; City of Los Angeles Economic and Workforce Development Department – Los Angeles, CA; and New York State Department of Labor – Albany, NY. Each cross-system site consisted of at least three youth serving agencies that were working together through the demonstration project to build staff capacity and improve youth outcomes within their geographic region. As part of the demonstration, each site had to:

• work with cross-system partners to assess professional development needs and develop shared goals and strategies for supporting professional development.
• engage their staff in completing four full-day cross-agency trainings aligned to the professional development needs identified in the assessment.
• assign two staff to serve as cadre members in training who assist NCWD/Youth trainers with delivering the training and provide follow-up support to the trainees in-between training sessions and during the follow-up period.

NCWD/Youth contracted an independent third party evaluator, the Systems Improvement Group, Institute on Community Integration at the University of Minnesota, to conduct an external evaluation of the YSP/KSA Demo. The evaluation was designed to examine the effects of professional development on youth service professionals’ practices, organizational culture, and outcomes for youth. Based on Kirkpatrick’s Four Level Evaluation Model, the evaluators collected and analyzed data for each of four levels – 1) Reaction; 2) Learning; 3) Behavior; and 4) Results. Participants’ reaction and learning (Kirkpatrick Levels one and two) were evaluated during, and at the conclusion of, the training delivery (a short recap of these results is included in this executive summary, complete results can be found in the interim evaluation report, “Pre-Post Survey Results from Site Trainings”). At six-month and 12-month intervals post training delivery, data was collected from training participants and site administrators regarding the behavior and results (Kirkpatrick Levels three and four). In broad terms, Levels one and two examine participants’ acquisition of skills, while Levels three and four examine participants’ application of those skills.

Download the PDF.