By Dan Tsin, Director of Evaluation, Urban Alliance
Unemployment rate. College drop-out rate. Program retention and attrition. It's easy to get lost in the expanding world of numbers and figures. At Urban Alliance, we make sure to focus on the right data so our youth don't end up being just another statistic.
When discussing the work of nonprofit organizations, the closing question (right before "What's for lunch?") is usually, "How do you know it works?" It's a fair question, especially given the competitive nature of grants and fundraising. But aside from the responsibility to our funders, we've strongly committed to our students. We're constantly looking to improve our program, ensuring more and more young people stay on the path toward self-sufficiency. Here's how we do it:
1.Choose the Right Metrics. Ultimately, we know how a jobs-training program is successful: Its participants have secure employment, earning wages high enough to support themselves and their families. The problem with this outcome is that it won't come to fruition for many years. To remedy this, UA creates short-term proxies to identify program success and improvement areas that can be evaluated and acted upon right away.
To start, each UA intern receives ratings by their worksite supervisor on 20 unique job skills. These assessments are performed quarterly and allow the program staff to evaluate not only the young person's growth on a specific skill, but also the variety of skills the Intern is exposed to. This perpetual data collection and appraisal allows for mid-program course-correction that's both timely and youth-specific.
In addition, we track each intern on the completeness of his post-secondary plans -- a checklist of indicators used as a harbinger of future success. Here, we verify completion of college application staples like the FAFSA and the personal statement. Not college bound? We confirm resumes, cover letters and formal interview training. Our program doesn't end with job skills -- it's where you take those skills that matters.
2.Look Beyond the Stats. Data junkies hate admitting it, but even the most complicated spreadsheets won't have the answer to every question. That's why we look for feedback beyond the metrics. Through surveys, interviews and focus groups, our aim is to capture the views and reactions from all stakeholders and understand the "why" behind the numbers. Here's one example:
Last year, UA engaged the National Student Clearinghouse, a group that specializes in college enrollment verification. We looked into our most recent graduating class and were surprised by what we found. Students who completed our program and were set to enroll in college as late as July and August after their senior of high school never enrolled. This was a big revelation, albeit a disappointing one. So we called all the former interns to find out what happened. Our stats told us we had a high college acceptance rate, but it was during this post-program interview that we learned about the break-downs. It turned out that some of our Interns struggled with financing their education and wanted to learn about different pathways. We took that lesson and adjusted the curriculum to highlight scholarships and college alternatives. That leads us to...
3.Learn From Your Youth. At Urban Alliance, we believe in our program. But we know there's more work to do. With each successive cohort of interns comes new information to process. Forward-thinking nonprofits embrace results-driven growth and change. Systems and methods to track and collect data points are important to develop, but it's the willingness to use their results that will lead to improved outcomes.
What data does your organization collect? What makes your organization successful?
To contribute to Urban Alliance through the JobRaising Challenge, please visit our CrowdRise page here.