At UTEC (United Teen Equality Center), we work hard to ensure that our data has integrity, which requires that it be accurate, reliable, timely and useful. The essential question for us is “How do we get that data?” Obtaining and documenting good data is particularly difficult at a place like UTEC where we serve youth who often find themselves in crisis or facing barriers to success, which leaves staff little time and energy for data!
We’ve found that buy-in from the direct service staff who enter the data is crucial to getting good data. If staff understand the value of data, they are likely to ensure the data are right.
In particular, we’ve found four strategies are important to staff-buy-in.
Start with the end in mind.
The possibilities of what to collect are infinite. We’ve learned to distinguish between nice to know and need to know. We assess data collection through the lens of how the data relates to our theory of change, and how it will help us better serve our youth. This helps staff understand why we collect the data.For example, there are countless risk factors that we could track at intake, such as family functioning or income level. However, to make the intake form manageable, we focus on data related to our target population, particularly their school dropout, employment, and pregnant/parenting status, as well as their criminal and gang involvement. We also collect limited data on other risk factors that directly affect youth’s ability to participate in the program, such as mental health and substance abuse issues.
Cultivate data champions and curiosity about data.
Line staff’s own peers offer the most compelling reasons to collect and analyze data. They convey to one another how they use data and the benefits they see. We encourage staff’s help in informing our data strategy. This is done through a group we call The Inquirers, a team of direct service staff helping to create a learning agenda for UTEC. We build on staff ideas and strategies related to collecting and using data.For example, one of our case managers developed an “awesome board,” a dry erase board above his desk that tracks youth progress across key milestones. All case managers adopted this, and we celebrate youth who “complete” this board at a monthly house meal. See photo of an “awesome board” below. Note that names have been hidden to protect confidentiality.
Conduct regular data review at all levels.
Staff review and discuss reports, at varying frequencies, with data tailored to their needs. In particular, UTEC’s leadership team reviews dashboard data comparing actual data against a set of quarterly and annual targets to identify red flags and inform course corrections. In addition, line staff review youth progress reports to identify additional needed supports. Youth also see their own data through formal “youth check-ins,” where they meet with staff to discuss their progress.
Integrate data into the culture and physical space of the organization.
UTEC aims to foster a culture that reflects our shared values–including the value that we place on data—and to integrate those values into our surroundings. As part of our “Outcomes Transparency and Questioning Campaign,” we display data around our building to highlight successes and show progress. For example, our public café highlights our youth outcomes. See photo below.
As part of an ongoing process of learning and improvement, we share two continuing challenges:
1. Making changes to systems while still maintaining buy-in.
We make changes to our data systems based on our program model and staff feedback. However, changes require retraining staff, which can create staff frustration in having to relearn systems. We’re working to minimize these disruptions by making major changes annually, as opposed to monthly or quarterly. But this means that systems aren’t as efficient as they could be in the meantime.
2. Simplifying data entry while still getting data that is detailed enough to be meaningful.
We constantly seek ways to simplify data processes. We scrutinize every data point and ask, “Why do we want to know this?” and “What will we do with this information?” However, there’s still a lot that’s important to collect. Our challenge becomes making data collection manageable for busy staff. We’ve addressed this by providing staff with tablets to enter data, since staff are often on-the-go.We also work to make data accessible through staff-customized dashboards. We assess the manageability of our data processes based on: (a) whether the amount of time it takes to enter the data is reasonable, (b) how easy it is to access needed data in a timely way, and (c) the degree of flexibility in our data systems to accommodate changes in programming.
As we’ve learned, getting and keeping staff buy-in for data requires constant attention. The work to achieve data with integrity is never done: there’s always room for improvement! Special thanks to Jephine Ajwala for help writing this blog post. Jephine is a Data and Evaluation Analyst at UTEC and is passionate about developing evidence-based strategies aimed at improving the circumstances of marginalized populations.
If you are interested in learning more about UTEC, visit our website. You can also sign up for our mailing lists.
The preceding is a cross post from Markets for Good. To view the original article, click here. Many thanks to Erin and Jephine for sharing such creative and useful examples of how to create an organizational culture that supports data with integrity! Feel free to share your thoughts in the comments below, and to share this article on social media using #dataintegrity.