30
Dec
Recapping Our Top 5 Posts of 2015
in Data Sharing
Comments
Happy December! To end the year, we’d like to recap our five most popular blog posts of 2015.
One of our most-read posts of the year was the introduction to our Data With Integrity blog series. Our goal with this first phase of the series was to explore what the term “data with integrity” might actually mean in non-profit and community settings. Our hope was to gather stories that would “detail the guts and messiness of real, on-the-ground implementation and evaluation,” and also describe “small wins” in bringing integrity to data practices in these settings.
We were lucky to receive many contributions to the series, three of which landed among our top five posts of the year.
In a widely-read post on “The Challenges of Achieving High Quality Data in Nonprofit Agencies,” Laura Beals of Jewish Family and Children’s Services gave four examples of these challenges: missing data, different interpretations of what should go in specific data fields, inaccurate reports, and low response rates. Laura recommended focusing data collection efforts on a few high-priority data fields to make best use of limited resources. When JF&CS focused on quality of data over quantity, they were able to create and implement new tools and processes that ensured that the data they did collect were accurate and complete.
In another of our most-read posts, Jen Appleyard addressed another common problem – too much data, and no sense of how they fit together into a larger, more meaningful story. She described how her organization, which provides data services to schools, used a process called “characterization” to identify and test indicators as elements of a larger theory of change. They started with the big picture, then drilled down to specific measures. In doing so, they found potential links between short-term measures (e.g., educators’ confidence in a planning tool their organization provided) and long-term outcomes (e.g., performance gains in academic outcomes). This process helped Jen and her colleagues identify promising avenues for program improvement while also generating new research questions that strategically built on the findings of previous explorations.
Our last guest post that made the top 5 was from Jennifer Brown Urban of Montclair State University and Monica Hargraves of Cornell University. They introduced the Systems Evaluation Protocol (SEP) – a research-based and practice-tested approach to program evaluation. The SEP takes principles from systems thinking (e.g., the idea that programs exist in the context of systems that are dynamic and changing) and also highlights the importance of listening to and incorporating front-line practitioner perspectives in crafting a valid theory of change to serve as the basis for evaluation. We particularly appreciate Prof. Urban’s and Dr. Hargraves’s contributions as academic researchers who are committed to partnerships with practitioners.
Rounding out our five most-read posts of 2015 was an excerpt from our new case study, The Quad Cities: Making Shared Data a Reality. This case study provides real-life context and guidance to communities seeking to share data to achieve collective impact by describing the ins and outs of a multi-year effort between a local United Way, multiple school districts, and community-based organizations to build the relationships and strategy needed to sustain a shared data system.
While each of these posts reflects different perspectives on the idea of data with integrity, a common theme is that the foundation for gathering high-quality, meaningful data in nonprofit and community settings is a valid, focused program framework, such as a theory of change. In turn, the strength of that framework depends on how well it incorporates the perspectives and mental models of people across all levels of the system, ranging from those on the front-lines of service delivery and data collection to organizational executives and community leaders.
Thank you to all our guest contributors to the Data With Integrity series this year. We look forward to exploring these ideas further in the next stage of our work!
One of our most-read posts of the year was the introduction to our Data With Integrity blog series. Our goal with this first phase of the series was to explore what the term “data with integrity” might actually mean in non-profit and community settings. Our hope was to gather stories that would “detail the guts and messiness of real, on-the-ground implementation and evaluation,” and also describe “small wins” in bringing integrity to data practices in these settings.
We were lucky to receive many contributions to the series, three of which landed among our top five posts of the year.
In a widely-read post on “The Challenges of Achieving High Quality Data in Nonprofit Agencies,” Laura Beals of Jewish Family and Children’s Services gave four examples of these challenges: missing data, different interpretations of what should go in specific data fields, inaccurate reports, and low response rates. Laura recommended focusing data collection efforts on a few high-priority data fields to make best use of limited resources. When JF&CS focused on quality of data over quantity, they were able to create and implement new tools and processes that ensured that the data they did collect were accurate and complete.
In another of our most-read posts, Jen Appleyard addressed another common problem – too much data, and no sense of how they fit together into a larger, more meaningful story. She described how her organization, which provides data services to schools, used a process called “characterization” to identify and test indicators as elements of a larger theory of change. They started with the big picture, then drilled down to specific measures. In doing so, they found potential links between short-term measures (e.g., educators’ confidence in a planning tool their organization provided) and long-term outcomes (e.g., performance gains in academic outcomes). This process helped Jen and her colleagues identify promising avenues for program improvement while also generating new research questions that strategically built on the findings of previous explorations.
Our last guest post that made the top 5 was from Jennifer Brown Urban of Montclair State University and Monica Hargraves of Cornell University. They introduced the Systems Evaluation Protocol (SEP) – a research-based and practice-tested approach to program evaluation. The SEP takes principles from systems thinking (e.g., the idea that programs exist in the context of systems that are dynamic and changing) and also highlights the importance of listening to and incorporating front-line practitioner perspectives in crafting a valid theory of change to serve as the basis for evaluation. We particularly appreciate Prof. Urban’s and Dr. Hargraves’s contributions as academic researchers who are committed to partnerships with practitioners.
Rounding out our five most-read posts of 2015 was an excerpt from our new case study, The Quad Cities: Making Shared Data a Reality. This case study provides real-life context and guidance to communities seeking to share data to achieve collective impact by describing the ins and outs of a multi-year effort between a local United Way, multiple school districts, and community-based organizations to build the relationships and strategy needed to sustain a shared data system.
While each of these posts reflects different perspectives on the idea of data with integrity, a common theme is that the foundation for gathering high-quality, meaningful data in nonprofit and community settings is a valid, focused program framework, such as a theory of change. In turn, the strength of that framework depends on how well it incorporates the perspectives and mental models of people across all levels of the system, ranging from those on the front-lines of service delivery and data collection to organizational executives and community leaders.
Thank you to all our guest contributors to the Data With Integrity series this year. We look forward to exploring these ideas further in the next stage of our work!
