Outcome Harvesting for Wikipedia

What happens when the sum of all human knowledge is freely available in the world?

Wikimedia Foundation (Wikipedia) has been asking the question "What is the impact of grants?" for a long time. There have been a number of efforts to answer this question, focused primarily on short-term outputs. But these efforts miss the longer-term outcomes, the ripple effects of grantee work that are hard (and sometimes impossible) to measure in the short grant life-cycle. Calibrate Research Group's unique approach to outcome harvesting has helped the foundation capture the long-term impact of the world's greatest collective experience.

Check out the report here.

What we Do

Calibrate Research Group comprises of academics, professionals, and practitioners in the fields of international development, public health, evaluation and social work.  We’ve all worked for non-profits and know the value of measuring impact.  Instead of being plagued with questions regarding whether your non-profit is making a difference, we work with you to analyze your impact through our unique data-drive, mixed-methods and beneficiary-focused approach.  We enable our clients to tell a better story of impact in order to share with beneficiaries, employees and donors alike. 

Past projects have included work in a range of thematic areas including, but not limited to, trauma healing in the great lakes region of Africa, program effectiveness, HIV/AIDS initiatives, and working among oral cultures.

3 Lessons on Data

A decade of working with data in international development has taught me three key lessons:

Data without a strategy is dead. It becomes meaningless globs of numbers and stories easily blown about in a westerly wind. Data collection is very in-vogue right now.  According to IBM, about 2.5 quintillion bytes of data are created every day, we love measuring everything these days, even how many times we blink in one day. Unless there is a foundation of strategy that guides the collection, analysis and application of the data it becomes an unnecessary burden on partners, and employees alike. 

Data is not objective. There is a term in data science called veracity which measures the biases, noise, abnormality, and reliability in data-sets. John Giannandrea, Google’s AI Chief said before a recent conference that: “The real safety question, if you want to call it that, is that if we give these systems biased data, they will be biased.” Within international development, we need to identify the bias, communicate the bias, and prevent the bias through on-going feedback loops and asking key questions of how our data processes may be perpetuating bias.  

Data needs to be shared. The international development community needs a large data bank on complex issues such as human trafficking, global hunger, and poverty. The data bank would have the capacity to hold multiple different data types along with metadata that describes the datasets. Data is often siloed away, becoming outdated and dusty.  

Behaviorial Assumptions of Capacity Building Interventions

Capacity building is a buzzword and a touted solve-all for non-profit organizational and programmatic woes.  After reading a journal article on the behavioral assumptions of policy tools by Anne Schneider and Helen Ingram, I wondered if such a framework has been developed to track the behavioral assumptions of capacity building interventions.  The assumption I saw played out time and again working in East and West Africa with indigenous non-profits was one of knowledge access.  If non-profits have access to knowledge regarding (fill in the blank) budgeting, monitoring and evaluation, marketing, etc they will act differently, or so the theory goes.  Research and experience has shown that access is but one part of the puzzle when approaching capacity building efforts. What has been your experience with behavioral assumptions around capacity building interventions?  

Spasms of Passion

"The victims of injustice in our world do not need our spasms of passion; they need our long obedience in the same direction." - Gary Haugen

Gary Haugen’s quote causes great pause and reflection for me. I’ve worked under several large humanitarian grants desiring to address the issues of injustice, and yet each one was structured to fund spasms of passion, versus creating systems which would allow for a long-term focused approach. Unfortunately, philanthropic efforts are just as susceptible to passionate solve-all simplistic approaches to solving some of the dire issues of injustice, from poverty to human rights, to issues of national security. What if, as a community of practitioners, we stopped having knee-jerk responses and instead sought to listen, learn, and co-labor with indigenous organizations? What if, as a community of practitioners, we set metrics which allow for learning and failure? What if, as members of the human race, we moved away from falling into an emotive spiral and took courage to acknowledge pain and suffering and knowing that with listening, learning and co-laboring we can be obedient to addressing the issues of injustice for the long-term.