Thoughts on GIIN's "Impact Measurement (IM) and Performance Management Training"

At Upaya, the process of collecting and assessing beneficiary data to track outcomes and measure impact is fundamental to our work. After all, four years ago the organization was founded with a hypothesis that providing jobs -- and not handouts -- is the most efficient and effective way for the “ultra poor” to progress out of poverty. To date, we have sought to prove or disprove this hypothesis through social performance measurement (what we internally refer to as “SPM”), the systematic collection and analysis of beneficiary level information.

I recently had the wonderful opportunity of attending the “Impact Measurement (IM) and Performance Management Training” in Bangalore organized by the Global Impact Investing Network (GIIN) in collaboration with international consultancy Steward Redqueen and Social Value International, to reflect on Upaya’s methodology and revisit some of these ideas. The purpose of the training was to introduce practitioners to different IM frameworks and ways in which these frameworks could be used to suit the context of an individual organization and its mission.

Some of my key takeaways from the training were:

• Don’t overstress the rigour; it should be “good enough”
• Data is not just for reporting but also for decision making
• Consider establishing a counterfactual in the setting

Financial and business decisions are made everyday on the basis of imperfect information provided by financial accounting frameworks. Yet when it comes to evaluating impact, we tend to hold ourselves to a higher standard or rigour --where randomized control trials are considered to be the gold standard. However, sometimes the fear of publishing imperfect impact data gets in the way of doing any IM activity. If the data is ”good enough” to spot trends, aid in decision-making and undertake some course correction, and not overly tax human or financial resources, then it is worth undertaking.

Secondly, these days it appears that impact data is mostly used for reporting outwardly to stakeholders, such as philanthropists and investors. In doing this, we overlook the critical role data can play in helping an entrepreneur and/or management team make decisions about the business itself. The challenge most often cited among practitioners is one of constrained resources - the buy-in needed to undertake the exercise of data collection and the need for doing it all. For an early stage enterprise that is working to expand the business, build the team, formulate marketing strategy, and raise funding, IM is a weighty commitment.

Data collected through impact measurement, however, should not be seen separate and distinct from business data. Instead, if an IM framework were designed to provide useful insights for the business about its beneficiaries, that could then inform refinements to the core product or service, then there exists a higher probability of entrepreneur’s buy-in and also better quality data.

Last, but certainly not least, is the need to accommodate the counterfactual in the actual impact assessment. A counterfactual simply put means “what would have happened anyways, without the intervention in question.”  A counterfactual could be in the form of state and national averages based on government data. Or it could be a more statistically rigorous “control group” or “non- intervention group.” The counterfactual, when compared with data that depicts the outcomes of the intervention, can give us a clearer indication of impact, or what positive effects can be attributed to the intervention. In essence, it can provide the necessary context and help us paint a picture of the impact that has occurred.

In the coming months, we will integrate some of these learnings into our SPM framework. We hope to make our system of impact assessment more robust so as to provide high quality information on the progress made out of poverty by our beneficiaries. Our goal is to simultaneously provide valuable insights and learnings to our entrepreneurs, to support the continued growth of their businesses and the creation of hundreds more jobs.

Four interesting takeaways from the GIIN/ Dalberg “Landscape for Impact Investing in South Asia” report

On 18 December 2014, the Global Impact Investing Network (GIIN), in partnership with Dalberg Global Development Advisors, released a report that provides a “state of the market” landscape analysis of the impact investing industry in South Asia. The Landscape for Impact Investing in South Asia looks at the $8.9 billion in deployed impact investment capital in six countries – India, Bangladesh, Pakistan, Myanmar, Nepal, and Sri Lanka – and paints a picture of a “diverse but growing impact investing market across South Asia.”

As the report has circulated amongst the Upaya team, four main points have jumped off the page. Some of them mirror observations we’ve seen through our own experience investing in India, and others shed light on issues we are wrestling with. In no particular order:

Two of the top five areas for impact investment in South Asia – Manufacturing and Agriculture/ Food Processing - are directly contributing to livelihood enhancement. That said, they are still only 17% of the identified market.

Broadly speaking, manufacturing and agri-processing are two broad areas under which our team has focused its livelihood development efforts and we are excited to see them crack the top five areas for impact investment. 

However, the fact that the two segments combined are still a smaller percentage of the impact investment market than each of the top two categories - Financial Services and Energy – shows that we still need to expand the conversation about job creation and its social benefits in the impact investment community.  

“There is also a need to bring less-exposed enterprises into the fold in a number of countries. Even in India, where formal networks of entrepreneurs exist, it is difficult to find enterprises that are not part of these networks.”

This point really hits the heart of one of the biggest challenges in the impact investment space – pipeline. Right now, too many impact funds are only looking at the businesses that self-identify as social enterprises, and are only doing due diligence in that limited pool. The result is a sort of “social venture ghetto” where a subset of entrepreneurs are continually showcased together at business school competitions and conference panels, thus creating the impression that they represent the full scope of social ventures in the space.

Not coincidentally, the investors who have been successful are the ones who have not limited their purview to that ghetto. For Upaya, the majority of entrepreneurs we support do not necessarily self-identify as social enterprises, but simply as businesses operating in poor communities. Friend of Upaya Artha Initiative has taken a similar view of the issue with their Artha Venture Challenge, a competition that has uncovered several great companies outside of the mainstream social enterprise conversation. In both cases, Upaya and Artha have had success in finding the types of investment opportunities that were sitting outside the standard impact investor conversation but are having a positive impact through their work.

“[In India] funds are shifting toward a less opportunistic and more hypothesis-driven approach to selection; in this new approach, these funds start with the identification of a problem in a given sector, then identify a potential solution (hypothesis), and subsequently seek organizations that contribute to this solution.”

Among the team we’ve long been wary of the proliferation of impact investing funds whose portfolio companies are united only by a broad notion of “positive impact” rather than a specific type of change they are working toward.  Our concern is that, without a unifying objective, funds will scatter investments across a variety of issue areas and miss the opportunity to aim significant resources at a specific problem.

Of course, Upaya has developed its own hypothesis – support Small & Growing Businesses that can be large scale employers in ultra poor communities – and are pleased to see that others are starting to bring their own theses into sharp relief. I would certainly point to our friends at Omnivore Partners as a great example of what can happen when a fund pursues clear and measurable outcomes in a specific area (in their case, agricultural supply chains).

In India, “foreign funds are prohibited from investing in debt and, as a result, most of the capital from [foreign] impact funds is deployed through equity instruments. Consequently, small domestic funds are emerging to fulfill the need for early stage debt.”

Accessing affordable working capital debts is a continual challenge for many SGBs in India, including some of Upaya’s partners at various points in their early lifecycle.

For much of the past year, our team has worked with domestic lenders to find creative and effective working capital solutions for our partners. What they are now coming to see that, while smaller domestic lenders are playing a role, these funds still have a big gap to bridge if they are to fulfill the credit needs of SGBs. It is an issue that Sreejith, Tanya, and the team are working hard on, and we are all glad to see this observation in the report.