Since Upaya’s founding in 2011, impact measurement and management (IM) has been a core function of our organization. Upaya’s mission is to fight extreme poverty from the ground up by building scalable businesses and dignified jobs. To ensure our work has the desired impact, we go beyond simply counting the jobs our partner entrepreneurs create. In addition, we conduct in-depth surveys with jobholders to really understand and analyze how these jobs enable the jobholders’ progress out of extreme poverty.
Upaya’s early impact surveys were collected with pen and paper. While our methods have evolved, Upaya’s in-depth surveys are still conducted one-to-one.
Sachi Shenoy, Upaya’s Co-Founder and Chief Impact Officer, can still remember when we had successfully collected the first 100 jobholder surveys, back in 2012. We’ve come a long way since then, with more than 1,300 surveys in our database today. While collecting, cleaning, and analyzing the survey data isn’t easy, as Daphne Delaski, Upaya’s Senior Associate, described in our Upaya Fundamentals blog series, it’s crucial for measuring—and furthering—Upaya’s impact. These data points are as integral in shaping our programs today as they were on day one, 10 years ago.
What we learn from our data collection isn’t always rosy. Shrishti Puri, Upaya’s Senior Data Associate recently shared our team's findings when we applied a gender-lens to our impact data. However, remaining transparent in our work and data analysis allows us to continually learn, fine-tune our model, and share insights with the broader industry.
One of Upaya’s enumerators surveys a dairy farmer during an impact study with Laymen Agro in 2019
Upaya’s IM efforts have evolved a lot since those early years when the first 100 surveys were collected and analyzed. Today, we work with enumerators in India who have helped us scale our surveying efforts. We also use Tableau to analyze and share our findings in a series of in-depth dashboards available to the public. And more recently, as Rachna Chandrashekhar explained, we’ve adapted our approach to collecting surveys in light of the Covid pandemic.
Today, we also have a dedicated IM team and four team members regularly contributing to our IM efforts! Led by Sachi, the IM team works diligently behind the scenes to collect and interpret our impact data. Throughout the year, we also get to feature them and their work on our blog!
So, in honor of Upaya’s 10th anniversary, and to celebrate how far our IM efforts have come over the years, we are proud to shine a spotlight on the team members who make this work possible today: Sachi, Shrishti, Daphne, and Rachna.
Upaya’s IM Dream Team
Sachi Shenoy
Co-Founder & Chief Impact Officer
Describe your role on the IM team.
“I have the fun and enviable job of working with my dazzling IM colleagues to design surveys, collect feedback from our entrepreneurs and jobholders, analyze that feedback, and communicate our findings to the outside world. I also regularly plug into industry working groups—to ensure Upaya stays current with best practices—and advise other organizations at a high level on their impact measurement strategies.”
What is one of the most challenging aspects of your IM work?
“Setting up an environment to collect as objective feedback as possible. We want to be careful to let the respondent guide her own responses, and not introduce bias in any way — even through something as simple as how a question is worded (and then translated into local language). We are constantly refining our data collection process, but the effort is worth it.”
Why do you feel impact measurement and management is important?
“Fifteen years ago when I started this work, I thought impact measurement was more of an obligation… something we had to do in the context of standard reporting, kind of like regular financial reporting. What I now appreciate, and what drives my work, are the insights we glean from the data we collect. The feedback gives us ideas for advice we can pass along to our entrepreneurs, ways in which they can improve working conditions, and overall create a positive and sustainable environment for everyone. Data can be a powerful force for change.”
What contribution do you hope Upaya’s IM work will make in the development industry?
“Our IM practice is proof that any organization can leverage impact data to effect change—they need only to put in the effort. I want our approach to dispel the myths that this work is too expensive, too complicated, or too academic. Even for the most resource-constrained organizations, taking the first step to collect feedback from your most important stakeholders is critical and can quickly lead to regular practice.”
Shrishti puri
Senior Data Associate
Describe your role on the IM team.
“My role with the IM team starts with working with the data... exploring stories, building analysis out of our entrepreneurs’ surveys, and connecting our impact data with the entrepreneurs financial portfolio. I support our end-to-end visibility through data and reporting.”
What is one of the most challenging aspects of your IM work?
“The most fascinating part of any impact evaluation study is constructing the evaluation to capture not just the stories and trends that the data directly speaks to but also the hidden nuances and realities that are extremely critical to test our hypotheses. We are cognizant of the challenge, and hence, consistently work towards asking meaningful questions. In simple terms, “Is it enough?” is always on our minds.”
Why do you feel impact measurement and management is important?
“We belong to an industry where our work is incomplete without an accurate evaluation, measurement, and management of the impact that we intend to create through our investments. As impact torchbearers, we understand the criticality of complex data, the nuances of our various analyses, and the importance of adhering to the industry standards, which is supported by our initiative to create an impact evaluation benchmark in the impact investment industry.”
In sifting through so many surveys and data points, is there a story or trend that has really stuck with you? Please share.
“Our data reflects our investment thesis to the point, targeting the poorest of the poor with each new company coming in. It’s striking to see how our partner companies are employing people from the most disadvantaged backgrounds— those previously working in unstable workplaces, earning no or meagre income, with no education or skill set. 63% of our jobholders are in their prime age of 21-40, with 20% of them with no school education... 40% of them previously worked as manual laborers. This lens validates our work each day.”
Daphne delaski
Senior Associate
Describe your role on the IM team.
“As a Senior Associate at Upaya, I wear many hats and have the opportunity to do a variety of things. On the IM side of things, I focus on our partner surveys. Because I'm based in Seattle, I don't actually conduct surveys, but I oversee the process. I create the surveys and do all of the back-end work on our survey platform, and I coordinate with our team in India and the enumerating agency we work with. Once surveys have been conducted, I do the first rounds of cleaning and analysis on the data, creating reports to share with our partners, and working with the other members of the IM team to continually assess our learnings from each survey.”
What do you enjoy most about your IM work?
“Honestly? I love data cleaning! There's something very satisfying about taking a messy Excel file and cleaning it up. However, a close second would be problem solving in collaboration with the other IM team members. There are four of us split between India and Seattle, and the past year has presented us with numerous unexpected challenges. Learning how to approach and work through these challenges as a team has taught us a lot, and I think we've become a stronger team because of it. I believe our different roles complement each other well, and working so closely with one another is definitely a highlight for me.”
What inspires you to start your work every day?
“For me, the most important quality of any position I hold is that it's doing something for the greater good. Simply knowing that the time and energy I put into my work can have a positive impact on individuals other than myself is what inspires me to continue showing up each day. Being Seattle-based and so far away from our work physically does present an extra challenge—I rarely get to meet jobholders and hear their stories firsthand. But working with their survey responses, and learning about the positive impact that employment has on their households, is extremely rewarding for me, and is more than enough to make me excited to continue doing this work.”
Why do you feel impact measurement and management is important?
“Our model is unique in that we do not typically interact directly with the jobholders—instead, we work with their employers. Conducting impact surveys is our primary opportunity to hear directly from beneficiaries themselves, and learn about ways in which their lives are being impacted through our work.”
Rachna chandrashekhar
Manager, Accelerator & Impact Management
Describe your role on the IM team.
“As part of the IM work we do at Upaya, we also want to build knowledge and capacity of our partner companies to better understand the impact they are creating and how they can measure it. I work with the partners to build their Theory of Change and identify indicators and key data points that would enable them to monitor their impact. I also work with them and with our third-party survey partner to plan and ensure effective implementation of Upaya’s IM surveys.”
What do you enjoy most about your IM work?
“I absolutely love engaging with our partners on building out their Theory of Change. I think it's a framework that really puts into perspective the work each of them are doing and the current and potential impact that they will create over a period of time. It gives such an insight into the entrepreneur's vision—not just for the business, but also larger shifts they want to create and the positive impact they hope to have on the communities they work with.”
What is one of the most challenging aspects of your IM work?
“The challenge has primarily been in the context of COVID-19. Over the course of last year, we were able to shift our otherwise in-person surveys to phone surveys. We've seen more hurdles with the operational logistics in this case, and it is more time consuming. This requires us to be more patient, but we are grateful to have our partner companies, as well as survey partners, support us in this process.”
What contribution do you hope Upaya’s IM work will make in the development industry?
“What makes Upaya unique is that we really commit time and resources to undertaking the responsibility of measuring and monitoring the impact of all our portfolio companies. I hope this sets an example for other funders and investors in the sector to shift or share the burden of IM work with the entrepreneurs. This could give investors key insights on their portfolio to really understand the impact that is being created and make informed decisions on investment and support to businesses on ground. We are always happy to share our experiences and learnings on this front.”
Upaya’s work of supporting social entrepreneurs as they create dignified jobs for people in extreme poverty is part of a much broader conversation about the dignity of work.