Sharing Negative Results From Evaluation

It isn't surprising at all when organizations from social sector refuse to share negative results from evaluation, as the stigma attached to failures could be overwhelming. For many, it means the loss of reputation, possibly subsequent loss of support and funding as well. But for those prominent organizations involved in large-scale social programs that could influence public policy and create lasting social impact, they may have no option in avoiding to share. The reality is that we're still a long way from getting all to be open in sharing negative results from evaluation.

Ideally, we hope all organizations will perform evaluation, then take action to improve and also share negative results to inform program effectiveness and for collective learning. But the scenarios below are probably more familiar in reality:

1. Evaluate and act on findings but reluctant to share in order to avoid losing support/ competitiveness 
2. Evaluate but refuse to accept negative results, no major change in strategy or willingness to share 
3. Do none of the three key tasks seriously and simply rely on marketing hype to gain support 

There's a great deal of interest in social sector to move toward the ideal direction, and it's necessary not to be trapped in the belief that organizations that tackle social issues carry the sole burden to do more in evaluation. It's really counterproductive for funders/ the public to react negatively to the first hint of program ineffectiveness, or we simply withdraw support for those organizations that admit falling short on delivering promises without much patience although we acknowledge the complexity of social issues tackled by them. Critics also used to complain about over reliance on feel-good stories without solid data, isn't it partly our faults for demanding quick and positive results only? We are simply given what we're obsessed about.

It's obvious that broader measures are needed if we want more to come forward in sharing results from evaluation. It's really beyond developing evaluation standard and infrastructure, the mindset and perception change of all stakeholders in social sector matter equally. I think what we used to say in enterprise environment is also relevant in tackling social mission – “Experimental failures are simply not fundamental organization failures but necessary for breakthrough.”

One Size Fits All Social Innovation

When it's popular, people follow. This common principle probably also applies to social innovation. If we perform a correlation study, it wouldn't come as a surprise to me if replication of any type of social innovation is heavily linked to popularity and intensity of media coverage. Is there really much careful consideration involved when it comes to replicating social innovation?

Talk about light, solar lanterns become the 'definitive solution' at this moment. Banking? It's mostly narrowed into mobile banking immediately. And sanitation? Mobile toilet/ toilet kiosks are on the rise. It's important that we don't lose track that those are promising initiatives but not the permanent solutions. We have to explore other potential solutions within the same fields.

We always stress on poor-centric product design, but I'm not always convinced that their needs have been carefully considered during replication of popular products. The marketing slogans almost always quote on significant number of the poor living without certain products to justify the potential. It has become the standard way to sell the idea in communities with distinctive needs from different parts of the world. I think we need to be careful that it's not popularity, media coverage, simplistic market size and funder attention that drive replication. Social innovation shouldn't be turned into default solutions.

Again, For-Profit Social Enterprises Suck?

For-profit social enterprises state that they need profit to scale social impact effectively, it's not surprising why skepticism abounds when they could hardly prove impact. The controversial for-profit microfinance institutions are able to show remarkable growth in loan volume at global scale, but their social impact is still debatable. As it's widely known now, microfinance access alone doesn't equate impact and could also cause over indebtedness. The point is let's not kid ourselves into believing that for-profit model is in the mainstream and widely accepted now.
 

For-profit social enterprises especially those in poverty alleviation need longer investment horizon, likely more technical assistance and below-market returns. Hence there are criticisms quite often like the accusation that they generally hide behind weak business plan. Is the accusation fair?

True, there are some pretentious firms that market themselves as social enterprises but what comparison are we trying to make in general? Multinational consumer companies are well documented to struggle to establish themselves in poor regions for decades even with their level of resources. Selling is tough in those regions let alone proving impact and it usually takes longer to refine the business model. For skeptics who advocate improved/ new approach in poverty alleviation but condemn for-profit social enterprises right away, do they imply that there should never be any attempt on for-profit approach at all? Their critique can be helpful to practitioners, beyond that, their premature condemnation serves only contradiction and a disservice to early stage innovation.

Small Business or Employment Better?

Recent post in Social Edge explored an important question of whether we should work with the poor as employees or entrepreneurs. This question has important implication especially to social enterprises that prepare the poor as entrepreneurs. As highlighted in the post, not all will want or have the characteristics to be entrepreneurs. The transition from welfare is challenging since entrepreneurial work is often associated with fluctuating income. Business education needed and learning cycle may also take away some earning potential. I think the key consideration to effectively blend entrepreneurial and employment characteristics can broadly fall into 2 areas.

Specific living context – Their poverty condition, their desire and capacity to run small business could yield initial insight, further assessment along the transition challenges from welfare to work, education and support that they need to follow through the intervention programs will provide firmer indication on how to blend effectively.

Programs and goals – Income generation does not necessarily mean intervention should incline toward business ownership, whether it's better to own small business or to be employed should not be turned into ideological debates. It takes evidence from practices to determine which route is actually more effective under different contexts. For example, locals recruited as sales agents in the rural supply chain obviously display entrepreneurial and employment characteristics but without the need to own the business. They have to be entrepreneurial to sell while they could be attached to outsourcing agencies or the product companies with stable basic income and support to work. 

No matter how much we want to leverage on market-based approach, I think another important aspect that we should not neglect is how to blend in welfare/ subsidized assistance throughout the intervention, which ideally should cease in significance as the poor make more progress.