Measuring Social Impact

Part 3 of a series based on a paper I co-authored with Christine Cooper and Darlene Himick.

In the previous article in this series, we learned how a model for social impact bonds (SIBs) was developed in the UK to fund homelessness interventions. We saw that the model depends on individual social workers acting as entrepreneurs. These social workers, or “navigators, would be assigned a specific set of homeless clients, chosen from the 830 in London who had experienced more than one homelessness episode but were not yet considered chronically homeless.

In this article, we will learn how the SIB contract, negotiated between the government and the St Mungo’s charity in 2012, measured the navigators' success with these clients.

Impact Measurements

For a social impact bond to work, the return to investors needs to be based on the delivery agency achieving certain performance metrics. Social Finance, the consulting company that helped set up the St Mungo's SIB, said:

The most important criteria for any outcome metric is whether it incentivises a service that ultimately improves outcomes for those who use it.… The key is to identify an outcome metric which is measurable and objective. (Social Finance, 2013, p. 11)

The St Mungo's SIB contract included five outcome metrics negotiated by the various parties. These metrics, along with the corresponding weighting of payments under the contract, are listed in the following table:

 Outcome metrics and payment weights  Source:  Contract obtained by authors through a Freedom of Information Request

Outcome metrics and payment weights
Source: Contract obtained by authors through a Freedom of Information Request

The table reveals the importance placed by the state on each outcome. St. Mungo’s was primarily tasked with (and rewarded for) removing individuals from the street. This is reflected in the three primary outcome metrics in the table.

The third of the metrics, “Supporting clients into sustained reconnection to a country where they enjoy local connections,” although carefully worded, reflects the fact that 52% of homeless people in the UK are non-UK citizens, including the 28% who are from European Union countries and are therefore entitled to state benefits. You can imagine the political appeal of this metric at a time when the governing Conservative party needed support from UKIP, the most xenophobic party in parliament.

The model depends on individual social workers acting as entrepreneurs.

The payment weightings were part of the SIB contract negotiation. St Mungo’s argued that the weighting for “sustained reconnection” was insufficient to cover its costs, so it was increased. The contract required St Mungo’s to have staff travel with the homeless person back to their country of origin and ensure they got access to appropriate support there. According to the official documentation, St Mungo’s staff would not “walk away” until they were sure the participant was likely to remain outside the UK.

The fourth and fifth metrics were assigned considerably less weight. The fourth was about connecting the homeless individuals to employment, volunteer work, education, or training. An interviewee said this would be the most difficult measure to achieve. The fifth had to do with reducing emergency ward (A&E) visits.

20140608_141946.jpg

This fifth metric was particularly problematic. Research shows that homeless people in the UK are 3.2 times more likely than the general population to be admitted to hospital, and cost 1.5 times more when they are there. Since most homeless people lack a general practitioner, they are six times more likely to present themselves at A&E when they do need medical care. Reducing A&E visits would therefore be a logical metric to show the impact of the proposed homelessness services on broader societal costs.

Unfortunately, after the contracts were completed and signed, it was realized that hospitals in the UK are legally prohibited from releasing the required information, due to privacy legislation. So, this metric proved useless.

 

An Ideal World

These five outcome metrics embody certain neoliberal ideals. They create incentives for the navigator to try to increase the human capital of each individual homeless client, on the path to employability and self-sufficiency. If the navigators are being asked to become entrepreneurs, so too are their homeless clients. The underlying message here is that they need to build up their personal brand or go home.

These individual outcomes were not the only consideration, however. The final piece of the SIB puzzle was the value for money calculation. This will be explained next.


Photo of a rather direct attempt to end homelessness taken in Bath in 2014.

 

References

Social Finance. (2013). A Technical Guide to Developing Social Impact Bonds. London: Social Finance Ltd.