>>>Click here to read and download our booklet that summarises the findings of this evaluation!
From the beginning, IRISi’s interventions have been evidence and research based. This means that each one of our projects to tackle Domestic Violence and Abuse in healthcare settings was carefully conceived in an academic context, tested in controlled environments to confirm its effectiveness and to identify eventual points of improvement. Only then were our projects brought into the real world for further implementation. Our flagship intervention, the IRIS Programme, is a specialist domestic violence and abuse (DVA) training, support and referral programme for General Practices that has been positively evaluated in a randomised controlled trial and in real-world settings.
The first areas implemented IRIS in 2010. From that point onwards, many other studies have been conducted to show the amplitude of this intervention, which is now running across the UK in more than 40 localities. Produced in 2022, “The social value of improving the primary care response to domestic violence and abuse: A mixed methods Social Return on Investment analysis of the IRIS programme” is one of those many studies and it concludes:
- For each pound invested in the IRIS Programme, a monetary return of £16.79 is expected.
- For each pound invested in the IRIS Programme, a social return of £10.71 was obtained.
This evaluation was supported by a grant received as part of the Health Foundation ‘Exploring Social Franchising Programme’, and it was led by Dr Anna Dowrick, Qualitative Researcher in the Department of Primary Care Health Sciences at the University of Oxford; Dr Meredith K.D. Hawking, a Research Fellow in the Wolfson Institute of Population Health at the Queen Mary University of London; and Dr Estela Capelas Barbosa, a Senior Research Fellow in the Violence and Society Centre at the City University of London.
“We decided to commission this evaluation to provide social franchise pilot sites with the kind of in-depth evaluation that we were often asked for. At the same time, we wanted to develop a kind of template or framework which addressed social value and social return on investment, which other sites will be able to make use of in the future”, explains Lucy Downes, IRIS Network Director.
The research establishes the social value attached to IRIS while the intervention works to improve the primary care response to DVA. After exploring the social value of IRIS, the researchers go beyond and estimate the Return on Investment (ROI) and Social Return on Investment (SROI). Return on investment (ROI) calculations are commonly used to guide investments of public funds, underpinned by a view that services should be at least cost-effective and preferably cost-saving. There is increasing emphasis on examining the social return on investment (SROI) of public-funded services. This approach enables decision-makers to examine and quantify if a service delivers valuable social impacts, in addition to understanding the solely financial returns.
Here are the 10 key points from the evaluation that we want to highlight.
#01 – Behind the data and the evaluation: which sites have joined this research
Five UK sites were selected for data collection due to their being early adopters of the social franchise approach to replicating the IRIS model, and represented varied provider types at different stages of the commissioning and delivery model at the point of evaluation initiation:
- Greenwich (service provider: Her Centre);
- Middlesbrough (service provider: My Sister’s Place);
- Gwent (service provider: Llamau);
- Swansea Bay (service provider: Calan DVS); and
- Swale (service provider: SATEDA).
The researchers interviewed key stakeholders in each IRIS site, reviewed core policy documents from their area, observed steering group meetings and conducted follow up surveys with participants. From these data, they developed and refined a social value framework. This framework was then used as the basis for a SROI analysis of the IRIS programme.
#02 – Understanding the guiding social values for IRIS
IRIS as a programme was expected to contribute to guiding values and these were important outcomes identified by stakeholders that were challenging to quantify. They were often expressed in terms of feelings about the worth of IRIS, rather than ‘hard’ outcomes. The list of benefits attached to the intervention according to stakeholders includes:
- Optimism;
- Visibility of DVA,
- Alignment between different efforts to address DVA at a system level.
#03 – Understanding the Social impact of IRIS
Social impacts were the specific outcomes considered important in each site. While there were local and contextual variations, the broad categories described below were conceptual tools for assessing the contribution IRIS was making across all sites:
- Professional role;
- Service user outcomes;
- Connectedness;
- Compatibility with existing health and DVA infrastructure;
- Credibility.
#04 – That said, why a mixed-methods evaluation approach?
The three guiding social values mentioned above – whilst providing important insights into the additional perceived value of IRIS – are difficult to measure or quantify with readily available routinely collected or research-generated data. This is a key challenge that researchers and stakeholders face when attempting to fully comprehend the social value of a complex, community-based health intervention in numerical analyses. This explains the need for a mixed-methods evaluation approach.
On the other hand, the social impacts illustrated by the research team are more suited to an economic assessment of social value. Assessment of progress against these values was different in each site relative to their local context and four out of five of these impacts were found to be quantifiable and were therefore used to underpin a social return on investment analysis.
The study reports, “We have sought to find a monetary figure that conveyed the benefit of the measure. Wherever possible, we relied on previously conducted Social Return on Investment Analyses or Economic Evaluations to obtain such values”.
#05 – The Social Value of IRIS: what did they find?
The study demonstrated that the value of IRIS extends far beyond increasing referrals to DVA services or improving service users’ lives. At the macro level, it can make DVA more visible in the primary care system, raising it as a priority for healthcare and shining a light on people affected by abuse who may otherwise be overlooked. It can improve alignment in values, priorities and responsibilities between organisations and localities working towards tackling DVA. As a programme, it can increase optimism and reinvigorate a sense of hope that DVA can be addressed for those working towards this goal.
At the local level, it can connect primary care professionals, service providers and service users, creating a new route for addressing patient needs and information sharing. It can improve the knowledge and confidence of primary care staff around dealing with DVA, increasing ongoing engagement and the perceived quality of consultations around this issue. In terms of outcomes for patients, increased referrals combine with locally negotiated impacts such as improved feelings of safety, reduced escalation of violence, reduced medicalisation of social issues, and providing support at all stages of the relationship.
#06 – The Return on Investment (ROI) of IRIS
Return on investment (ROI) is a performance measure used to evaluate the efficiency of an investment in monetary terms. It only takes into consideration monetary costs and monetary benefits, and thus, is considered useful as a comparative measure between investments. While ROI is commonly used by private companies to ascertain how effective their different investments are, more and more ROI has been becoming common in government departments as a measure of value for money. Programmes funded by public money have also been incentivised to estimate their return on investment (ROI). This is the case of the IRIS Programme, where the researchers found:
- A Return on Investment (ROI) analysis for IRIS showed the efficiency of investment in purely monetary terms. For every pound invested in the IRIS Programme, a return of more than £16 was achieved (ROI: £16.79).
- For every pound of taxpayers’ money invested in the IRIS Programme, one pound and 21 pence of further public spending is avoided as a result (ROI: £1.21 : £1 or 21%).
- On average per site the IRIS Programme cost £97,926 and covered a target population of just over 230,000 women.
#07 – The Social Return on Investment (SROI) of IRIS
A Social Return on Investment (SROI) is a methodology used to estimate the net social value created by organisations, which are not normally included in a traditional Return on Investment Analysis due to their inherent difficulty in translating into a monetary figure. As explained above, the approach to estimating the SROI for the IRIS Programme comprised three stages:
1) Identifying social impact of the IRIS Programme.
2) Developing a social return framework including attribution and monetary benefits from the IRIS Programme.
3) Estimating the Social Return on Investment (SROI).
This is what the researchers found out about the IRIS Programme:
- For every pound invested in the IRIS Programme, on average, there was a £10.71 social return (ROI: £10.71 : £1).
- The total net social benefit of the IRIS Programme was estimated to be £1,049,215. This resulted in a SROI ratio of £10.71 : £1, meaning that for each pound invested in the IRIS Programme, a social return of £10.71 was obtained.
#08 – Implications for policy-makers
For policy-makers, this research has demonstrated the importance of conducting both financial and social return on investments when considering service developments. As well as generating a broad evidence-base to support commissioning decisions, clear narratives about the social value of a proposed or existing service are important for communicating investment decisions to diverse stakeholders. Aligning economic analyses with qualitative participant narratives provides a persuasive account of the social value attributed to a service. Specifically, we have contributed evidence about the value – both financial and social – of investing in pathways from primary care into DVA services.
#09 – Implications for commissioners
For those involved in commissioning programmes in health care, this study has provided insights into the social values that drive the IRIS programme and how they can be quantified. These can be taken into consideration in discussions about the value of starting or continuing investment in the IRIS service in both the participating sites and in other locations. Furthermore, examining the social values and social impacts that are locally important directs attention to activities with can in turn maximise a social return on investment. In the case of IRIS, this includes activities that would increase professional engagement in the programme and increase referral numbers.
#10 – Implications for the IRIS network
For those involved in the delivery of the IRIS service the study has ventured ‘beyond the numbers’ to more accurately capture the full social impact of their work and efforts in this important area. By drawing out findings such as the ‘guiding values’, we have gone some way to highlighting the complexity of fully recognising the multitude of values that arise through DVA service provision. In doing so, this report shows the ongoing importance of the IRIS programme and sheds light on the additional value of the work of all those involved in DVA interventions.
In 2022, we celebrated our 5th anniversary. For this reason, IRISi’s “16 Days of Activism against Gender-Based Violence” campaign was themed “IRISwe: tackling gender-based violence in healthcare settings together”, a celebration of our efforts and our history alongside our partners and stakeholders.
During the campaign, we launched the results of “The social value of improving the primary care response to domestic violence and abuse: A mixed methods Social Return on Investment analysis of the IRIS programme”, an evaluation of the IRIS Programme that, once again, reinforces IRISi’s commitment to regularly review the evidence-base of its interventions.
Throughout the 16 days, we shared the results of this study to explore how our flagship intervention, IRIS, has benefited general practice teams and, consequently, the lives of thousands of women who have been affected by domestic abuse.
To conclude our campaign, we held a webinar where the researchers who conducted the evaluation presented the main qualitative findings. You can watch it here:
