Beyond The Numbers: Measuring What Really Matters to Strengthen Health Systems

Many health stakeholders have identified the need to prioritize strengthening health systems and addressing disparities in access as a key step to achieving the Sustainable Development Goals (SDGs) — specifically SDG 3 on health and wellbeing for all.

Policymakers, private organizations and implementers throughout the health sector have been shifting their focus accordingly, spearheading access to health care initiatives, including diverse projects that address critical weaknesses in health systems in low- and middle-income countries (LMICs).

While evaluation measurements traditionally focus on counts and outputs, this shift in health care approaches also implies the development of new measurement tools to ascertain the real impact of these initiatives. This includes defining metrics that span from individuals to the population level, including more nuanced analysis of progress than a binary success or failure outcome. It also aims to capture actionable data on the quality and the extent of the care provided.

In the case of noncommunicable diseases (NCDs), and particularly in the field of oncology, incomplete treatment for non-medical reasons is one of the most common causes of treatment failure in sub-Saharan Africa. Learnings embedded in the social fabric and wider cultural and national contexts are critical to understanding the full picture of progress and impact.

Discussions at this Devex Ecosystem event will aim to challenge existing measures of progress. We will hear from stakeholders in health policy, program implementation, and biopharmaceutical companies on why they think these metrics matter — as well as their motivations behind using counts and/or outcomes-based approaches.

The session will unpack the main challenges to rethinking M&E in health care, discussing how M&E systems could help shift the focus to sustainable outcomes — including systematically incorporating more granular data points and working with local ecosystems in LMICs — as well as the frameworks needed to drive sustainable impact.

Some key challenges will be identified and serve as a basis for discussions:

• Transparency and accountability: shifting focus towards assessing the needs, investing resources and implementing activities to yield impactful and sustainable outcomes goes hand-in-hand with rethinking which data points are considered as evidence. Validation and acceptance of these throughout the health care community, particularly in a culture of hard data, can risk incidences of confirmation bias and opinion (UNICEF). Alignment across the industry and recognition of the validity of those new data points will be necessary, in order to ensure impact toward strengthening health systems in LMICs. Key questions could include whether there are cases in which it is better not to publish metrics, for example where organisations cannot fully evaluate the progress of an initiative.
• Accuracy and timeliness: re-thinking M&E practices to highlight real-time measurements — while ensuring the accuracy of findings and the possibility of extracting actionable content — is key to creating a shift toward more people-centered analysis. How could these new tools help the industry keep on track with its long-term goals? Better collaboration with country-level implementation toward self-sustained program outcomes and continued behavior change is essential — and can be impacted by a variety of external factors, including emergencies, political will, and access to populations in need. These factors may in turn also affect the accuracy of the evaluation metrics. Discussing paths to avoid double counting funders and implementers’ KPIs will also play an important role in tracking progress toward achieving the SDGs
• Funding: only about 2% of global health financing is spent on NCD-focused initiatives that improve access to products and health services (National Library of Medicine, 2020), presenting an opportunity for private sector involvement and local policy makers in supporting NCD-focused initiatives that improve access to products and health services. Part of this financing will hinge upon proposing adjusted measurement tools to accurately evaluate progress toward the SDGs.