UNRISD Sustainable Development Performance Indicators Pilot Testing — Synthesis Report
Executive Summary
By Bill Baue with Ralph Thurm
On 20 July 2022, the United Nations Research Institute for Social Development (UNRISD) released the Synthesis Report assessing the pilot testing of its Sustainable Development Performance Indicators (SDPIs). We at r3.0 managed the pilot testing process, and wrote this report, with our Senior Director Bill Baue handling the writing and our Managing Director Ralph Thurm handling the data analysis.
To introduce this Report, we share the Executive Summary here.
Executive Summary
While the concept of sustainability was introduced more than three centuries ago — in response to the emerging recognition of the unsustainability of industrializing economic activities — predominant human culture nevertheless staunchly refuses to abandon our unsustainable ways (von Carlowitz 1713). Economic entities, in particular, exemplify this irrational obstinance: two decades after the introduction of Sustainability Context, this mechanism for measuring sustainability performance vis-à-vis normative thresholds remains essentially unutilized — an ostrich strategy (“what we don’t know can’t hurt us”) that only hurtles humanity ever deeper into the path dependence of unsustainability (GRI 2002; Bjørn et al 2017).
Recognizing that even the United Nations Sustainable Development Goals (SDGs) generally lacked these normative thresholds (Baue & Thurm 2020), the United Nations Research Institute for Social Development (UNRISD) launched a four-year project on Sustainable Development Performance Indicators (SDPI) in late 2018. A 2019 Working Paper introduced a Three-Tiered Typology of Sustainable Development Performance Indicators (Baue 2019), predicated on the Sustainability Quotient (S=A/N) that determines Sustainability Performance (S) by assessing Actual Impacts (A) on the carrying capacities of the capitals in the numerator, in relation to Normative Impacts (N) on the carrying capacities of the capitals in the denominator (McElroy 2008). The Three-Tiered Typology is explained as follows:
- Tier One: Incrementalist Numeration
Numeration indicators focus on actual impacts, which include absolute indicators as well as “intensity” indicators that describe performance relative to a nonnormative counterpart (such as unit of production), and are therefore incrementalist by definition.
- Tier Two: Contextualized Denomination
Denomination indicators contextualize actual impacts against normative impacts. Also known as “Context-Based” indicators, denominator indicators take into account sustainability thresholds in ecological, social, and economic systems, as well as allocations of those thresholds to organizations and other sub-system entities such as sectors, portfolios, or bioregional habitats (McElroy 2008; McElroy & van Engelen 2012).
- Tier Three: Activating Transformation
Transformation indicators add transcontextual elements of implementation practices and policies (as well as more ephemeral emergence) to normative indicators in order to instantiate sufficient change within complex adaptive systems.
Working with its Expert Advisory Group, UNRISD developed a full set of 80+ SDPIs across these three tiers in 2020, and in 2021 collaborated with r3.0 (Redesign for Resilience & Regeneration) to pilot test these indicators with about two dozen For-Profit Enterprises (FPEs) and Social and Solidarity Economy Organizations and Enterprises (SSEOEs) as well as other intermediary organizations (such as the World Bank, World Benchmarking Alliance, and Impact Management Project). This report summarizes and synthesizes the findings of this pilot testing, and points toward future directions to scale out the use of these pioneering indicators (and the underlying thinking).
The report documents both Quantitative and Qualitative Results. In general, the results carried significant implications on at least two levels:
- Implementability, or the degree to which the full suite of indicators is implementable by FPEs and SSEOEs; and
- Transformativity, or bigger questions around the degree to which indicators — and the performance measurement, management, and reporting they entail — can serve as levers for more significant and necessary systems change.
The report looks at three key elements of Quantitative Results:
1) The heatmap scorecards — which assess full (3), partial (2), or (1) no data provided for each indicator — find that data for almost all indicators were supplied by at least some piloting organizations, but that no piloting organizations were able to supply data for all indicators.
2) The reasons why varied significantly, but one reason that the pilot testing project tracked was claims of immateriality.
3) The report also assessed performance on the Hard Context indicators (i.e. those that apply Sustainability Context via ecological, social or economic thresholds). On this front, one interesting result was relatively flat findings of unsustainability performance on some indicators over 5-year periods, but improved sustainability performance over these years. The pilot testing project did not generate sufficient information to attribute these results to specific causes, so we offer possible explanations.
Findings on the Qualitative Results clustered around seven areas:
1) The value of the indicators and the piloting process.
For example, GLS Bank found pilot testing participation to be “an incredibly exciting and enriching path, in which we have learned a lot,” and accordingly GLS Bank integrated the SDPIs in its 2020 sustainability report in a way that “puts sustainability in a global context and proposes indicators for budgets (that is, thresholds or allocations) and for norms.” (GLS Bank 2021)
Dr. Stefan Siemer, Head of Corporate Sustainability for the Weleda Group, said: “The Weleda Board of Directors and the Weleda Management Board have now decided that, as part of the new corporate strategy, Weleda will develop an inclusive reporting framework by 2025 — a framework that is Multi-Capital and Context-Oriented.”
2) Urgings to elevate the importance of the Hard Context indicators.
Manulife Director of Global Sustainability Kyle Cahill characterized the predominance of peer pressure demand for incrementalist ESG (Environment, Social, Governance) as “the disclosure tail wags the sustainability dog, resulting in effort and strategic decision making focused on ESG data vs. putting resources toward efforts that result in a more beneficial impact — such as what the UNRISD SDPIs address.” He continued: “So we need to elevate Context so that other organizations integrate it into their annual reporting calendar… Where and when do some of these Context-based metrics start to make their way into this cycle?”
Cabot Creamery Cooperative Sustainability Director Jed Davis echoed these sentiments: “Cabot has been putting out Context for more than a decade to a deaf market that is demanding in-depth information that largely lacks Context.” Given the finite resources economic entities have to expend on performance assessment, Davis strongly advocates for wise prioritization: “If we’re going to spend time on indicators, our time is best spent on Context-based indicators, so why not focus on the ~19 indicators that are Context-based?”
3) Complaints on the work burden due to the broad scope of the indicators.
4) Encouragement to hone in on the indicators of true value.
Cahill of Manulife summarized that “The UNRISD approach should not be, ‘more is better.’ It should really hone in on the indicators of true value.”
SDPI Expert Advisory Group member Peter Utting proposed categories of “true value” indicators as those which support the ability to analyse “i) hard context performance, ii) the trajectory of change, and iii) variations in performance via granular disclosure (e.g. showing variations by occupational hierarchy).”
5) Identification of missing indicators that should be added to the set.
6) General feedback on specific indicators.
7) Technical problems with the indicator manuals / data collection spreadsheet / pilot testing process.
The report then analyses the findings in the Synthesis section, which focused on five areas:
1) Universal Indicators
The Heatmap Scorecards revealed that no pilot test participants were able to provide data for all the indicators, and immateriality claims were made on almost all of the indicators. Accordingly, the hypothesis of universal indicators was disproved by the pilot test.
Davis of Cabot Creamery Coop said, “I can appreciate the desire for universality, but materiality will always trump universality. One way to integrate the desire for universality is to provide sector-based indicators that are broadly applicable to the sector as a starting point, but still apply organization-specific context-based materiality.”
2) Core Indicators: Hard Context
Strong support emerged for focusing primarily on Hard Context indicators.
Weighing in on the concern over “requesting too much data from the organizations,” Expert Advisory Group member Peter Utting suggested a solution: “one option might be to prioritize hard contextualization… we need to be able to use all the data gathered to provide an analysis of context-based performance.”
3) Materiality: Context-Based
Following directly on the previous two points, piloting organizations support the idea of determining which indicators are relevant to a given economic entity by applying Context-Based Materiality, which transcends the limited lens of traditional finance-oriented materiality by focusing instead on impacts that organizations have on vital capital resources that stakeholders also rely on for their wellbeing, which creates normative duties and obligations for organizations to manage their own impacts on these resources sustainably — in other words, making sure they both do not deplete and continually regenerate resources necessary for ongoing stakeholder wellbeing.
Davis of Cabot Creamery Coop asserted: “Context-based materiality is fundamental — traditional materiality actually undermines a context-based approach to materiality.”
4) SDPI Integration
The indicators are currently assessed in isolation, highlighting the need for a means to integrate performance assessment across indicators, which requires an approach that assesses the sustainability of impacts on vital capital resources in commensurate ways, without substituting capitals.
Davis of Cabot Creamery Coop, who has piloted the MultiCapital Scorecard, which integrates sustainability performance assessment across all the capitals — traditional financial capitals as well as natural, human, and social capitals (Baue 2020) — strongly supports this approach:
“Sustainability performance is totally analogous to financial capital measurement — profitability is a threshold that measures sustainability — but financial reporting on its own doesn’t have to deal with commensurability between impacts on other capitals. Environmental & social reporting doesn’t have that luxury — since these forms of reporting cross capitals, they have to contend with commensurability and the non-substitutability of different capitals, by definition.”
5) Scaling Up and Out
For the indicators to fulfil their transformative potential, they will need to scale up through widespread usage, and scale out via implementation by diverse players operating across a broad spectrum of intervention points in the marketplace.
In Conclusion, the UNRISD Sustainable Development Performance Indicators pilot test has shown that it is not impossible to implement indicators that assess performance relative to sustainability thresholds and transformation. Quite the opposite: implementing thresholds- and transformation-based performance indicators is altogether feasible! In fact, not only are such thresholds- and transformation-based performance indicators feasible to implement, but also, enterprises are eager to implement them.
Systems change research suggests that social tipping points can be triggered by significant minorities of a reference population (as little as 25%) with the proper combination of passionate commitment and ideas whose time has come (Centola et al 2018). This report documents evidence supporting the idea that the time for thresholds- and transformation-based measurement has come.
Jed Davis of Cabot Creamery Cooperative summed up this pilot testing project succinctly: “The SDPI indicators are ground-breaking in a very positive way.”
References
Baue, Bill. 2019. Compared to What? A Three-Tiered Typology of Sustainable Development Performance Indicators: From Incremental to Contextual to Transformational. United Nations Research Institute for Social Development (UNRISD). Sustainable Development Performance Indicators Project. Working Paper 2019–5. October 2019. https://www.unrisd.org/baue
Baue, Bill. 2020. From Monocapitalism to Multicapitalism: 21st Century System Value Creation. r3.0. White Paper №1. December 2020. https://www.r3-0.org/wp-content/uploads/2020/12/r3-0-White-Paper-1-2020-From-Monocapitalism-to-Multicapitalism.pdf
Baue, Bill, and Ralph Thurm. 2020. Thresholds of Transformation: a Common Denominator to Transcend Incrementalism: UNRISD & r3.0 Partner to Launch Pioneering Sustainability Indicators Pilot Project. r3.0. 29 October 2020. https://r3dot0.medium.com/thresholds-of-transformation-a-common-denominator-to-transcend-incrementalism-unrisd-r3-0-74fff499bcdb
Bjørn, Anders, Niki Bey, Susse Georg, Inge Røpke, Michael Zwicky Hauschild. 2017. Is Earth recognized as a finite system in corporate responsibility reporting? Journal of Cleaner Production. Volume 163, 1 October 2017, Pages 106–117. https://doi.org/10.1016/j.jclepro.2015.12.095
Centola, Damon, Joshua Becker, Devon Brackbill, and Andrea Baronchelli. 2018. Experimental evidence for tipping points in social convention. Science. Volume 360, Issue 6393. Pages 1116–1119. 8 June 2018. https://www.science.org/doi/abs/10.1126/science.aas8827
Global Reporting Initiative (GRI). 2002. Sustainability Reporting Guidelines. Second Generation (G2). https://www.r3-0.org/wp-content/uploads/2020/03/GRIguidelines.pdf
GLS Bank. 2021. Integrated Sustainability Report 2020. https://nachhaltigkeitsbericht.gls-bank.de/files/About-GLS-Sustainability-Report-2020_English.pdf
McElroy, Mark W. 2008. Social Footprints: measuring the social sustainability performance of organizations. Dissertation. University of Groningen, the Netherlands. https://pure.rug.nl/ws/portalfiles/portal/13147569/DISSERTATION-2.pdf
McElroy, Mark W. and Jo M.L. van Engelen. 2012. Corporate Sustainability Management: The Art and Science of Managing Non-Financial Performance. New York: Earthscan.
Von Carlowitz, Hans Carl. 1713. Sylvicultura Oeconomica.
