Sustainable Development in Europe: A Multicriteria Decision Analysis

• Published date: 01 June 2021
• Author: Giuliano Resce,Fritz Schiltz
• Date: 01 June 2021
• DOI: http://doi.org/10.1111/roiw.12475

© 2020 International Association for Research in Income and Wealth

509

SUSTAINABLE DEVELOPMENT IN EUROPE: A MULTICRITERIA

DECISION ANALYSIS

by Giuliano Resce*

Italian National Research Council (CNR)

SOSE S.p.A. (Italian Ministry of Economy and Finance)

AND

FRitz schiltz

Leuven Economics of Education Research (LEER), Faculty of Economics and Business,KU Leuven

McKinsey & Company

This paper proposes a novel approach to evaluate European countries on Sustainable Development

Goals (SDG) by means of Hierarchical Stochastic Multicriteria Acceptability Analysis (HSMAA).

HSMAA produces rankings with Monte Carlo generation of weights, overcoming the need to choose

one specific set of weights. The main contribution of this paper lies in the possibility of quantifying the

probability by which each country receives a given ranking. Furthermore, HSMAA allows to take into

account the hierarchical nature of SDG measurement given that each of the 17 Goals also consists of

several indicators. Our results show that Denmark outperforms other European countries, while lower

levels of performance are observed in Romania and Bulgaria. In between bottom and top performers,

we also find that many countries’ rankings vary widely by the chosen set of weights, exemplifying the

need to rank countries based on multiple weightings and to quantify the probabilities of each ranking.

JEL Codes: C43, O13, Q01

Keywords: composite indicators, hierarchical stochastic multicriteria acceptability analysis, sustainable

development goals

1. intRoduction

In 2015, more than 190 world leaders committed to 17 Sustainable Development

Goals (SDGs).1 In contrast to their predecessors, the Millennium Development

Goals, the SDGs are designed to tackle the cause of the problems, taking into

account their interconnectedness (Hák et al., 2016). Another key feature is their

focus on the mobilization of financial resources, as well infrastructure and technol-

ogy (Zilberman et al., 2018). The goals and corresponding sub-goals were set for

2030. As shown in Table A8, the majority of the goals agreed upon does not include

time-bound targets. This lack of targets for each (sub-)goal does not allow a

straightforward comparison of countries’ position to the level of ambition put

1See Tables A7 and A8 for a full description of the goals, indicators and targets.

*Correspondence to: Giuliano Resce, Italian National Research Council (CNR); SOSE S.p.A.

(Italian Ministry of Economy and Finance), Via Mentore Maggini n. 48C00143 Roma, Italy (giuliano.

resc@gmail.com).

Review of Income and Wealth

Series 67, Number 2, June 2021

DOI: 10.1111/roiw.12475

bs_bs_banner

Review of Income and Wealth, Series 67, Number 2, June 2021

510

© 2020 International Association for Research in Income and Wealth

forward in the SDG agenda (Miola and Schiltz, 2019).2 An alternative approach to

hold governments accountable is by benchmarking performance between countries.

To do so, one needs to define a set of indicators, and develop a weighting method

to combine the indicators into one composite indicator, either at the goal level or

overall.

To define a set of indicators when measuring SDG performance, a major diffi-

culty is picking one that is relevant for all countries being benchmarked—irrespective

of the aggregation method chosen to construct a composite indicator. For example,

in the SDG index,3 several targets for hunger and poverty were already achieved in

most EU countries before the goals were adopted. This follows directly from the fact

that the prevalence of stunting, undernourishment or people living on a budget below

$1.9 is very low in EU countries, and not because all forms of poverty are already

defeated. As is clear from this example, different indicators will be needed depending

on the context where the benchmarking is done. Therefore, several local initiatives

followed the ongoing implementation phase, a process often referred to as “Localizing

the SDGs.” At the European level, Eurostat provided a policy framework to monitor

SDG performance by constructing the EU SDG indicator set, containing 1454 indi-

cators for all European countries. This indicator set facilitates the monitoring of

SDG targets by the European Commission while publicly available data improves

transparency and allows researchers to construct measures of performance and

rankings.

To measure SDG performance, the chosen set of indicators needs to be aggre-

gated into one metric. Measuring SDG performance has been a topic of fierce

debate among researchers, policymakers and other stakeholders (Radermacher,

2015). The intrinsic multidimensionality of SDGs poses new challenges for eco-

nomic analysis, decision-making, and policy-making. The issue on how to synthe-

size multidimensional information into one metric has recently paved the way for

the development of Composite Indicators (CIs) (Nardo et al., 2008; Costanza et

al., 2016; Burgass et al., 2017; Greco et al., 2019). Despite their advantages, CIs may

send simplistic and misleading messages, if the construction process is not trans-

parent and/or lacks conceptual and statistical principles (Nardo et al., 2008). The

main concern is directly related to the choice of weights as only slight changes may

give rise to significant differences in the final evaluation (Sharpe, 2004; Saisana et

al., 2005; Cherchye et al., 2008; Permanyer, 2011; Decancq and Lugo, 2013; Foster

et al., 2013; Costanza et al., 2016; Becker et al., 2017; Greco et al., 2019).

A common approach to compare countries’ SDG performance is to impose a

set of weights for each goal and indicator. For example, this approach is adopted

when constructing the Bertelsmann Index where all components are equally

weighted, both within and among goals (Lafortune et al., 2018, p. 20). An import-

ant limitation of this conventional approach is that countries are not allowed to

prioritize goals. Although equal weighting is often supported by the argument that

2OECD (2017): Measuring Distance to the SDG Targets 2019. An Assessment of Where OECD

Countries Stand.

3The SDG index was constructed by the Bertelsmann Stiftung, in cooperation with the UN to

measure how all 193 member states are performing relative to the 2030 Agenda.

4Eurostat (2019). Note that some indicators are considered “multi-purpose,” and hence the number

of unique indicators is equal to 100.