Complex benchmarking and efficiency measurement of public agencies: the case of Estonian rescue service brigades.

Author:Reiljan, Janno

    The efficiency of public agencies is inevitably in the public spotlight, because these institutions use the people's common property and spend taxpayers' money. The size and structure of a public agency's budget are quite clearly defined and measurable. Unlike private companies, it is unclear what the total utility (volume and quality) of a public agency service is--public services do not generally have consumer assessments of the resulting benefits reflected by market value. In addition, public officials are subject to the various interests of stakeholders such as consumers, taxpayers, politicians at various levels of government, and other organizations (including other state agencies). Another feature (which is often the result of the aforementioned circumstances) is that a public agency has to fulfill a variety of tasks. Taxpayers expect efficiency gains in the work, and simultaneously, consumers expect improvements in the availability of the service (Dixit 2002: 697). The diversity of the objectives and stakeholders make evaluating the efficiency of the work of a public agency a multifaceted problem. In addition to objective criteria, consumer cognition also has to be taken into account (that is, whether the taxpayer is satisfied with the resulting public services) (Bolton 2003: 21).

    To evaluate efficiency in a private company, sales revenue (turnover, profit) from products (services) sold is compared to the costs or the amount of resources applied, and efficiency (productivity, profitability) estimates are obtained. The volume and quality of the administration services provided by a public authority (parliament, central and local governments, their agencies) have no monetary estimate of their benefits to compare to the expenditure and the use of resources incurred by these agencies. Therefore, the question arises whether public goods (national defense, internal security, etc.) or individually consumed services (general education, health care, care for the elderly, drug prevention, etc.) are guided by the principle that taxpayers' money should be used efficiently and effectively. The efficiency of the utilization of the general government budget is of great importance, because the percentage of GDP dedicated to their budget averages about 50% in the European Union and about 40% in Estonia.

    For public agencies, including the rescue service brigades selected as the research object of this paper, the main task is to design the readiness for service calls in accordance with the structure and demands of society, in terms of both volume and quality, thereby achieving the greatest possible economic efficiency of the use of inputs (level of preparedness or volume of services per unit of input). It follows that the specific problems of public agencies are performance measurement and evaluation of these relationships. Rather than using market-driven price signals, the needs of society must be defined by developing performance indicators that continuously monitor and analyze desirable outcomes and the most efficient way of achieving them. These problems (especially related to the management of efficiency) have not yet been adequately covered in Estonia, and this hinders public management in its search for opportunities surrounding constant efficiency improvement. At the same time, the theoretical foundations and methods for measuring the efficiency of public agencies also require further development.

    A public agency's work can be described in two stages (see Figure 1). In the first stage, the inputs (labor and capital) are used to create the potential (readiness) to provide the service (in the example of rescue service brigades, the number of employees on watch, staff qualifications to provide services, quantity and quality of the equipment, etc.). In the second stage, the potential is realized as the desired outcome for society (characterizing the rescue services: extinguished fires, rescued lives, people's sense of security).

    On the basis of these two stages, a variety of indicators can be used to describe a public agency's work efficiency:

    * the ratio of the potential created to the volume of inputs (process efficiency)

    * the ratio of the outputs generated to the volume of inputs (output efficiency)

    * the ratio of total utility (effectiveness) to the volume of inputs and potential (outcome efficiency).

    The aim of this paper is to work out an approach for complex benchmarking of the public agencies' efficiency by using frontier analysis methods, taking Estonia's national rescue service brigades as an example. To this end, the following research tasks are undertaken:

    * analyze the theoretical background of the efficiency benchmarking of public agencies based on the scientific literature

    * characterize the implementation options and challenges of frontier analysis methods

    * provide the opportunities and limitations of benchmarking evaluation of the process and output efficiency using Estonian national rescue service brigades as an example.

    The paper consists of three substantive parts. In the first part, the theoretical problems of efficiency benchmarking of public agencies are analyzed. In the second part, the methods used to benchmark the efficiency of public agencies are characterized and compared. The third part presents the methodology and efficiency benchmarking results of Estonian national rescue service brigades. The analysis is carried out using the statistical package Stata SE 12.0.


    Socially acceptable performance, in general, can be defined as doing the right things in the right way. Technical efficiency characterizes the economy of resources available to a public agency: the agency creates the maximum quantity or quality of output using a fixed amount of resources, or creates a fixed volume and quality of outputs using the minimum amount of resources. Technically effectively implemented inputs characterize cost efficiency (efficiency of input implementation), and technically effectively implemented outputs characterize output efficiency. The current approach is based on the works of Daraio and Simar (2007), Fried et al. (2008), Bogetoft and Otto (2011), and Cooper et al. (2011), which in turn are based on the seminal work of Farrell (1957).

    Frontier analysis as a benchmarking method does not evaluate the absolute level of productivity but is aimed specifically at identifying the possible scope (or the frontier) of efficiency improvement by comparing a specific public agency to the "best practice." Without directly finding the ratios of the resources and inputs, the frontier analysis method indicates the relative distance of observations (in the evaluated agency) of outputs and inputs relative to the reference agency (which is on the cost [or production] frontier of the observed data set) (Daraio & Simar 2007: 14). The comparison may be presented in two forms: the agency's ratio of output to a fixed volume of input compared to the "best practice," and the agency's ratio of input to a fixed volume of output compared to the "best practice." The question remains as to whether the "best practice" used as the benchmark truly represents the maximum possible efficiency levels.

    Introducing performance indicators enables us to compare and rank public agencies on the basis of performance. It must be recognized, however, that the agencies are providing their services in different socioeconomic conditions. Thus, the results do not reflect only the contributions of public agency employees but also the effects of general resource supply and the general environment. Therefore, the results achieved are affected by environmental differences. It is also not possible to ensure that comparable organizations have similar amounts of work, structures, and resources. Due to the historical development of comparable agencies, the nature of the work, the resource supply, and the interrelationships between the external environmental conditions are inevitably different. The following also highlights these two issues of performance benchmarking (Smith 2006: 76):

    * To what extent are differences in the performance of the work due to uncontrollable circumstances, and how much are they due to employees' contributions?

    * Is it possible to benchmark public agencies' performance by taking into account both of these aspects?

    The performance benchmarking options for public agencies involved in the provision of qualitatively different public goods are extremely limited. This article, however, analyzes public agencies that are essentially providing similar services--national rescue service brigades, as the (local) structural sub-units of the rescue services. Qualitative differences (in the tasks and their proportions) also occur between agencies that are providing similar services. In that case, the opportunities for developing performance benchmarking are greater, and the results can provide valuable information about the possibilities of improving the efficiency of public agencies.

    Theoretically, it should be the aim of a central resource distributor (generally a government or central authority) to find the best financing scheme that can be used to ensure that the required volume and quality of service are provided and any differences due to environmental conditions are compensated for. Such mechanisms normally ensure a certain standard level of service by shifting more resources into these local sub-units, where environmental conditions cause higher costs of providing the standard service. In practice, however, it is difficult to find a comprehensive financing scheme that can take into account all or even the main environmental factors that cause cost differences between the sub-units of public agencies (Smith 2006: 77).

    The data used to benchmark performance characterizes mostly the supply side of services, particularly the costs of offering the service. Demand-side...

To continue reading