'MAKING-UP DURING DATA ANALYSIS' THE PERFORMANCE MEASUREMENT SYSTEM IN KARNATAKA.

Author:Maurya, Dayashankar
 
FREE EXCERPT

INTRODUCTION

The public sector is argued to be a massive producer of measures and rankings to compare, certify, codify and evaluate performance (Le Gales, 2016). The state is termed as a 'performance state' in this 'ranking world' as the state is getting quantified and measured. The downside of performance measurement in public management is as old as the megatrend of performance measurement itself (Dahler-Larsen, 2014). Yet, even after half a century of research, performance measurement remains a contested issue (Johnsen, 2005). Designing a performance measurement system that works to achieve performance remains elusive.

The growing application of information technology (IT) in the public sector has led to an increase in the adoption of performance measurement systems even in the developing countries. There is growth in both demand and supply of performance measurement systems driven by reforms such as decentralization, marketization, anti-corruption programs and so on, in the public sector (Putu et al, 2007). The public sector in developing countries is characterized by low institutional capacity, limited involvement of stakeholders, high level of corruption and high level of informality and so, its environment is not conducive for the design and implementation of a robust performance measurement system (Tillema et al., 2010). Given this unfavorable context, it is expected that the design and implementation of performance measurement systems will face some unique challenges in developing countries which have otherwise not been observed in the developed countries. Though there have been studies examining performance management (2), very few have looked specifically at performance measurement. Gao (2015a) observes that "research on the implementation of performance measurement, especially in developing countries, is thin... more studies of the experiences and lessons in developing countries will help to enhance understanding of performance management in a global context" (p 94).

To address this gap, we analyze the design of a performance measurement system to assess the extent to which the design can generate an authentic performance improvement. We use the framework suggested by Henman (2016) to assess whether the conditions of 'authentic' performance measurement are met. We do a critical case study of performance measurement system in India, which is used to track performance in 668 public services in the state of Karnataka. The Karnataka government, through "The Karnataka Guarantee of Services to Citizens Act - 2011" also known as SAKALA, assures delivery of public services to its citizens within a stipulated period. The performance measurement system tracks the timely delivery of all the 668 services covered under this Act. We use a case study approach to assess the extent to which the performance measurement system in SAKALA meets the conditions of authentic performance measurement. We identify gaps in the existing performance measurement system and propose a new framework to analyze performance by conditions of an 'authentic' performance measurement system. We demonstrate the application of this framework using two-year administrative data for a district (3) in the state, Udupi. We compare the outcome of performance measured under the existing performance measurement system vis-a-vis new approach.

The present performance measurement system lacked features of an authentic performance measurement system. It lacks accuracy, does not pinpoint low-performing services and departments, and misrepresents performance by wrongly aggregating it to create a favorable impression. Thus, it rules out opportunities to identify what contributes to poor performance, critical for any effort to diagnose it. Therefore, the present performance measurement system does not identify the reasons or the weak links in the service delivery leading to poor performance. Subsequently, it provides little guidance on how performance can be improved. Hence, the present performance measurement system does not perforce performance. We suggest a five-stage framework for analyzing performance data that overcomes the limitations of the existing performance measurement system. The application of the new framework leads to the identification of weak links in service delivery. The proposed framework can be applied for measuring the performance of services beyond the case.

We make the following contributions. First, the study addresses the call by Gao (2015a) and contributes to the limited literature on performance measurement system in developing countries. Second, there have been very few studies on the intentional misinterpretation of performance data at the analysis stage to create a favorable impression of performance. The gaming of performance measurement system during data analysis has been observed by Hood (2007) and Radnor (2008). However, our findings differ. In the presented work, these gaming strategies are not put to use intentionally. Thus, the study highlights the lack of analytical capacity in the public system critical for designing and managing a performance measurement system. Fourth, we empirically test the framework of authentic performance measurement system as suggested by Henman (2016) which has not been done so far. We illustrate how in spite of the extensive adoption of IT-based tools to measure performance by the public sectors in developing countries, performance measurement systems are still far from providing an authentic performance measurement. Fifth, we develop a framework to analyze the performance of services specifically on the dimension of timely delivery as a critical performance dimension of public services delivery. Such a framework, to the best of our knowledge, does not exist in the public service performance literature. The proposed framework can be adapted to different contexts and different dimensions of performance, in addition to timely delivery.

THEORETICAL BACKGROUND

Performance in the public sector has been defined from different perspectives. One of the commonest perspectives is that of production logic, according to which performance is considered as the conversion of input to activities and then to output. Another perspective from a Public service Excellent Model considers performance as achieving social outcomes (Talbott, 1999). According to this approach, public organizations exist to achieve social outcomes. Third, from a public value theory, performance is considered as a realization of public value (Boyne, 2002; Van Dooren et al., 2015). Further, performance is discussed at different levels ranging from a public organization at the micro level, to policy sector a meso-level concept; to performance of government across sectors at the macro level (Van Dooren et al., 2015). Performance measurement is considered as a technology, tool or device of the government and "the purpose is to use data to govern, manage or steer the objects of performance and/or the actors that contribute to that performance" (Henman, 2016, p 597). It is conceptualized as a technical tool that evaluates public sector performance (Van Dooren et al., 2015) as well as a rationalized governance tool (Myonihan, 2009). Performance measurement has been identified to be used in more than 40 different ways (Van Dooren, 2006). However, the real purpose remains to improve performance (Behn, 2003). Apart from improving performance, there are other uses of performance measurement - to evaluate, control subordinates, make budgetary decisions and requests, motivate employees, promote the organization to stakeholders and political principles, celebrate accomplishments, and learn about program efficiency (Behn, 2003).

An ideal type of performance measurement process could be conceived as consisting of five steps - defining a measurement object, the formulation of indicators, data collection, and data analysis and reporting (Van Dooren et al., 2015). However, there are significant challenges in each step. As it is impossible to measure everything given complexity and multidimensionality of public programs, therefore, prioritizing what to measure becomes challenging. Task complexities and multiple goals characterize public programs. Besides, various stakeholders associated with these public services bring different perspectives to the table, which have to be taken into consideration while making specific choices to achieve these goals (Moynihan et al., 2011). Even when the decision is made on what to measure, deciding about measurement indicators is fraught with challenges. Ideally, measurement indicators should be relevant, timely, sensitive to change, precisely defined, easy to understand and comply with data requirements (Van Dooren et al., 2015). In the public sector, democratic values, collaborative and global nature of governance complicate the measurement of performance (Moynihan et al., 2011). Therefore, precise measures of performance may be impossible. Rather than systematically assessing performance, public sector performance measures generally assess it partially as measuring all aspects will be overwhelming. Thus, in the public sector, performance measures are generally proxies for performance and there is a performance phenomenon beyond what is measured (Henman, 2016; Christopher and Hood, 2006). Most of the performance indicators are 'tin openers rather than dials' (Carter et al., 1995, p 49) - that is, they do not give answers but prompt investigation and inquiry, as they provide an incomplete and inaccurate picture.

Further, how the data is collected has an important implication as different data sources and collection method pose different advantages and disadvantages. Next, the collected data needs to be analyzed and interpreted in a manner that aids decision making. The data could be interpreted to assess if the performance is meeting the norm or set target (norm and target setting); aggregated or...

To continue reading

FREE SIGN UP