Composite indicators are regularly used for benchmarking countries' performance, but equally often stir controversies about the unavoidable subjectivity that is connected with their construction. Data Envelopment Analysis helps to overcome some key limitations, viz., the undesirable dependence of final results from the preliminary normalization of subindicators, and, more cogently, from the subjective nature of the weights used for aggregating. Still, subjective decisions remain, and such

more »

... ing uncertainty propagates onto countries' composite indicator values and relative rankings. Uncertainty and sensitivity analysis are therefore needed to assess robustness of final results and to analyze how much each individual source of uncertainty contributes to the output variance. The current paper reports on these issues, using the Technology Achievement Index as an illustration. * This paper is an offshoot of the KEI-project (contract n° 502529) that is part of priority 8 of the policy orientated research under the European Commission's Sixth Framework Programme (see http://kei.publicstatistics.net/). Laurens Cherchye thanks the Fund for Scientific Research-Flanders (FWO-Vlaanderen) for his postdoctoral fellowship. 1 For an overview, see the JRC information server on composite indicators: http://farmweb.jrc.cec.eu.int/ci/. The construction methodology that is used in the present paper is rooted in Data Envelopment Analysis (DEA). The original question in the DEA-literature is how one could measure each decision making unit's (e.g., a firm's) relative efficiency, given observations on input and output quantities in a sample of peers and, often, no reliable information on prices (e.g., Charnes and Cooper, 1985) . One immediately appreciates the conceptual similarity between that original problem and the one of constructing CIs. In the latter case, quantitative sub-indicators for overall benchmarking are available, but as a rule there is only disparate expert opinion available about the appropriate weights to be used in an aggregator function. Yet there are differences between the two settings as well, the most notable one perhaps being that CIs typically look at 'achievements' without taking into account the input-side. Though there are some interesting exceptions (see the work of the European Commission on the Summary Innovation Index in 2005) A known remarkable feature of the DEA-methodology is that it looks for endogenous (possibly constrained) weights/shadow prices, yielding an overall score that depicts the analyzed decision making unit in its best possible light relative to the other observations. This quality explains a major part of the appeal of DEA-based CIs in real settings. For example, several European policy issues entail an intricate balancing act between supra-national concerns of the centre and the country-specific policy priorities of member states. If one opts to compare composite performance of member states by subjecting them to a similar weighting scheme, this may prevent acceptance of the entire exercise. To take an example: with reference to European social inclusion policy, Atkinson et al. (2002) remark that "in the context of the EU, there are evident difficulties in reaching agreement on such weights, given that each member state has its own national specificity." As the essence of DEA is that it yields most favourable, country-specific weights, it may help to counteract such problems. However, the typical DEA set-up, which only requires the endogenous weights to be nonnegative, is insufficient to guarantee peer acceptance. Usually some expert information about the most appropriate weights to be used for aggregating the individual sub-indicators is available, and such opinions should ideally be incorporated to make the weights acceptable. We will provide a typical example below. DEA-based CIs have inter alia been used to assess European labour market policy (Storrie and Bjurek, 2000), European social inclusion policy (Cherchye, Moesen and Van Puyenbroeck, 2004) , and internal market policy (Cherchye, Lovell, Moesen and Van Puyenbroeck, 2005) . A similar model has been tested to assess progress towards achieving the so-called Lisbon objectives (European Commission, 2004, p. 376-378). Similarly, some authors have proposed a DEA-approach for the well-known Human Development Index (Mahlberg and Obersteiner, 2001; Despotis, 2005) . In this paper, we will use the Technology Achievement Index (TAI) to illustrate our approach. Together with the Human Development Index, the TAI is developed by the United Nations for the Human Development Reports. The main reason for using it here as an illustrative example is that it figures likewise, and in an extensive fashion, in the JRC-OECD Handbook on Constructing Composite Indicators (see Nardo et al., 2005a; 2005b) . 2 We will complement the handbook's results by providing a more in-depth analysis of the DEA approach. We will start in section 2 by briefly discussing the TAI as well as the available information on possible weighting schemes, obtained by a panel of experts. Section 3 presents the basic model and indicates its relationship with more conventional DEA-models. We then 2 Regarding possible methodologies for composite indicator construction, both references have a considerably broader scope than the current paper, which only focuses on DEA-based indices. For instance, the interested reader may find there sensitivity and uncertainty analyses, using the TAI-data, that compare DEA-based results with those stemming from otherwise obtained indices (e.g. via exogenous weighting, or via a non-compensatory multicriteria approach).