| I.
INTRODUCTION AND BACKGROUND
1. This paper aims to summarize how issues
in the field of Development Indicators have
evolved over the last year. The Working Group
may recall that the issue of Development Indicators
has already been discussed at past sessions
of the Committee on Statistics as well as of
the Working Group itself. Development Indicators
are defined as those indicators which have been
selected to monitor the achievements towards
development goals set by global United Nations
conferences. There were several such conferences
during the nineties, and at the Millennium Summit
of September 2000, Heads of State and Prime
Ministers reaffirmed their commitment to achieving
a number of key development goals.
2. A number of problems affected the possibility
of ensuring a coherent process in using indicators
to monitor the development goals. These included
problems of coordination, both between countries
and international organizations and among international
organizations themselves; additional problems
of burden on national statistical offices; and
instances of non-relevance or doubts about the
statistical validity of certain indicators.
These problems caused the Economic and Social
Council to turn to the United Nations Statistical
Commission to solve the issue of identifying
a limited set of core indicators to be used
for monitoring progress towards the development
goals. In this regard, progress towards the
achievement of goals is usually assessed against
the benchmark year 1990, with goals generally
set to be achieved by 2015.
3. The Statistical Commission decided to form
a "Friends of the Chair" expert group on Development
Indicators in order to carry out the work. The
"Friends of the Chair" expert group was formed
with representatives of national statistical
offices worldwide, including some from the Asia-Pacific
region, and was chaired by Mr Tim Holt. The
process reviewed the goals of the conferences,
existing indicator frameworks, and the indicators
themselves. The "Friends of the Chair" also
benefited from comments and feedback that were
provided to the Statistical Commission by other
international statistical players, including
the ESCAP Committee on Statistics; the Committee's
views were most recently conveyed at the thirty-first
session of the Statistical Commission, held
in March 2001.
4. The "Friends of the Chair" last met at the
end of October 2001, and finalized the review
of a comprehensive framework of indicators that
could serve to monitor achievement towards the
goals set by several global conferences and
the Millennium Summit. The Working Group may
wish to note that a meeting, the "United Nations
Workshop on Development Indicators for the ASEAN
Countries", took place in Manila from 1 to 5
October 2001, prior to the final meeting of
the "Friends of the Chair", and the meeting's
views were brought to the attention of the "Friends
of the Chair" in its final review. That meeting
was organized by the United Nations Statistics
Division as part of a broader project for strengthening
the statistical capacity of ASEAN countries,
and was attended by nine ASEAN countries. The
ESCAP secretariat also attended and presented
its views on the issue.
5. The issue of Development Indicators will
be discussed, but perhaps not entirely settled,
at the 2002 session of the Statistical Commission.
The final report of the "Friends of the Chair",
completed as this paper was being written, appears
in the Annex. The list of indicators that have
been identified by the "Friends of the Chair"
is divided into three tiers, with the first
tier being the most important, and the second
and third tiers having lower priority, even
though they are all necessary for providing
a complete picture of the state of development
of a country. Indicators on human rights or
on governance were found to be either of a non-statistical
nature or as having inadequate technical validity;
therefore they have been excluded from the framework.
The "Friends of the Chair" recommended that
further work be done on indicators in these
two fields. The Working Group's attention is
drawn to the recommendations of the "Friends
of the Chair" which are summarized after paragraph
110 of the Annex (page 31). The Working Group's
views and comments would be valuable input to
discussions on the topic at the next session
of the Statistical Commission in March 2002.
6. Development Indicators have already been
discussed at previous sessions of the Committee
on Statistics and of the Working Group of Statistical
Experts. The main conclusions that emerged on
this topic are as follows:
- National Statistical
Offices (NSOs) of the region have scarce resources
that must not be diverted from national priorities;
- Analysis and policy making
require statistics of good quality;
- International organizations
mandated to monitor progress towards the development
goals should not increase the data provision
burden on member countries, and every attempt
should be made to reduce it;
- International organizations
must coordinate among themselves and avoid
overlap of work;
- International organizations
and their regional agencies should give priority
to strengthening statistical capacity of those
members which need technical assistance in
developing their own statistical systems;
- Among the various lists
of indicators none emerged as uniformly better
than the others, but it was noted that the
Common Country Assessment (CCA) had the widest
coverage among the frameworks that were examined;
- Composite indicators,
such as the Human Development Index (HDI)
of the United Nations Development Programme
(UNDP), have a role in advocacy, but are of
limited utility as far as decision making
and analysis are concerned, and therefore
need to be complemented by other sets of statistics;
- Countries need to be
involved in the process of selecting the indicators
to monitor the development goals;
- The selected indicators
must take into account the needs of national
as well as international users;
- Comparisons of living
standards should be based on Purchasing Power
Parities (PPPs).
II.
ISSUES FOR DISCUSSION
A.
Compliance
7. One of the issues which needs particular
attention is that of compliance by signatory
countries on the compilation of indicators for
monitoring the progress towards the agreed development
goals. Governments that subscribe to international
declarations, such as the development goals,
should also provide their national statistical
systems with adequate resources for carrying
out the necessary monitoring work. The Working
Group might wish to recommend to governments
of the region that in order to achieve full
compliance with the goals committed to, their
statistical offices should be supplied with
the necessary funding support.
8. This particular issue also calls for more
dialogue between statisticians and policy-makers,
as part of this political process; statisticians
could provide at an early stage the needed technical
support in identifying what is easily, or at
least feasibly, collected and what is not. Thus,
the issue of dialogue at national level between
official statisticians and policy makers remains
of prime importance.
9. Another issue of importance is the potential
conflict that might arise between country priorities
and the three-tier classification recommended
by the "Friends of the Chair". There may well
be, for instance, situations where indicators
belonging to a tier with lower priority in the
classification have the highest importance in
a particular country's context. The converse
is also possible. Thus the three-tier classification,
while it tries to take into account information
needs from a global point of view, could not
be expected to be perfectly suited to all countries.
The Working Group might be interested to comment
on these potentially conflicting needs.
B.
Standardization and harmonization
10. Development Indicators serve two main categories
of users: national and international. In designing
national plans, or in monitoring the effectiveness
of policies, indicators must be consistent among
themselves, at least within the national context.
One of the problems that countries may face
is that of internal consistency, which usually
has implications at international level as well.
Problems arise when different national agencies
collect data and compile statistics, in the
process often not adhering to international
standards, which are not strictly necessary
for national use only. When those indicators
need to be used at international level, differences
in definitions, practices, quality of sources,
and so forth, have a strong impact on the comparability
of statistics.
11. With increasing globalization, countries
are increasingly interested in how they stand
in the international community. It is important
that indicators be more closely harmonized among
countries. Actually, such adherence of concepts
and methodologies to international standards
already exists, and is widely accepted as a
good characteristic, in economic statistics
fields like trade statistics or national accounts.
Harmonization is needed between other areas
of statistics too: social, demographic, and
environmental statistics are fields where stronger
efforts at standardization need to be made.
12. The Working Group might wish to reaffirm
that Development Indicators should be harmonized
in such a way that they serve equally well both
national and international needs.
C.
Compilation of indicators
13. At the Manila meeting mentioned in
paragraph 4, it was noted that not all countries
compile the indicators that have been selected
for the monitoring of development goals. Sometimes,
perhaps because of lack of interest among users,
the indicators are not compiled, even though
the raw or the underlying data are already available
to the statistical offices. In these cases a
simple ratio, or other basic manipulation of
raw data, could supply the missing indicators.
In other cases, relatively sophisticated or
time-consuming operations would be needed in
order to compute the indicators. Some national
statistical offices in the region have also
highlighted the fact that sometimes data are
available to line ministries, or other government
agencies, but not to the NSOs; in such cases
it is crucial that good collaboration and communication
between government agencies is pursued, especially
in the context of decentralized statistical
systems. In other instances slight changes to
existing survey questionnaires might enable
the compilation of indicators, but resistance
to changing long-running series has to be overcome
if statistical series relevant to today's needs
are to be produced. In all such cases, there
seems to be a need for a national statistical
coordinating agency, which should be enabled
by means of an adequate legal framework to manage
the compilation of statistical indicators for
the country; this would help to ensure that
indicators are reliable, timely, relevant and
used for monitoring national policies.
14. Therefore, the Working Group might
comment on the opportunity for countries to
broaden the compilation of indicators to those
cases where only slight changes to current practices
would be needed. The Working Group might also
wish to invite countries to promote arrangements
for exchange of data among government agencies,
and to attach sufficient metadata to their statistics.
D.
Reporting
15. Reporting arrangements on Development Indicators
have not been finalized yet, but given the fact
that the United Nations Statistics Division
has to report to the General Assembly on the
level of achievement of the development goals,
they will certainly be heavily involved in the
international collection of data in this field.
16. Several of the Development Indicators already
feature in the data collection activities of
UNSD and of other international statistical
players. In these cases there will be no additional
reporting burden on countries.
17. Indicators that are not part of existing
data collection activities will also have to
be reported. In order to avoid the risk of overburdening
countries, international organizations should
expand the existing data sharing arrangements
that aim to minimize multiple requests for the
same statistics.
18. The Working Group may note that, on their
part, countries should also assign an agency
to report data to the designated international
collector. Government agencies will anyway keep
their existing relationships with specialized
international organizations like the World Trade
Organization (WTO), the United Nations Children
Fund (UNICEF), and so forth, and arrangements
are necessary in order to have coordinated relationships
with international organizations.
III.
THE ROLE OF ESCAP
19. The Working Group is invited to comment
on the best role for the ESCAP secretariat in
this field. The international and national roles
seem to be well assigned, as the United Nations
Statistics Division has been supporting the
efforts of the "Friends of the Chair" expert
group towards the identification of a unique
set of indicators able to serve the developmental
goals set out in United Nations global conferences
or in the Millennium Declaration. An additional
role of UNSD would seem to be that of international
compiler. At national level the United Nations
Development Programme might facilitate the collection
of raw data through the Country Common Assessment
set of indicators, and possibly integrate these
raw data with other sources available from countries.
Data would be then collected, processed and
disseminated at international level by UNSD.
20. This leaves the statistical offices
of the Regional Commissions, such as the Statistics
Division of ESCAP, with a role that in practical
terms remains to be fully elaborated. What emerged
from the "United Nations Workshop on Development
Indicators for ASEAN countries" was that the
role of regional commissions could be two-fold.
The first important contribution that regional
commissions could give to the process could
be on controlling and supplying metadata. This
would be needed as a support for the enormous
amount of data and information that UNSD will
have to collect from all over the world. It
was felt that support in collecting metadata
information by regional commissions, which are
closer and know better the characteristics of
their regions, could be a useful role. The second
role would be continuing the regional programmes
of statistical capacity building. In fact it
was recognized that only sustainable capacity
in statistics could enable countries to produce
the needed indicators. Therefore, Regional Commissions
should pursue initiatives that are designed
to strengthen statistical capacity rather than
promoting ad hoc data collection initiatives.
Additionally, the ESCAP secretariat believes
that it could integrate the first tier of Development
Indicators in its regular publications. Further,
as described in document STAT/WGSE.12/2, the
Poverty Centre/Unit to which the Statistics
Division is contributing will be undertaking
analytical work to make country-level indicators
more comparable for the purpose of regional
synthesis.
21. The secretariat suggests that it
might pursue improvement of the coordination
of its activities, both data and statistical
capacity building work, with the regional arms
of concerned international organizations. Furthermore,
ESCAP will continue to coordinate its statistical
capacity building activities with organizations
operating in the region such as ASEAN, the Asian
Development Bank (ADB), and the Secretariat
of the Pacific Community (SPC). Views on the
desirability and feasibility of extending this
traditional coordination among regional statistical
players to monitoring progress towards the Millennium
Development Goals are also sought.
22. The Working Group may wish to comment
and advise the secretariat on the above aspects
of its role, and on any other means by which
ESCAP might contribute to the Development Indicators
process.
IV.
CONCLUDING REMARKS
23. The Working Group's attention is drawn
to the recommendations of the "Friends of the
Chair" which are summarized after paragraph
110 of the Annex (page 31). The Working Group's
views and comments would be valuable input to
discussions on the topic at the next session
of the Statistical Commission in March 2002.
- Para 5.
24. The Working Group might wish to recommend
to governments of the region that in order to
achieve full compliance with the goals committed
to, their statistical offices should be supplied
with the necessary funding support - Para 7.
25. The Working Group might be interested to
comment on the potential conflict between country
priorities and the three-tier classification
recommended by the "Friends of the Chair" -
Para 9.
26. The Working Group might wish to reaffirm
that Development Indicators should be harmonized
in such a way that they serve equally well both
national and international needs. - Para 12.
27. The Working Group might comment on the
opportunity for countries to broaden the compilation
of indicators to those cases where only slight
changes to current practices would be needed.
It might also wish to invite countries to promote
arrangements for exchange of data among government
agencies, and to attach sufficient metadata
to their statistics. - Para 14.
28. The Working Group may invite countries
to enhance their arrangements for having coordinated
relationships between their various statistical
units and international organizations. - Para
18.
29. The Working Group may wish to comment and
advise the secretariat on its suggested role
and on any other means by which ESCAP might
contribute to the Development Indicators process.
The views of the Working Group on the desirability
and feasibility of extending the traditional
coordination of activities among regional statistical
players to monitoring progress towards the Millennium
Development Goals are also sought. - Paras 21
and 22.
Annex
AN ASSESSMENT OF THE
STATISTICAL INDICATORS DERIVED FROM UNITED NATIONS
SUMMIT MEETINGS1
Prepared by
Friends of the Chair
of the United Nations Statistical Commission
for the 2002 UNSC Meeting, November, 2001
This
report of the "Friends of the Chair" expert group
on Development Indicators is to be considered
final although some parts of it would need to
be further edited and completed. The report is
provided to the Working Group of Statistical Experts
in order to clarify the process followed, the
decisions taken and the recommendations made by
the "Friends of the Chair". The report will be
posted on the Statistical Commission's web site,
together with the documentation for the 33rd session
of the Commission, at http://www.un.org/Depts/unsd/statcom/sc2002.htm.
EXECUTIVE
SUMMARY:
1. This report results from a request from
the Economic and Social Council (ECOSOC) to
the UN Statistical Commission (UNSC). It arose
out of a concern about the large number of seemingly
uncoordinated demands for statistical indicators
to monitor a wide range of policy issues that
had been agreed at various UN summits and major
conferences. At the 2000 meeting (E/CN 3/2000/27)
ECOSOC turned to the Statistical Commission
(UNSC) as its authoritative technical advisory
body to:
- Provide leadership in
the field of conference indicators;
- Conduct an in-depth technical
analysis of conference indicators;
- Make recommendations
regarding a limited list of conference indicators,
and
- Develop and recommend
to the Council a mechanism of statistical
review for future proposed indicators.
2. The technical assessment was carried out
on over 280 statistical indicators derived from
UN summits and major conferences held over the
last 10 years. Seven expert groups were established
to cover the common division of policy (e.g.
economics, health, education etc) that is reflected
in Ministerial responsibility in most countries.
The expert groups had members drawn from many
countries. The indicators were assessed on technical
criteria and the relevance to the policy goals.
A web site has been created that contains all
of the indicators and the technical assessment
of each.
3. In response to the request for a limited
list of conference indicators, we propose an
indicator framework containing three priority
tiers. Each tier contains about 50 statistical
indicators. In addition a further category contains
indicators that would be useful for a more detailed
understanding of any policy area. The framework
is arranged to reflect the major policy areas
referred to above. However important additional
policy areas cut across this arrangement and
typically cut across Government Department policy
responsibilities in many countries. Such policy
areas include Poverty, Gender and Child Welfare.
Indicators covering these issues are contained
within the framework.
4. Also there are areas in which the indicators
need improvement or indeed simply do not exist
and need to be developed (e.g. indicators for
Human Rights and Good Governance). These tasks
were too extensive to undertake in the time
available. However we make recommendations to
the UNSC to establish processes to do this.
5. A correspondence between the proposed framework
and the existing high-level indicator sets is
provided.
6. The development of statistical indicators
and the statistical capacity that allows higher
standards to be met are dynamic. Initiatives
exist within international agencies that will
require the proposed framework to be reviewed
if it is to remain relevant to changing needs.
Hence the framework must be kept under review
and we make recommendations to achieve this
and to improve co-ordination between international
agencies.
7. Finally we turn to the question of future
summits and major conferences and the need to
propose mechanisms that will allow further development
of the framework in response to emerging needs.
The existing arrangements for indicator development
are clearly unsatisfactory. We recommend procedures
to improve this situation.
8. These are based on the recognition that
the stakeholders in the indicator programme
span policy officials and statisticians in both
international organisations and member states.
Mechanisms are needed to ensure that all can
play a full part in indicator development and
priority setting. The process of turning a policy
goal into a statistical indicator that is feasible
to measure and technically sound should involve
all stakeholders.
9. Another important issue is the level of
statistical capacity needed for countries to
support the information needs of national and
global policies. Developing statistical capacity
goes beyond financial and technical support
from international donors narrowly focussed
on specific statistical production to monitor
a specific global policy. It calls for more
support for systemic development.
10. A further issue is the reconciliation of
information needs for national and global purposes.
In the long run, financial support for statistical
programmes must depend upon national rather
than international provision. This in turn depends
upon national governments using and valuing
statistical information in support of policy
development, policy monitoring and good public
administration in general. Hence it is essential
that the national statistical system supports
national policy goals.
11. The report contains a series of recommendations
that are intended to address these issues. In
particular mechanisms are proposed to ensure
greater participation for member states in the
development and adoption of statistical indicators
for global and national purposes.
1.
INTRODUCTION
12. In the last decade or so United Nations
summits and major conferences (averaging almost
two per year) have covered a wide range of economic
and social issues. These meetings have resulted
in declarations related to future goals and
targets that have been endorsed by member states
and are intended to improve the wellbeing of
the world's population. Goals and targets call
for a commitment to monitor progress towards
them and, consequently indicators (usually statistical
indicators) have been identified in relation
to each goal. The intention is to monitor and
report on these so that progress towards the
declared goals and targets can be measured.
13. However, there is concern that this process
has gone on with too little co-ordination between
officials concerned with the separate UN summits
and major conferences in terms of the number
and choice of indicators to be monitored. The
meetings have varied considerably in terms of
the number of resulting indicators (ranging
from a handful or less to as many as 70 being
identified from a single UN conference). In
total over 280 indicators have been identified.
14. The perception is that this uncoordinated
process has resulted in a plethora of indicators
of different levels of importance in policy
terms. Also there is potential for confusion
among users because of an apparent inconsistency
and lack of coherence among the indicators.
The ongoing addition of indicators has resulted
too in a large demand for statistical information
from each member state: a demand that has to
be set alongside the demands for statistical
information for national policy purposes. For
countries with less well-developed statistical
infrastructure this total demand can be disproportionate
to the resources available to meet it.
15. Attempts have been made to distil core
sets of indicators that might be afforded greater
recognition and therefore higher priority. The
United Nations Statistical Commission (UNSC)
identified the Minimum National Data Set (MNDS:
15 indicators). The OECD Development Assistance
Committee - in co-operation with the UN, World
Bank and IMF - identified the International
Development Goals (IDG: 21 indicators). This
set drew heavily on international summits up
to 1995. The United Nations Development Group
identified indicators to support Common Country
Assessment again based on an analysis of the
requirements of UN summits (UNDAF-CCA: 57 indicators).
Similarly the need to promote and assess sustainable
development has resulted in an additional set
(CSD: 57 indicators). There is also Basic Social
Services for All (BSSA: 12 indicators). The
group has also been aware of the work within
the European Union on 35 Structural Indicators.
16. And this process goes on. While this report
was in preparation the choice of statistical
indicators to support the UN Millennium Goals
was announced (MDG: 48 indicators), and constitutes
another high-level set of indicators which will
be monitored.
17. The Economic and Social Council (ECOSOC)
considered these issues in 1999 and 2000 and
there is a general recognition that better co-ordination
is needed and that full participation and ownership
by Member States was needed in all stages of
indicator development. At the 2000 meeting (E/CN.3/2000/27)
ECOSOC turned to the Statistical Commission
(UNSC) as its authoritative technical advisory
body to:
- Provide leadership in
the field of conference indicators;
- Conduct an in-depth technical
analysis of conference indicators;
- Make recommendations
regarding a limited list of conference indicators,
and
- Develop and recommend
to the Council a mechanism of statistical
review for future proposed indicators.
18. As a consequence the UNSC, at its 2001
meeting, established a group of 'Friends of
the Chair' to consider the issues further and
to report back to the UNSC at the 2002 meeting
so that the UNSC could report to ECOSOC.
19. The members of the group were:
Tim Holt (United Kingdom): Chair
Guest Charumbira (Botswana)
Claudia Cingolani (Italy)
Francisco Guillen (Mexico)
Hasan Abu Libdeh (Palestine)
Jil Matheson (UK)
Yue Renfeng (China)
Hussain Shakhatreh (Jordan)
Bounthavy Sisouphantong (Laos)
Ken Tallis (Australia)
20. In order to carry out the required in-depth
technical review the group subdivided the 280
identified indicators into 7 domains:
- Demography,
- Health and Nutrition,
- Environment and Energy,
- Economics and Poverty,
- Employment and Labour,
- Education, and
- Other Social Indicators.
Seven indicators related to Human Rights and
Good Governance were excluded from this framework.
This was because all of the indicators were
qualitative in nature and no statistical indicators
had been identified. We return to this point
in due course.
21. The Domains represent major divisions of
policy responsibility that are commonly reflected
by separate Ministries in many countries (e.g.
Economics, Health, Education etc.). Additionally
there will be important cross-cutting policy
areas such as Poverty, Child Welfare or Gender
that are distributed across these Domains.
22. We considered the requirement to make recommendations
about a limited number of indicators and, following
the UNSC discussion in 2001, approached this
by establishing a hierarchy of indicators containing
three priority tiers and a category of 'additional'
indicators. The first tier contains statistical
indicators that might be regarded as of the
highest priority and are essential for broad
monitoring. This tier includes a small number
of indicators in each domain. The second and
third tiers contain additional indicators that
progressively add to the overall picture and
include indicators that allude to additional
policy priorities. A fuller description is given
in section 3.
23. We are very aware of the need for countries
to reconcile their statistical needs for national
policy purposes with the international requirements.
The hierarchical structure offered is not meant
to be mandatory nor to impose a straightjacket
on member states although we think that all
countries should be encouraged to compile all
indicators within the first tier unless there
are overwhelming national reasons not to do
so. In our view the second tier and many of
the indicators in the third tier would be valuable
in most countries. However, it is likely that
countries with particular concerns or policy
initiatives would wish to collect extensive
statistics for some domains (including those
in the 'additional' indicator category) and
less for others. Also the statistical requirements
for national policy purposes - in effect most
of the output of the national statistical system
- would go beyond the indicators identified
in the framework. Nonetheless the framework
is intended to enable countries to assess their
statistical priorities and to reconcile the
statistics that are needed for national purposes
with the global requirements. As such we hope
that countries will find the framework useful.
24. For each domain an expert group was established
drawn from member states across the world. Some
members of each expert group were official statisticians
and others were more concerned with policy issues.
25. In addition useful discussions were held
with representatives of UNSD, UNFPA, UNESCO,
ILO, OECD and World Bank and we attended the
ACC Subcommittee on Statistical Activities meeting
in September 2001 at which an initial draft
of our report was discussed.
26. As a further consultation phase a draft
version of the report was circulated to all
National Statistical Offices, UN Regional Commissions
and International Agencies. It was also placed
on the UNSD web site. The final version of the
report takes account of the responses received
to this consultation. Representatives of a number
of international agencies also attended the
meeting of the Friends of the Chair at which
the draft report was effectively finalised.
We acknowledge and thank all contributors but
the final responsibility for this report rests
with the Friends of the Chair.
2.
KEY ISSUES
27. The request from ECOSOC to the UNSC and
the terms of reference established for the Friends
of the Chair reflect concern over the current
process for identifying indicators. This concern
includes the lack of co-ordination between stakeholders,
insufficient involvement by member states in
the process and the lack of structure of the
resulting indicator sets. A number of key issues
need to be recognised and taken into account.
2.1
The Stakeholders, Competing Needs, Statistical
Capacity and The Burden on Countries
The Stakeholders
28. Identifying statistical indicators for
monitoring purposes is neither a pure policy
nor a pure statistical issue. The basic expression
of the policy goal must drive the monitoring
requirement but turning that expression into
a statistical indicator that will be relevant,
reliable and acceptable to the various stakeholders
is a statistical function. The tension between
the policy view of what is needed and the statistical
view of what is feasible and technically sound
should be resolved by joint determination.
29. A second stakeholder issue is that although
the statistical indicators that are derived
from UN conferences and summits are motivated
by international needs, they are based on policy
issues that have to be reflected in the national
policy agenda if the desired progress is to
be achieved. However, there can be differences
between national and international priorities
and the need to reconcile the national and
international priorities needs to be addressed.
30. The third stakeholder issue rests on the
simple fact that most of the statistical indicators
are derived from national statistical programmes.
These are predominantly funded from national
resources and reflect a range of user needs
of which the international need is but one.
It falls to national statisticians to try to
respond to often disparate user needs within
the resources available. Their ability to respond
will depend heavily on the general level of
statistical capacity in the country and
the extent to which additional demands create
a response burden on countries or whether
existing statistical sources can be used or
adapted to meet additional needs. Thus national
statisticians are stakeholders. Their expertise
is different from statisticians working within
international agencies and they have an important
contribution to make to the process of developing
statistical indicators.
National and International
Priorities
31. Relevance is a dominating requirement of
statistical information. If the statistics are
not relevant to the policy need, then they will
not command the attention, nor have the impact
that they should. Failure to meet national needs,
in particular, will undermine the requirement
to develop sustainable statistical capacity
since in the long term this must depend on national
governmental funding and support. It will also
undermine evidence-based policy as a basis for
good governance and public administration within
countries. From the UN perspective this would,
as a consequence, undermine the provision of
statistical indicators for international monitoring
purposes.
32. To an extent the tension between national
and international needs may be reduced if the
statistical system is rich enough and flexible
enough to support diverse needs. For example
a well-designed Household Budget Survey can
estimate the proportion of the population below
an international poverty standard and against
a national poverty standard. In such cases the
conflict between national and international
requirements is avoidable. In other cases the
resolution may call for additional resources
- to collect a wider range of data, or to fund
larger sample sizes so as to meet competing
needs. In our view all efforts should be made
to reconcile national and international needs
so as to support both. This implies that countries
recognise the need to support international
needs and international agencies accept the
need to support statistical activities focussed
on national, as well as international, needs.
Investment in modular frameworks or analytical
capacity that allows countries to exploit core
sets of survey data for a variety of purposes
would be valuable.
33. Thus any rationalised set of indicators
should be applicable (or readily adaptable)
to both national and international priorities.
In the time available we have not been able
to assess this as comprehensively as we would
have wished although we have drawn upon the
experience of the members of the expert groups
and international agencies. In our view this
assessment should be done more systematically
before the proposed framework of indicators
and their priority levels are 'set in stone'.
The recommendations that we make for the UNSC
to maintain the indicator framework will permit
this.
Statistical Capacity
34. The ability to produce consistent, reliable
statistical information on an ongoing basis
requires a sustained statistical capacity. This
requirement is not a one-off capability but
implies the ability to produce statistics on
a regular basis and with the timeliness needed.
35. In particular a sound statistical infrastructure
is essential. By this we mean:
- Underpinning systems
to create and maintain sampling frames for
business and household surveys.
- A critical mass of ongoing
statistical activities: survey design, data
collection and analysis in order to nurture
the basic professional skills.
- The technical and professional
capacity to maintain and develop systems in
accordance with international standards as
these are developed over time.
- A developed analytic
capacity.
- Adequate statistical
frameworks and IT infrastructure.
- Good management to make
the most use of the resources that are available.
- All of the above embedded
within a wider legal and administrative structure
that recognises the importance of good statistical
information and the need to sustain the conditions
in which it can be produced with high professionalism
and integrity, consistent with the UN Fundamental
Principles of Official Statistics.
36. Without this core capacity and the ongoing
resources to support it, neither the statistical
needs of the country nor those of the international
community will be reliably served. In many countries
adequate ongoing financial support is a key
issue. Where this core capacity is fragile the
sporadic provision of additional funds to satisfy
a particular statistical need will be much less
effective and is no substitute for what one
might term 'statistical sustainability'.
37. In this regard, statistical indicators
need to be viewed as the end product of often
complex statistical infrastructures that are
essential if the indicators are to be produced
with adequate quality. Population estimates
for example, that are fundamental to many indicators
that are expressed as rates or per capita estimates,
depend on periodic Censuses to provide benchmarks
and on systems of vital registration or other
sources to permit inter-censal population estimates.
Many social statistics depend upon social surveys
that need sustained expertise if they are to
be well conducted. Complex measures such as
GDP require an extensive framework of business
surveys, administrative sources and underpinning
infrastructure if the statistics are to be of
adequate quality. Too much emphasis has been
placed on the indicators (which are the end
product) and too little on the statistical sources
and infrastructure that underpin these. The
majority of aid agencies and donors are perceived
to provide aid to conduct studies needed to
fulfil their objectives without considering
the national capacity building.
38. Countries and international donors need
to recognise that each statistical initiative
depends on the core statistical capacity within
the country and that internationally sponsored
activities must contribute to this sustainable
capacity. It is essential that these activities
support both national and international statistical
needs rather than being perceived as being driven
by international goals alone. The effective
use of statistical information within national
governments needs to be promoted and ECOSOC
and international donors have an important role
to play. This is essential if the statistical
system is to command consistent financial and
political support from the national government
of the day.
39. It is important to note that donor resources
are often tied to specific (international) objectives
while being characterised as supporting statistical
capacity building. Whilst these may provide
financial support there is a frequently expressed
concern that such programmes may consume the
statistical expertise available within the country
and so distort the overall priorities. If this
is so it represent not statistical capacity
building but statistical capacity diversion.
It is important the donor-supported programmes
genuinely add to the sustainable resource within
the country.
40. We believe that an indicator of statistical
capacity should be developed and monitored.
This measure could be based on the level of
regular statistical activity within a country,
an ongoing critical mass of survey taking and
statistical analysis and the existence of basic
elements of statistical infrastructure. A task
team within the PARIS21 initiative has this
work in hand, building on the IMF's Data Quality
Assessment Framework. One concern is that this
initiative, and the resulting indicator should
not be dominated by economic statistics but
should span the wide range of statistical areas
covered by national statistical systems and
the indicators considered in this report. Also
the membership of the task force has no country
representation. We RECOMMEND that this be
remedied and that the eventual proposals be
made to the UNSC.
41. Building and monitoring statistical capacity
is a systemic issue. In our recommendations
we have taken account of this in several ways.
First we have focussed on indicators (especially
in the first two tiers of the framework) that
should be feasible for most countries to compile
(perhaps initially with statistical assistance
but as part of the ongoing statistical programme
in due course). Second we propose a systematic
assessment of the availability and frequency
of indicators in the priority categories. Third
we have in some cases defined a sequence of
successive approximations to ideal indicators
that countries might compile as their statistical
capacities develop. We commend this approach
for the maintenance and development of the framework.
The Response Burden on
Countries
42. A frequently heard concern is that the
uncoordinated demand for a wide range of statistical
indicators places a burden on National Statistical
Offices (NSO's) that cannot be responded to.
Linked to this view is that such a burden is
incompatible with the national statistical needs
and diverts scarce resources (skills as well
as finance) from other priorities. NSO's generally
wish to respond to all expressed needs so long
as these are technically well founded but the
concern is that these cannot be met within the
resources (both financial and skills) available.
43. International agencies have taken steps
in recent years to align their statistical requirements
and to improve the co-ordination when requesting
statistics from countries, particularly by establishing
joint data collection mechanisms. This process
should go on with a view to streamlining the
demand on countries further.
44. There are two solutions to the general
problem of burgeoning demand: to reduce the
demand or increase the resources and hence the
statistical capacity. The latter would serve
user needs better and is preferred but in the
short term both are needed:
Managing demand
A number of steps will help:
- Reconciling the international
and national statistical requirements will
reduce the burden.
- Establishing a hierarchical
structure of statistical indicators so that
nations may determine their priorities more
systematically.
- Producing more guidance
on best practice and measurement processes.
- Further co-ordination
between international agencies on data needs
and joint data collection from member states.
- Increasing Resources
and Enhancing Capacity
- But in addition increasing
the funding available for the less well-developed
statistical offices is essential and this
will be needed on an ongoing basis. In the
long term this must come from within the country
but in the short term often comes as a partnership
between national governments and international
donors.
- A climate of support
for the statistical system within the country
will be developed only if national governments
see statistical information as essential in
support of national policies and good governance.
In seeking efficient and effective public
administration governments need to view statistics
as part of the solution rather than simply
as an additional claim on public expenditure.
- Developing a core statistical
infrastructure and a critical mass of professional
and technical skills is essential.
- In the case of the donors,
they must ensure that all statistical activities
strengthen the sustainable statistical capacity
and, by taking account of national needs,
strengthen the value that national governments
place on statistics.
45. The resource implications for new statistical
outputs may be very different in different countries
and depend on the existing level of statistical
capacity. From the lowest additional cost to
the highest one may set out a hierarchy of resource
implications.
- In some cases it is simply
a question of analysing existing data in a
different way in order to provide the required
output. An analysis by gender is an example
of this so long as the basic information on
the subject's gender is available for each
data record. In such cases the resource requirement
(assuming professional skills are available)
is small.
- An approach more demanding
of professional skills is the use of modelling,
synthetic estimation and other analytical
techniques applied to exploit existing data
sources for new purposes. The financial cost
may be low but the technical knowledge to
produce high quality outputs is significant.
- In other cases the new
requirement may call for a small number of
additional items to be collected and analysed
using an existing survey. The resource implications
are a little higher but so long as the core
statistical capacity is in place it is generally
feasible to support the requirement.
- More seriously the new
requirement may call for a substantial increase
in the sample sizes employed. Regional and
other sub-national estimates that are often
required for national purposes are a good
example of this; estimates of population subgroups
is another. This can add significantly to
the existing costs and the need for analytic
skills.
- Even more seriously,
the new requirement may call for an entirely
new data collection system: for example a
new household survey or a new business survey.
This is generally an order of magnitude more
demanding in terms of time for development,
in terms of costs including interviewer and
data processing costs, and also in terms of
diverting often scarce professional and technical
skills from existing programmes to the new
survey. In order to avoid this issue, there
are examples where existing surveys become
overburdened with competing and potentially
conflicting data requirements to the extent
that one must question whether they are manageable.
Also the burden on the respondents who participate
in the survey is very severe.
- Where the primary data
source is an administrative system new needs
may call for the system (or the underpinning
software) to be redeveloped. This can be a
major undertaking unless the administrative
system is being redeveloped for other purposes
but for some statistical uses this may be
the best long-term strategy for a statistical
office.
- Finally some new requirements
may call for an infrastructure which simply
does not exist in a particular country. For
example some administrative systems (e.g.
vital registration) may be non-existent or
in such poor state that their use for statistical
purposes is impractical. Or measurement processes
(for example as are often used for some environmental
indicators) may not exist. In such cases the
basic infrastructure must be established and
this can be a long and expensive process.
46. In general the better the core statistical
infrastructure the better a country can respond
to new statistical requirements. If the national
and international goals are to be met then a
strengthening of the core is required in many
countries.
47. In particular the statistical infrastructure
to support estimates of GDP and vital statistics
is particularly demanding and complex. Ideally
it requires both survey capability and access
to effective administrative systems as data
sources. Both are cornerstones of the whole
indicator programme since many indicators make
use of these.
48. Hence we make a set of inter-related RECOMMENDATIONS:
2.2
Quality and Technical Properties, Continuity
and Change
Quality and Technical
Properties
49. It is important that the chosen statistical
indicators are relevant to the purpose and satisfy
technical criteria. Measurement for statistical
purposes is an exacting discipline, calling
for specialist development. Definitions and
concepts need to be as precise as possible consistent
with the intended use. The resulting statistics
need to satisfy statistical quality criteria
and conform to international standards where
established. The development of good quality
statistical indicators takes time and may well
require field tests and evaluation before a
suitable indicator is developed.
50. Over the years, largely independent of
the need to monitor Conference goals countries
have developed suites of core statistics such
as population estimates, GDP or life expectancy.
These have been developed through extensive
processes over time; international guidelines
exist to support best practice and the statistical
properties are relatively well understood. So
long as such indicators are relevant to the
conference goal they are readily available for
monitoring purposes. Nevertheless, even for
indicators such as these, actual quality varies
between one country and another depending on
the strength of the statistical infrastructure
in each country and the basic statistical capacity.
51. But for new policy areas such as Human
Rights and Good Governance no established statistical
indicators exist. Their development will take
time and the process needs to involve statisticians
and policy officials.
52. An additional difficulty for some newly
developed indicators is that targets related
to future improvements from a baseline date
may be agreed (for example reducing by a third
the incidence of a particular event within a
period of 10 years). If the statistical indicator
that is used to monitor this target is not widely
available at the baseline time then there is
no base value from which to measure progress.
There is no easy solution to this problem but
when such targets are adopted there is a need
for the conference to recognise the need to
support the development of baseline measures.
If not, it risks bringing discredit to the whole
process of target setting. We RECOMMEND that
the need for baseline measures be taken into
account when targets are adopted that require
change to be measured from a specific point
in time.
Continuity and Change
53. For all statistics there needs to be a
regular process of review and development. As
the economic and social environment change so
the statistics that are used to monitor development
need to change if they are to capture the new
situation and so remain relevant. This is as
true for statistical indicators monitoring goals
as it is for all other statistics. If this process
of review and renewal does not occur the statistical
indicators will become increasingly less relevant.
For global statistics there is another reason
for continuous development. The need to establish
an indicator quickly may reasonably mean that
technical standards are chosen to reflect the
reality of what can be achieved in the short
term. However, as statistical capacity develops
the technical standards that one may apply to
any indicator may be increased: definitions
may be refined and the quality of the indicator
at a global level improved. This process creates
a tension between continuity over time and necessary
change to improve quality and relevance. This
balance needs to be recognised and often will
call for continuity but there are established
methods, such as statistical revisions, to address
the need for consistency of time series.
54. We RECOMMEND
- that all statistical
indicators should be subject to periodic review
and improvement and
- that when such a review
results in change, an approach be provided
to support countries in moving to the improved
indicator while maintaining continuity with
the recent past.
3.
A TECHNICAL ASSESSMENT AND FRAMEWORK FOR INDICATORS
55. From the UN Summits of the 1990's about
280 separate indicators were identified of which
the overwhelming majority were statistical in
nature. This list was based on 15 global conferences
listed in the 1999 Report of the Secretary-General.
In consultation with the ECOSOC secretariat
this list was augmented to take account of the
World Conference on Education for All (Dakar
2000) and special sessions of the General Assembly
that followed other conferences (e.g. Fourth
World Conference on Women, Beijing 1995) up
until March 2001. The list included indicators
that were identified by cross-conference initiatives:
MNSDS, UNDAF-CCA, BSSA and IDG. After this no
more conferences were added but the 'road map
towards implementation of the UN Millennium
Declaration' was included.
56. These indicators covered a wide range of
topics. But this set, albeit large, does not
include all of the statistical indicators that
have been identified as desirable by the UN
and other international organisations. It includes
only those indicators identified from the UN
summits and major conferences. Future meetings
will surely identify new areas that require
policy monitoring. We address this through the
section on future processes. This section is
essentially concerned with the 280 indicators
identified.
The Expert Groups and
their Task
57. As described in paragraph X
the indicators were subdivided into 7 Domains
and expert groups established for each domain
(Demography, Health | 
