4. STRENGTHENING FEDERAL STATISTICS
Federal statistical programs produce key information
about a range of topics of interest to public and private
decision makers, including the economy, the population,
agriculture, crime, education, energy, the environment,
health, science, and transportation. The ability of governments,
businesses, and citizens to make appropriate
decisions about budgets, employment, investments,
taxes, and a host of other important matters depends
critically on the ready availability of relevant, accurate,
and timely Federal statistics. The Federal statistical
community remains on alert for opportunities to
strengthen these measures of our Nation’s performance.
For example, during 2004, Federal statistical agencies
launched the first new economic indicator survey in
40 years—the Quarterly Services Survey (Census Bureau);
expanded regional economic data from 318 to
934 American communities (BEA); successfully adopted
new collection and processing technologies that will
greatly accelerate the release of data from the National
Health Interview Survey (NCHS); and completed the
first data collection on the cyberinfrastructure of academic
and biomedical facilities (NSF’s SRS).
For Federal statistical programs to effectively benefit
such a wide range of users, the underlying data systems
must be viewed as credible. In order to foster this credibility,
Federal statistical programs seek to adhere to
high quality standards and to maintain integrity and
efficiency in the production of statistics. As the collectors
and providers of these basic data, Federal statistical
agencies act as data stewards—balancing public
and private decision makers’ needs for information with
legal and ethical obligations to minimize reporting burden,
respect respondents’ privacy, and protect the confidentiality
of the data provided to the Government.
This chapter discusses the development of standards
that principal statistical programs can use to assess
their performance and presents highlights of their 2006
budget proposals.
Performance Standards
Statistical programs maintain the quality of their
data or information products as well as their credibility
by setting high performance standards for their activities.
The statistical agencies and statistical units represented
on the Interagency Council on Statistical Policy
(ICSP) have collaborated on developing an initial
set of common performance standards for use under
the Government Performance and Results Act and in
completing the Administration’s Program Assessment
Rating Tool (PART). Federal statistical agencies have
agreed that there are six conceptual dimensions within
two general areas of focus that are key to measuring
and monitoring statistical programs. The first area of
focus is Product Quality, encompassing the traditional
dimensions of relevance, accuracy, and timeliness. The
second area of focus is Program Performance, encompassing
the dimensions of cost, dissemination, and mission
achievement.
Statistical agencies historically have focused on measuring
performance in the area of product quality, especially
the dimensions most amenable to quantitative
measurement, specifically accuracy and timeliness. Relevance,
also an accepted measure of quality, can be
either a qualitative description of the usefulness of
products or a quantitative measure such as a customer
satisfaction score. Relevance is more difficult to measure,
and the indicators that do exist are more varied.
Program performance standards form the basis for
evaluating effectiveness. They address questions such
as: Are taxpayer dollars spent most effectively? Are
products made available to those who need them? Are
agencies meeting their mission requirements or making
it possible for other agencies to meet their missions?
The indicators available to measure program performance
for statistical activities currently are less well developed.
Product quality and program performance standards
are designed to serve as indicators when answering
specific questions in the Administration’s PART process.
Chart 4–1 presents each principal Federal statistical
agency’s assessment of the status of its current
and planned use of indicators on the six dimensions.
During the past year, four agencies (BTS, EIA, NCES,
and SRS) have improved the status of their indicators.
Use of the indicators may be for internal management,
strategic planning, or annual performance reporting.
The dimensions shown in the figure reflect an overall
set of indicators for statistical activities but the specific
measures vary among the individual programs depending
on their unique characteristics and requirements.
Annual performance reports and PARTs contain these
specific measures as well as additional information
about performance goals and targets and whether a
program is meeting, or making measurable progress
toward meeting, its performance goals. The examples
below illustrate different ways agencies track their performance
on each dimension.
54 ANALYTICAL PERSPECTIVES
Chart 4-1. ICSP Statistical Quality and Program
Performance Dimensions, 2006
Dimension BEA BJS BLS BTS Census EIA ERS NASS NCES NCHS ORES SOI SRS
Product Quality
Relevance
Accuracy
Timeliness
Program Performance
Cost
Dissemination
Mission
Achievement
P P P P P P
P
P
P Indicator in development Indicator Available
Description of Dimensions
Product Quality
Relevance: Qualitative or quantitative descriptions of the degree to which products and services are useful to users and responsive to users’ needs.
Accuracy: Qualitative or quantitative measure of important features of correctness, validity, and reliability of data and information products measured as degree of closeness
to target values.
Timeliness: Qualitative or quantitative measure of the timing of information releases.
Program Performance
Cost: Quantitative measure of the dollar amount used to produce data products and services.
Dissemination: Qualitative or quantitative information on the availability, accessibility, and distribution of products and services.
Mission Achievement: Qualitative or quantitative information about the effect of, or satisfaction with, statistical programs.
Key to Statistical Agencies
BEA = Bureau of Economic Analysis, Department of Commerce
BJS = Bureau of Justice Statistics, Department of Justice
BLS = Bureau of Labor Statistics, Department of Labor
BTS = Bureau of Transportation Statistics, Department of Transportation
Census = Census Bureau, Department of Commerce
EIA = Energy Information Administration, Department of Energy
ERS = Economic Research Service, Department of Agriculture
NASS = National Agricultural Statistics Service, Department of Agriculture
NCES = National Center for Education Statistics, Department of Education
NCHS = National Center for Health Statistics, Department of Health and Human Services
ORES = Office of Research, Evaluation, and Statistics, Social Security Administration
SOI = Statistics of Income, Internal Revenue Service, Department of the Treasury
SRS = Division of Science Resources Statistics, National Science Foundation
Product Quality: Statistical agencies agree that
product quality encompasses many attributes, including
(but not limited to) relevance, accuracy, and timeliness.
The basic measures in this group relate to the quality
of specific products, thereby providing actionable information
to managers. These are ‘‘outcome-oriented’’
55 4. STRENGTHENING FEDERAL STATISTICS
measures and are key to the usability of information
products. Statistical agencies or units establish targets
and monitor how well targets are met. In some sense,
relevance relates to ‘‘doing the right things,’’ while accuracy
and timeliness relate to ‘‘doing things right.’’
Relevance: Qualitative or quantitative descriptions
of the degree to which products and services are
useful and responsive to users’ needs. Relevance
of data products and analytic reports may be monitored
through a professional review process and
ongoing contacts with data users. Product relevance
may be indicated by customer satisfaction
with product content, information from customers
about product use, demonstration of product improvements,
comparability with other data series,
agency responses to customer suggestions for improvement,
new or customized products/services,
frequency of use, or responses to data requests
from users (including policy makers). Through a
variety of professional review activities, agencies
maintain the relevance, accuracy, and validity of
their products, and encourage data users and
other stakeholders to contribute to the agency’s
data collection and dissemination programs. Striving
for relevance requires monitoring to ensure
that information systems anticipate change and
evolve to appropriately measure our dynamic society
and economy.
Accuracy: Qualitative or quantitative measures of
important features of correctness, validity, and reliability
of data and information products measured
as degree of closeness to target values. For
statistical data, accuracy may be defined as the
degree of closeness to the target value and measured
as sampling error and various aspects of nonsampling
error (e.g., response rates, size of revisions,
coverage, edit performance). For analysis
products, accuracy may be the quality of the reasoning,
reasonableness of assumptions, and clarity
of the exposition, typically measured and monitored
through review processes. In addition, accuracy
is assessed and improved by internal reviews,
comparisons of data among different surveys, linkages
of survey data to administrative records, redesigns
of surveys, or expansions of sample sizes.
Timeliness: Qualitative or quantitative measure of
timing of information releases. Timeliness may be
measured as time from the close of the reference
period to the release of information, or customer
satisfaction with timeliness. Timeliness may also
be measured as how well agencies meet scheduled
and publicized release dates, expressed as a percent
of release dates met.
Program Performance: Statistical agencies agree
that program performance encompasses balancing the
dimensions of cost, dissemination, and mission accomplishment
for the agency as a whole; operating efficiently
and effectively; ensuring that customers receive
the information they need; and serving the information
needs of the Nation. Costs of products or programs
may be used to develop efficiency measures. Dissemination
involves making sure customers receive the information
they need via the most appropriate mechanisms.
Mission achievement means that the information program
makes a difference. Hence, three key dimensions
are being used to indicate program performance: cost
(input), dissemination (output), and mission achievement
(outcome).
Cost: Quantitative measure of the dollar amount
used to produce data products or services. The
development and use of financial performance
measures within the Federal Government is an
established goal, and the intent of such measures
is to determine the ‘‘true costs’’ of various programs
or alternative modes of operation at the
Federal level. Examples of cost data include full
costs of products or programs, return on investment,
dollar value of efficiencies, and ratios of
cost to products distributed.
Dissemination: Qualitative or quantitative information
on the availability, accessibility, and distribution
of products and services. Most agencies
have goals to improve product accessibility, particularly
through the Internet. Typical measures
include: on-demand requests fulfilled, product
downloads, degree of accessibility, customer satisfaction
with ease of use, number of participants
at user conferences, citations of agency data in
the media, number of Internet user sessions, number
of formats in which data are available, amount
of technical support provided to data users, exhibits
to inform the public about information products,
issuance of newsletters describing products,
usability testing of web sites, and assessing compliance
with Section 508 of the Rehabilitation Act,
which requires Federal agencies to make their
electronic and information technology accessible to
people with disabilities.
Mission Achievement: Qualitative or quantitative
information about the effect of, or satisfaction
with, statistical programs. For Government statistical
programs, this dimension responds to the
question—have we achieved our objectives and
met the expectations of our stakeholders? Under
this dimension, statistical programs document
their contributions to the goals and missions of
parent departments and other agencies, the Administration,
the Congress, and information users
in the private sector and the general public. For
statistical programs, this broad dimension involves
meeting recognized societal information needs and
also addresses the linkage between statistical outputs
and programmatic outcomes.
However, identifying this linkage is far from
straightforward. It is frequently difficult to trace
the effects of information products on the public
56 ANALYTICAL PERSPECTIVES
good. Such products often are necessary intermediate
inputs in the creation of high visibility
information whose societal benefit is clearly recognized.
For example, the economic statistics produced
by a variety of agencies are directly used
by the Bureau of Economic Analysis in the calculation
of the Gross Domestic Product (GDP),
which analysts universally use to assess changes
in the level of domestic economic activity. Similarly,
statistics from specific surveys are directly
used by the Bureau of Labor Statistics in the calculation
of the Consumer Price Index (CPI), which
is widely used in diverse applications, such as indexing
pensions for retirees. As a result, a number
of statistical agencies can claim credit for contributing
to the GDP and/or the CPI and to the many
uses of these information products. In addition,
the statistics produced by statistical agencies are
used to track the performance of programs managed
by their parent or other organizations related
to topics such as crime, education, energy, the environment,
health, science, and transportation.
Moreover, beyond the direct and focused uses of
statistical products and programs, the statistical
agencies and their products serve a diverse and
dispersed set of data users working on a broad
range of applications. Users include government
policy makers at the Federal, State, and local levels,
business leaders, households, academic researchers,
analysts at public policy institutes and
trade groups, marketers and planners in the private
sector, and many others. Information produced
by statistical agencies often is combined
with other information for use in the decisionmaking
process. Thus, the relationship between
program outputs and their beneficial uses and outcomes
is often complex and difficult to track. Consequently,
agencies use both qualitative and quantitative
indicators to make this linkage as explicit
as feasible.
In the absence of preferred quantitative indicators,
qualitative narratives can indicate how statistical
agency products contribute to and evaluate
progress toward important goals established for
government or private programs. In particular,
narratives can highlight how statistical agencies
measure the Nation’s social and economic structure,
and how the availability of the information
influences changes in policies and programs.
These narratives contribute to demonstrating mission
accomplishment, particularly in response to
questions in Section I of the PART, ‘‘program purpose
and design.’’ Narratives may describe statistical
information’s effects on measuring agency
policy or change of policy, supporting research focused
on policy issues, informing debate on policy
issues, or providing in-house consulting support.
In addition to narratives, quantitative measures
may be used to reflect mission achievement. For
example, customer satisfaction with the statistical
agency or unit indicates if the agency or unit has
met the expectations of its stakeholders.
Of the 14 principal Federal statistical agencies that
are members of the ICSP, six agencies have programs
that have been assessed using the PART process. Most
of these agencies’ programs have received PART summary
ratings of Effective or Moderately Effective, as
shown in Chart 4–2. While recognizing the strength
of the Energy Information Administration’s purpose and
management, EIA’s PART evaluation found that it
lacks specific annual performance measures, baselines,
and targets and should consider enhancing independent
expert evaluation of its major program areas. EIA is
correcting both of these shortcomings, which should
bring its PART rating into line with those of its sister
agencies. As additional ICSP agencies have an opportunity
to undergo the PART process, the agencies plan
to continue to use the results of the collaborative performance
standards development effort to help maintain
and extend their generally favorable assessments.
Chart 4–2. Most Recent PART Summary Ratings for Statistical
Programs
Summary Rating
Bureau of Economic Analysis Effective
Bureau of Labor Statistics Effective
Census Bureau
Current Demographic Statistics Effective
Decennial Census Moderately Effective
Economic Census Effective
Intercensal Demographic Estimates Moderately Effective
Survey Sample Redesign Effective
Energy Information Administration Results Not Demonstrated
National Agricultural Statistics Service Moderately Effective
National Center for Education Statistics
Statistics Effective
Assessment Effective
Highlights of 2006 Program Budget Proposals
The programs that provide essential statistical information
for use by governments, businesses, researchers,
and the public are carried out by some 70 agencies
spread across every department and several independent
agencies. Approximately 40 percent of the
funding for these programs provides resources for
twelve agencies or units that have statistical activities
as their principal mission. (Please see Table 4–1.) The
remaining funding supports work in 60-plus agencies
or units that carry out statistical activities in conjunction
with other missions such as providing services or
enforcing regulations. More comprehensive budget and
program information about the Federal statistical system
will be available in OMB’s annual report, Statistical
Programs of the United States Government, Fiscal
Year 2006, when it is published later this year. The
following highlights elaborate on the Administration’s
57 4. STRENGTHENING FEDERAL STATISTICS
Table 4–1. 2004–2006 BUDGET AUTHORITY FOR PRINCIPAL STATISTICAL AGENCIES
(in millions of dollars)
2004
Actual
Estimate
2005 1 2006
Bureau of Economic Analysis 2 .................................................................... 68 73 81
Bureau of Justice Statistics 3 ........................................................................ 32 34 63
Bureau of Labor Statistics ............................................................................ 518 529 543
Bureau of Transportation Statistics .............................................................. 30 30 33
Census Bureau 4 ........................................................................................... 629 765 897
Salaries and Expenses 4 ........................................................................... 213 216 240
Periodic Censuses and Programs ............................................................ 416 549 657
Economic Research Service ......................................................................... 71 74 81
Energy Information Administration ................................................................ 81 84 86
National Agricultural Statistics Service 5 ....................................................... 128 128 145
National Center for Education Statistics ....................................................... 187 185 208
Statistics .................................................................................................... 92 91 91
Assessment ............................................................................................... 95 94 117
National Center for Health Statistics 6 .......................................................... 90 109 109
Science Resources Statistics Division, NSF ................................................ 31 32 32
Statistics of Income Division, IRS ................................................................ 36 39 39
1 Reflects any recissions.
2 2005 estimate includes $2 million for a National Academy of Public Administration study of off-shoring.
3 The 2006 estimate includes funds for the Felony Arrestee Drug Use Reporting program (previously funded as the
Arrestee Drug Abuse Monitoring program within the National Institute of Justice) as well as funds for management and
administrative costs that were displayed separately in 2004 and 2005.
4 Includes Mandatory Appropriations of $20 million for each year for the Survey of Program Dynamics and collection
of data related to the allocation to States of State Chidren’s Health Insurance Program funds.
5 Includes funds for the periodic Census of Agriculture of $25, $22, and $29 million in 2004, 2005, and 2006, respectively.
The 2006 Budget includes an increase of $6.5 million due to cyclical activities including finalizing content,
developing mail lists, and streamlining and upgrading processing systems in preparation for the 2007 Census of Agriculture.
6 All funds from the Public Health Service Evaluation Fund. Funds for 2004 are shown comparably with 2005 and
2006. Administrative costs for NCHS that previously were displayed as part of the NCHS budget line are now reflected
in two consolidated CDC-wide budget lines for management and administrative costs.
proposals to strengthen the programs of the principal
Federal statistical agencies.
Bureau of Economic Analysis: Funding is requested
to: (1) make selected improvements to the timeliness
and comprehensiveness of the Nation’s international
statistics on multinational corporations and
trade in services; (2) complete work to accelerate the
release of gross state product, metropolitan personal
income, and county-level personal income; (3) enhance
the accuracy of BEA statistics by acquiring and incorporating
real-time data into core BEA accounts; (4) improve
data on international financial transactions by
working with the Department of the Treasury and the
Federal Reserve Board to incorporate newly developed
estimates of derivatives and other financial instruments;
and (5) produce up-to-date, annual estimates
of business investment spending by industry in order
to more accurately discern where high-tech and other
investments are being made in the manufacturing and
service sectors.
Bureau of Justice Statistics: Funding is requested
to provide for the maintenance of BJS’s core statistical
programs, including: (1) the National Crime Victimization
Survey, the Nation’s primary source of information
on criminal victimization; (2) cybercrime statistics on
the incidence, magnitude, and consequences of electronic
and computer crime to households and businesses;
(3) law enforcement data from over 3,000 agencies
on the organization and administration of police
and sheriffs’ departments; (4) nationally representative
prosecution data on resources, policies, and practices
of local prosecutors; (5) court and sentencing statistics,
including Federal and State case processing data; (6)
data on correctional populations and facilities from Federal,
State, and local governments; and (7) the Felony
Arrestee Drug Use Reporting program (previously funded
as the Arrestee Drug Abuse Monitoring program
within the National Institute of Justice).
Bureau of Labor Statistics: Funding is requested
to support program operations to measure the economy
through producing, disseminating, and improving BLS
58 ANALYTICAL PERSPECTIVES
economic measures, including: (1) modernizing the computing
systems for monthly processing of the Producer
Price Index (PPI) and U.S. Import and Export Price
Indexes (IPP), and producing new data outputs, such
as indexes based on the North American Industry Classification
System for the IPP; (2) maintaining continuous
updating of the Consumer Price Index (CPI) by
updating the expenditure and population weights biennially,
the superlative index annually, outlet samples
on a four-year cycle, and item samples in key categories
on a two-year cycle; and (3) releasing the 2004–2014
Employment Projections and publishing the 2006–2007
edition of the Occupational Outlook Handbook.
Bureau of Transportation Statistics: Funding is
requested to: (1) enhance the Freight Data Program,
a continuous source of data from shippers, carriers, and
receivers, to replace the current Commodity Flow Survey;
(2) move the Air Transportation Price Index, an
input to GDP and CPI indices, from experimental to
production mode; and (3) develop more timely and comprehensive
local and long-distance travel data.
Census Bureau: Funding is requested for the Census
Bureau’s economic and demographic programs, and
for a reengineered 2010 Census. For the Census Bureau’s
economic and demographic programs, funding is
requested to: (1) plan for the 2007 Economic Census,
(2) plan and implement the organizational phase of the
2007 Census of Governments and plan for the employment
and finance phases, (3) improve measurement of
services by expanding key source data for critical quarterly
and annual estimates of our Nation’s Gross Domestic
Product, (4) support improved coverage and electronic
reporting of trade statistics, (5) support the development
of a database infrastructure to integrate
State administrative data and Census Bureau data
products in order to fill critical data gaps at the State
and local levels, (6) continue efforts begun in 2003 to
eliminate data gaps by measuring migration across U.S.
borders, and (7) purchase furniture and relocate operations
and employees to the new headquarters facility
to avoid disruption of mission-critical operations necessary
for the successful completion of Census Bureau
surveys. For 2010 Census planning, funding is requested
to continue to: (1) conduct planning, testing,
and development activities to support a reengineered
2010 Census; (2) complete map feature accuracy within
7.6 meters of true GPS location for 700 of the Nation’s
counties; and (3) continue to conduct the American
Community Survey program to provide data on an ongoing
basis rather than waiting for once-a-decade censuses.
Economic Research Service: Funding is requested
to support ongoing programs and to continue the development
of an integrated and comprehensive data and
analysis framework of the food system beyond the farmgate
to provide a basis for understanding, monitoring,
tracking, and identifying changes in the food supply
and consumption patterns.
Energy Information Administration: Funding is
requested to continue ongoing operations, with a focus
on: (1) improving petroleum and natural gas data security,
reliability, and quality; (2) conducting the commercial,
manufacturing, and residential energy consumption
surveys; (3) implementing the enhanced Voluntary
Reporting of Greenhouse Gases program to support the
President’s Climate Change Initiative; and (4) developing
a program performance prototype to assess EIA’s
data collection and operations costs at a more
disaggregated level.
National Agricultural Statistics Service: Funding
is requested to: (1) continue restoration and modernization
of the agricultural estimates program to ensure
State, regional, and national level agricultural estimates
of sufficient precision, quality, and detail to meet
the needs of a broad customer base; (2) continue development
and implementation of a locality-based agricultural
county estimates/small area estimation program;
and (3) continue preparations for the 2007 Census of
Agriculture.
National Center for Education Statistics: Funding
is requested to: (1) support the ongoing data collection
and analysis of the Early Childhood Longitudinal
Study Birth and Kindergarten Cohorts, which provide
data to inform child development practices and early
education; (2) continue the Integrated Education Postsecondary
Data System, which collects information on
enrollment, completions, and finances from postsecondary
institutions; (3) sustain the ongoing data collection
efforts for the Beginning Postsecondary Students
Longitudinal Study; (4) maintain U.S. participation in
international assessments that compare educational
achievement in the United States with that in other
countries; and (5) continue the National Assessment
of Educational Progress (NAEP) program, including
funding to support the expansion of State NAEP to
grade 12.
National Center for Health Statistics: Funding
is requested to: (1) increase timeliness by upgrading
electronic systems for data collection and processing;
(2) expand the content of surveys, particularly those
addressing the health care delivery system; (3) redesign
the sample for the National Health Interview Survey,
NCHS’ largest population survey; and (4) work collaboratively
with States and other agencies on upgrading
the technology for collecting data from State birth and
death certificates.
Science Resources Statistics Division, NSF: Funding
is requested to: (1) continue to implement the results
of prior methodological, analytical, and planning
activities directed toward improving the relevance, accuracy,
timeliness, and accessibility of SRS products,
including the suite of Research and Development surveys
and the Survey of Graduate Students and
Postdoctorates in Science and Engineering; and (2) lead
a cross-agency effort to examine and revise current
59 4. STRENGTHENING FEDERAL STATISTICS
taxonomies used for classifying academic fields of study,
including the development of crosswalks between existing
taxonomies and any potential new taxonomy, as
well as strengthen methods to enhance the identification
and description of cross-disciplinary and multi-disciplinary
fields.
Statistics of Income Division, IRS: Funding is requested
to: (1) maintain and modernize core data collection
systems, including several major statistical programs
for the Department of the Treasury, the Congressional
Joint Committee on Taxation, the Bureau of Economic
Analysis, and SOI’s many other customers; (2)
implement a databank repository for SOI and IRS population
file data to more efficiently build longitudinal
databases and enable sub-national estimates; (3) examine
means to more effectively mask individual records
to minimize the possibility of identification in the Individual
Public Use sample files; and (4) modernize and
expedite dissemination of data and publications, including
a reengineered Internet website.
