Panel report: opening up government to better data
June 3, 2009
The Orwellian-sounding Committee on National Statistics (CNSTAT, part of the National Academy of Science) is a leadership group whose membership consists of a few dozen top government, industry, and academic statisticians. This past May, the group held its annual public symposium in Washington DC, where it sought to assess the present and plot the future of the Federal Statistical System. Approximately 250 attendees, many of them government statisticians, spent a full day trading ideas and hearing from leaders in the field and from members of the Obama Cabinet.
Present at the meeting was the government’s Chief Statistician, Katherine Wallman, who’s part of the White House Office of Management and Budget. She attempts to coordinate the dozens of statistics offices (Census Bureau, Bureau of Labor Statistics, Department of Health Statistics, National Agricultural Statistics Service, and on and on) in many Federal Government sectors (Department of Commerce, Department of Labor, Department of Health and Human Services, Department of Agriculture, respectively).
Wallman’s boss, OMB Director, Peter Orszag, got the CNSTAT meeting underway. He used the occasion to trumpet the administration’s practice of "evidence-based decision-making" and to raise the point that government decision-makers have undervalued government statistics for too long. Federal spending on Federal statistics has held steady at 0.02% of the GDP, even declining slightly since 1980, Orszag said. "But we are making very large decisions based on investments in the data that are themselves pretty modest."
The example he chose was taken from his current talking point that the irrational healthcare practices going on nationwide are also expensive. "In healthcare we are doing lots of things without measuring the impact and we are, I believe, wasting substantial amounts of resources on procedures that don’t work simply because we are not aggressive enough in measuring what works and what doesn’t."
Government statistics were not Orszag’s analytical source for this assertion, though. The work he was relying on comes from Dartmouth scientists who used Medicare data as part of a long-term research project called the Dartmouth Atlas Project (DAP), funded by the Robert Wood Johnson Foundation. The case illustrates how today’s statistics increasingly come from a variety of sources—nonprofit and for-profit organizations, academia, and government—that get mashed up as insights emerge.
The fact that statistics in the United States are decentralized is both a blessing and a curse. Entrepreneurs and academics can act opportunistically and establish measures on their own, but it is challenging. Government statistical agencies have an easier time because they operate in proximity to, and in coordination with, policy-making Federal agencies. Speaker Hermann Habermann, former Director of the UN Statistics Division and former Census Bureau Deputy Directory, told the CNSTAT group that improved trust and collaboration, as well as reduced risk from having multiple points of failure, are features of the US system not found in centralized national statistics agencies, as found in Canada or Australia.
It takes decades for a project like DAP to become the well-respected source it now is, and it may not ever gain the inside-the-Beltway policy impact of an official government stat. So symposium speakers, Habermann among them, spoke of the need to launch some type of R&D initiative that could improve the government numbers machine’s capacity to break out in new directions, asserting its credibility with federal policymakers in new statistical domains, like healthcare. Throughout the day, presenters identified statistical information gaps in education, immigration, and climate change. All of these are areas where the stakes are high, where evidence is much needed for better policy, and where, consequently, an investment in better stats would pay off.
The State of the USA
One emerging great hope for national statistics is an ambitious, nonprofit organization based in Washington DC called State of the USA (SUSA) which has spent the past few years building its government credibility and establishing partnerships with government, industry, academia, and foundations. The core of SUSA is a set of Key National Indicators, which are measures in issue areas like health, transportation and infrastructure, energy, aging, national security, and population, that reference the progress (or lack of progress) America is making in dealing with its toughest problems.
President and CEO of State of the USA, Chris Hoenig, hatched the idea back in 2003 when he was in charge of Strategic Initiatives at the Government Accounting Office. (See the original GAO Report.) "In our organization we’re trying to design what that common user experience might be which would bring statistics to tens of millions of people in the United States in new ways," he told the CNSTAT crowd.
Work by SUSA on a set of health indicators in partnership with the National Academies’ Institute of Health has put forward 20 measures by which individuals, companies, or any group can take the pulse of the nation’s health system. SUSA recently went into beta with its Web platform, where it expects to make heavy use of data visualization in a first draft of an information dashboard to help drive the country.
As promising as SUSA appears to be, Habermann thinks it would still be wise for the Federal Statistical System to harness the Internet’s collaborative power. The Web and the interaction it enables, he said, "are fundamental to whether a statistical agency lives or dies in the 21st century, and whether it can really become an effective provider of information."
That’s because in the era of open government, the stats agency constituency won’t just be a Cabinet Department. It will be the cohort of Americans who use the interfaces developed by the Obama administration and by interested hackers to participate in government. At present it’s discouraging to see the quality disparity between government Web developers who are using Web 2.0 to good effect (Recovery.gov, EPA.gov’s MyEnvironment) and examples from the government statistics community like FEDSTATS, the National Atlas, and Data Ferrett and OnTheMap from the Census Bureau. There just is no comparison.
Can evidence-based policy decision-making mesh with public, participatory decision-making? Yes, but not easily, believes Andrew McLaughlin, a Google executive who helped the Obama transition team, and who will soon rejoin the Executive Branch. He did not attend the CNSTAT symposium; the details come from April’s Web 2.0 Expo (see the video). "You’ve got some questions that government deals with that are very fact-specific. And you’ve got some questions that government deals with that are very values-driven," McLauglin said to the San Francisco crowd. "Online participation tends to work best in the fact-specific cases. It’s hard to weight the non-factual inputs in a decision-making process."
Where does this lead? According to McLaughlin, "The Obama administration and its various agencies are going to be experimenting with all sorts of small-scale tools like ‘Ask the President‘ or the ‘Citizens Briefing Book‘ that are focused, one-off experiments that will then evolve into platforms." There will be lots of trial and error, and it’s hard to see such willingness among the traits government statisticians are known for.
It’s long been the case that transparency, participation, and collaboration were undertaken within the silos of government agencies, but not necessarily between them. And it’s here that the practice and infrastructure is changing under Kshemendra Paul, Chief Architect for E-Government and IT in the OMB, and a former Chief Architect for the US Department of Justice. "It’s not unfamiliar in terms of how to do this. We do it in a mission context. What is new is doing it in this broad-based way, and doing it as more of a horizontal platform versus these vertical mission areas, and scaling it up. The numbers we’re talking about here are much larger, the impact more dramatic, and the velocity goes faster," Paul told a May 15 conference on transparent government held at the University of Texas’ LBJ School of Public Affairs.
Next on Paul’s timetable, a June 15 deadline for agency Chief Architects to complete their self-assessments and reports on their Enterprise Architecture—part of which will describe how these agency IT silos plan to undertake data collaboration and reuse (Paul’s memo). This isn’t exactly in step with the Administration’s rapid Open Government Initiative timeframe, but it’s progress. And it’s consistent with the way science moves forward, since individual insights often come from perches inside hefty bureaucracies as often as from outside, but they still manage to rationally coalesce.
John Holdren, the president’s science advisor and former university particle physicist, closed the CNSTAT meeting. "In the classes I taught in science, technology, and public policy, I usually began by saying to my students that in public policy, the numbers are almost never everything. But they are almost always something," before summarizing, "To make public policy on the basis of bad data or no data is almost always a hideous error."
Yes, the government is aware that the stakes are high, and the "open government" approach is new, but the pursuit of high quality, standardized, sharable data in all its forms is the best course of action. The three reasons for this are abundantly clear: transparency, participation, and collaboration. For government statisticians, it appears they’ll have to evolve in their new environment.
This post has been written by Brad Stenger on June 3, 2009 12:30 AM couresy of arstechnica.com.