Just a decade ago, the idea of the United States government being a Big Data warehouse for the public was silly. Today, however, the country has a chief data scientist for the first time in its history, and he showed up in Silicon Valley today to talk Big Data with the attendees of Strata.
A day after being named chief data scientist, DJ Patil told the attendees that Big Data and Hadoop have completely changed the way the United States is governed.
“The president has a data dashboard to track the spending of IT projects. We’ve got an executive order that ensures all data created by the U.S. government is open and machine-readable. That’s a massive change,” he said.
Patil added that the president is able to use data to make policy decisions, something that has been relatively unheard of in the government’s past. He said that the Data.gov site is also indicative of the change in tenor of the White House since President Barack Obama came into office. Patil said that the data sets hosted on Data.gov represent a great opportunity for developers who want to learn about Big Data.
Using these data sets, said Patil, developers can build a store of data that is complex and interesting enough to teach them about building Big Data applications. He hoped this idea would spark interest in attendees in joining the U.S. Digital Service to help the country revolutionize its IT and data infrastructures.
Patil wasn’t the only one trying to “establish a data-driven culture” in Washington and in government. Solomon Hsiang, professor of public policy at the University of California, Berkeley, spoke at the Strata keynote. “The work you are doing on Big Data analytics is having a fundamental impact on how we govern today,” he said.
“Innovations in the technology of governance have transformed our ability to deploy efficient government policies, and to make the world a better place. We are now in the midst of another revolution allowing us to better understand ourselves, and that helps us design better policies.”
Hsiang said that his work for the past few years has involved the quantifying of costs around climate change. “We’re causing the climate to change. We have the data and analytics to understand what is happening. The thing I call the fundamental climate-management problem is trying to determine if the costs of adopting a low emission scenario out-weigh the costs of climate change,” he said.