Government, citizen developers join forces to build new Federal Register 2.0 Website

Federal Register 2.0The Federal Register has launched a re-design of its Website, federalregister.gov. The new site is XML-based and was developed using open source code (now available on GitHub).

“The Daily Journal of the United States,” the FR is managed by the Office of the Federal Register (OFR) of the National Archives and Records Administration (NARA) and the U.S. Government Printing Office (GPO) and serves as “the legal newspaper of the U.S. government and contains rules, proposed rules, and public notices of federal agencies, as well Presidential documents.”

U.S. Archivist David Ferriero said this about Federal Register 2.0 on the White House Blog:

“Federal Register 2.0 takes into consideration the 21st century user and turns the Federal Register website into a daily web newspaper. The clear layout will have tools to help users find what they need, comment on proposed rules, and share material relevant to their interests. In addition to greatly improved navigation and search tools, the site will highlight the most popular and newsworthy documents and feature each agency’s significant rules.”

The idea for the re-design originated from Sunlight Labs’ Apps for America 2 contest. Developers Andrew Carpenter, Bob Burbach and Dave Augustine from WestEd Interactive built GovPulse.us, “The Federal Register at your fingertips” and won second place. They caught the attention of OFR, who contacted them to help with the official re-design.

A list of new features can be found here and more information about GovPulse and its technology can be found here here.

Video history of the Federal Register and overview of Federal Register 2.0:

Video overview with GovPulse.us developers:

Government, developers need to build a more structured, scalable approach to leveraging technology

The time has come to build a reliable, open platform that allows local governments to post development requirements and give private developers the ability to respond and build these applications for free.

Going a step further, we need to build a free, open source platform specifically for government, making it easier for government to install and implement and leverage plugins or modules for anything from standard contact forms to 311 citizen requests applications.

Fundamentally, we need a central repository for code and a governing organization, private or non-profit, that coordinates specifications and provides a reliable management process for deployment. Additionally, there needs to be sample usage and, ideally, implementation case studies that highlight how government is leveraging this tool and how others can follow suit.

We need a GitHub meets Taproot meets WordPress or Drupal for government.

Matthew Burton’s A Peace Corps for Programmers, comments like Kevin Curry’s recent “We need craigslist for government” tweet and inside open government baseball chatter echo these sentiments.

To date, contests to create killer Web and mobile applications from open data combined with developers with gumption have spearheaded much of the tech efforts. This approach has showed positive results, however, they don’t effectively address a customer-driven approach to product development (see Steve Blank), where the customer (government) defines the specification, instead of developers building applications of no direct benefit to government.

Government must begin to define the specification. Instead of putting it out to bid, government needs to put it out to BUILD.

Government needs to break the mold and take advantage of what Clay Shirky calls the cognitive surplus, leverage the enthusiasm of the civic developer and significantly lower the cost of its technology projects. Government must also move away from a ‘build our own’ approach to technology. This mindset is a waste of time and resources and financially irresponsible.

Sure, there are procurement hurdles around non-licensed software, but many of these can be re-defined, as done in places such as San Francisco, Portland, Vancouver.

Philanthropists or foundations with deep pockets need to step up and support a new organization or a current one truly dedicated to making this happen. Government could also ‘pay back’ with funding of its own, at a significant discount to what it would otherwise pay. Something like this needs sustainable investment and support.

If the private or non-profit sector and government could each eliminate any hurdles and actively engage an idea like this, we’d change the way government uses technology and how it serves its citizens.

Who can make this happen and how do we get started?

TransparencyData.com shines light on campaign contributions from last 20 years

Sunlight Foundation has launched TransparencyData, a new Website that lets users easily access the past 20 years of federal and state campaign contributions all in one place. The site merges data from OpenSecrets, FollowTheMoney.org and lobbying information from the Senate Office of Public Records.

Sunlight Labs Director Clay Johnson:

“This tool is focused on giving people bulk access to data. Instead of generating complex visualizations, and a slick user interface, we’ve focused on making it easy to query this large dataset, and walk away with a spreadsheet of the data you need. The ultimate output of this tool isn’t an HTML table, but a CSV file so you can take the data and do the research you need to do … Look for government contracting, earmarks, and congressional biographical data coming shortly.”

Video overview:

More from Sunlight:

Gov 2.0 guide to the Public Online Information Act (POIA)

The Public Online Information Act (POIA) of 2010, H.R.4858, was introduced on March 13 by Rep. Steve Israel (D-N.Y.) to put public information online in user-friendly formats in a timely fashion. The bill applies to Executive Branch agencies and is essentially a proactive approach to FOIA (Freedom of Information Act). Sunlight Foundation has launched Public=Online, a grassroots campaign to gain support for the legislation.

Overview:

To establish an advisory committee to issue nonbinding government-wide guidelines on making public information available on the Internet, to require publicly available Government information held by the executive branch to be made available on the Internet, to express the sense of Congress that publicly available information held by the legislative and judicial branches should be available on the Internet, and for other purposes.

Video intro to POIA:

Press conference with Rep. Israel, Sunlight Foundation Executive Director Ellen Miller and Personal Democracy Forum Founder Andrew Rasiej announcing the bill:

Israel and Miller discuss POIA on MSNBC:

More POIA

‘Design for America’ contest aims to make government data more accessible

Sunlight Labs is holding a civic design contest, Design for America, in an effort to “make government data more accessible and comprehensible to the American public.” Categories are Data Visualization, Process Transparency and Redesigning the Government.

From Sunlight:

This 10 week long design and data visualization extravaganza is focused on connecting the talents of art and design communities throughout the country to the wealth of government data now available through bulk data access and APIs, and to help nurture the field of information visualization.

Top prize is $5,000 for each of these categories:

  • Data Visualization of Sunlight Community Data
  • Visualization of Data from the Federal Budget and/or USASpending.gov
  • Visualization of Recovery.gov Data
  • Visualization of How a Bill Becomes a Law
  • Visualization of Congressional Rules/Floor Procedures
  • Redesign of a Government Form
  • Redesign of a .Gov website

Submissions are due May 17, and winners will be announced May 27 at Gov 2.0 Expo. Sponsors include Adobe, Google, O’Reilly Media, TechWeb, Gov 2.0 Expo and Palantir.

New on GovFresh: ‘Fresh from: Sunlight’

Fresh from: Sunlight is a new GovFresh feature that highlights the latest transparency and open government news directly from Sunlight Foundation and Sunlight Labs. Contributors will include Sunlight’s best and brightest, including transparency hunk Jake Brewer.

Follow the freshest from Sunlight at sunlight.govfresh.com.

What the Open Government Directive Means for Open Source

On the heels of the Open Government Memo of January 21st, 2009, the Obama Administration has issued the Open Government Directive. The Directive tells agencies what they must do to meet the expectations set by the Memo. The directive names many deadlines for agency compliance, most of them around reducing FOIA backlogs and increasing the amount of agency data released to the public. This isn’t surprising, since the Memo names transparency, collaboration, and participation as the guiding principles. Transparency is the easiest to articulate and implement — just get the data out there in a useful form. Josh Tauberer’s Open Data is Civic Capital: Best Practices for “Open Government Data” is an excellent handbook for doing this. If you want to track agencies’ progress, the Sunlight Labs folks have produced the outstanding Open Watcher.

What’s most interesting to me, and my friends at Open Source for America, though, are the more ambiguous orders. Although the Directive does not use the phrase ‘open source software’ at all, many of the principles and methodologies described are obvious references to open source. Many of these orders stand out as opportunities for open source developers, in the public and private sector, to demonstrate how our development model can help the Administration also make good on the last two principles: collaboration and participation. As Macon Phillips, the White House New Media Director said, “Open Source is… the best form of civic participation.”

Let’s take a look at the deadlines, helpfully produced by Daniel Schuman at the Sunlight Foundation.

45 days — January 22, 2010

“Each agency shall identify and publish online in an open format at least three high-value data sets and register those data sets via Data.gov” (p.2)

This is a wonderful opportunity for open source developers to demonstrate the power of citizen participation through software. The Administration has taken a great risk by pushing this data to the public. There are all kinds of reasons to not do it: privacy concerns, security issues, and the risk-averse culture in most of these organizations. Despite the instructions to be careful with citizens’ privacy, and the reminder to be sensitive to security issues, there’s still a chance that something could go wrong — plenty of reason to not follow through with this exercise. We need to help the Administration prove that this was a worthwhile cause. Just as we showed the power of citizen programmers in Apps for Democracy and Apps for America, we need to take these data sets and make them useful to the American public.

“The Deputy Director for Management at OMB, the Federal Chief Information Officer, and the Federal Chief Technology Officer will establish a working group that focuses on transparency, accountability, participation, and collaboration within the Federal Government. This group, with senior level representation from program and management offices throughout the Government, will serve several critical functions, including:

  • Providing a forum to share best practices on innovative ideas to promote transparency, including system and process solutions for information collection, aggregation, validation, and dissemination;
  • Coordinating efforts to implement existing mandates for Federal spending transparency, including the Federal Funding Accountability Transparency Act and the American Reinvestment and Recovery Act; and
  • Providing a forum to share best practices on innovative ideas to promote participation and collaboration, including how to experiment with new technologies, take advantage of the expertise and insight of people both inside and outside the Federal Government, and form high-impact collaborations with researchers, the private sector, and civil society.” (p.5)

Now here’s a working group I would like to speak with very much. If you read the language of the third subsection, it’s amazing how many words you have to use to not say the words “open source”: experiment with new technologies, using expertise inside and outside the government, high-impact collaborations with many communities of use… they’re all but begging to create open source software projects to support the release of this government data.

In this “forum for best practices” on open data initiatives, you can imagine how useful a recommendation of open source software might be. You can even imagine the working group recommending government open source projects to help handle data that may be in strange government-specific formats.

60 days — February 6, 2010

“Each agency shall create an Open Government Webpage located at http://www.[agency].gov/open to serve as the gateway for agency activities related to the Open Government Directive” (p.2)

“The Federal Chief Information Officer and the Federal Chief Technology Officer shall create an Open Government Dashboard on www.whitehouse.gov/open. The Open Government Dashboard will make available each agency’s Open Government Plan, together with aggregate statistics and visualizations designed to provide an assessment of the state of open government in the Executive Branch and progress over time toward meeting the deadlines for action outlined in this Directive.” (p.5)

Of course, if an agency is writing new software to support these new “/open” areas, I’d like to see that software made available under a open license. If there are any clever data analysis or visualization tools, those should be licensed as open source software, as well. That way, citizens would have the opportunity to help the agency with their own disclosures, and agencies could more easily share tools with each other.

90 days — March 8, 2010

“The Deputy Director for Management at OMB will issue, through separate guidance or as part of any planned comprehensive management guidance, a framework for how agencies can use challenges, prizes, and other incentive-backed strategies to find innovative or cost-effective solutions to improving open government.” (p.5)

This is a strangely oblique reference to Vivek Kundra’s Apps for Democracy project when he was CTO in Washington, DC, and the national-scale follow-on, Apps for America. Both of these contests asked that submissions be provided under OSI-approved licenses. This is important to keep these projects going. If contestant’s software is under a proprietary license, there is no momentum behind the contest, since nobody can contribute to it after the fact. You might as well hold no contest at all, and instead just bid the work out to a contractor.

120 days — April 7, 2010

“Each agency shall develop and publish on its Open Government Webpage an Open Government Plan that will describe how it will improve transparency and integrate public participation and collaboration into its activities. Additional details on the required content of this plan are attached. Each agency’s plan shall be updated every two years.” (p.4)

I would hope very much that these plans for additional public participation and collaboration include invitations to open source developers who would like to help an agency build tools that make them function more transparently and efficiently.

“The Administrator of the Office of Information and Regulatory Affairs (OIRA), in consultation with the Federal Chief Information Officer and the Federal Chief Technology Officer, will review existing OMB policies, such as Paperwork Reduction Act guidance and privacy guidance, to identify impediments to open government and to the use of new technologies and, where necessary, issue clarifying guidance and/or propose revisions to such policies, to promote greater openness in government.” (p.6)

I hope that this review would include an examination of FACA implementation guidelines, which is understood by many to prevent open source developers from directly participating with some Federal agencies, for fear of having offered the explicitly prohibited “volunteer help.” We believe this isn’t the case, and it would be great if OIRA published some clarifying language. If they were to provide an interpretation of OMB Circular 130-A that ensured it was safe for agencies to create open source software without running afoul of procurement regulations, that would be wonderful.

So here’s a tremendous opportunity for the open source community. We have been given an early Christmas gift: a pretty clear path for more open source software and (perhaps more importantly) more government-sponsored open source projects inside each agency. If you want to help take advantage of this opportunity, you can sign up at Open Source for America and join a working group. You’ll be glad you did.

A hearty thanks the Heather West of CDT and Melanie Chernoff of Red Hat for their invaluable comments.

Gov 2.0 guide to Sunlight Foundation

Sunlight Foundation is a Washington, DC-based 501c(3) non-profit organization founded in 2006 to focus on “making government transparent and accountable.” Its name comes from a quote by Supreme Court Justice Louis Brandeis: “Sunlight is the best disinfectant.”

Sunlight was co-founded by Michael Klein and Ellen Miller. Miller serves as its executive director.

Ellen Miller GovFreshTV interview:

Ellen Miller CSPAN interview:

Ellen Miller Web 2.0 Expo interview with Tim O’Reilly:

Projects

  • OpenCongress.org: Brings together official government data with news and blog coverage, social networking, public participation tools, and more. Free, open-source, not-for-profit, and non-partisan web resource with a mission to make Congress more transparent and to encourage civic engagement.
  • Foreign Lobbying: Foreign Lobbyist Influence Tracker, a joint project of ProPublica and Sunlight, digitizes information that representatives of foreign governments, political parties and government-controlled entities must disclose to the U.S. Justice Department when they seek to influence U.S. policy.
  • Congrelate: Lets users view, sort, filter and share information about members of Congress and their districts.
  • Transparency Corps: Lets anyone, anywhere have a positive impact on making our government more transparent by aggregating small actions that require human intelligence but not specialized political knowledge.
  • Party Time: Documents the Congressional fundraising circuit.
  • Transparency Jobs: Features jobs from both the US Federal Government and non-government organizations.
  • LouisDB: The Library Of Unified Information Sources, an effort, to paraphrase Justice Louis Brandeis, to illuminate the workings of the federal government.

Sunlight Labs

Sunlight Foundation also includes Sunlight Labs, a “community of open source developers and designers dedicated to opening up our government to make it more transparent, accountable and responsible.” Sunlight Labs has an online community and a Google Group. Clay Johnson is its director.

Clay Johnson GovFreshTV interview:

Clay Johnson Gov 2.0 Radio interview:

Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.

Blogs

Connect

More Sunlight Foundation

Open gov, Gov 2.0 leaders react to White House Open Government Directive

Here’s what open government and Gov 2.0 leaders are saying about the new White House Open Government Directive.

What’s your take?

Carl Malamud, Public.Resource.Org (@CarlMalamud)

Carl Malamud

“This is great. No equivocating, vacillating, hemming, or hawing. This is all good, big thumbs up to the folks that made this happen.”


Ellen Miller, Sunlight Foundation (@EllnMllr)

Ellen Miller

“The Open Government Directive demonstrates how the Obama administration is matching its aspirational goals with concrete policies and accountability measures. I expect it will create a sea change in how the government and public interact, what information we as citizens have at our fingertips, and that it will redefine that public information means that its online. It’s going to be up to all of us to participate and monitor how well government meets these goals.”


Craig Newmark, Craigslist (@craignewmark)

Craig Newmark

“The Open Government Initiative is a huge commitment to:

  • listening to all Americans, hearing what they have to say
  • telling people what’s going on in government, like where the money goes

The results will create effective large-scale grassroots democracy and far greater fiscal responsibility.

I feel that these efforts are complementary to the adoption of the US Consititution.”


Chris Vein, City and County of San Francisco (@Veinesque)

Chris Vein

The President’s Directive is a tremendous step forward. It not only further explains the President’s vision, but it provides an aggressive roadmap and timeline for getting Federal, State and local governments to improve transparency, increase participation and collaboration. San Francisco is proud to have responded early to the President’s call for open government with our Open Data Directive and DataSF initiatives. The President’s Directive will help San Francisco improve and extend our goal of a more transparent and open City.

Dustin Haisler, City of Manor, TX (@dustinhaisler)

Dustin Haisler

“The Open Government Directive is a great starting point for the open-gov movement in the federal government; however, one thing to consider is whether open data is truly “usable” data for our constituents. Instead of just putting datasets online for mashup artists, we should also focus on the interface our citizens will use to get the information. In addition, multi-agency collaboration starting on the local level will be a very important key to the overall initiative’s success. Overall, I think the directive is good move in the right direction for the federal government.”


Peter Corbett, iStrategyLabs (@corbett3000)

Peter Corbett

“We’ve all been eagerly awaiting the OGD and it’s not a let down by any stretch. It will lend support and clarification to what is a complex issue for our government: how to become more open, transparent and participatory. What we’re seeing here is the innovative use of technology and smart policy to unleash the talent of the American people. I’m most excited about how the work we’ve done on Apps for Democracy will soon be institutionalized throughout federal agencies when OMB releases guidance for how to use challenges, prizes and other incentives for stimulating citizen driven innovation.”


Andrew Wilson, Health & Human Services (@AndrewPWilson)

Andrew Wilson

“This directive represents a significant step toward the president’s goals of transparency, public participation and collaboration. One element that I would like to see emphasized as part of the implementation is a concerted, systemic effort to improve the tools government employees have available to collaborate internally. For me, improved internal collaboration is an essential element to developing the framework for a more fully engaged and responsive government. Imagine a world where cross-departmental information flow was so robust that citizens could interact with ANY agency on ANY issue and could get a timely, complete and helpful response.”


Steve Ressler, GovLoop (@govloop)

Steve Ressler

“Open Government Directive is a great first step in the open gov/Gov 2.0 movement. While the data and transparency piece is important, I’m most interested in how agencies create their own open gov plans and what actions they take from their planning exercise. I believe most of the movement for open gov starts when it is done at the agency level and solving true mission needs.”


Clay Johnson, Sunlight Labs (@cjoh)

Clay Johnson

“This is a great and ambitious plan that’s particularly challenging in terms of both logistics and technology. It is the equivalent of the “putting a man on the moon” of the Transparency movement in the federal government. Challenging, awe-inspiring and risky.”


Adriel Hampton, Gov 2.0 Radio (@adrielhampton)

Adriel Hampton

“I am concerned that some may use the document and its compliance deadlines as a simple checklist. However, as did the president’s January open government memo, this document empowers the growing ranks of Gov 2.0 innovators. Its guidance on data release and standards is also valuable and needed.”


Steve Lunceford, GovTwit (@dslunceford)

Steve Lunceford

“I think this is a great step to formalize a process and “movement” that has already been spreading throughout government. I would have like to have seen more guidance around transparency, participation and collaboration from an interagency standpoint versus just citizen interaction, but believe that could be a natural output as agencies strive to meet the various deadlines. It will also be interesting to see how quickly and enthusiastically agencies respond to a directive which lays out new unfunded mandates given the many priorities they are already juggling.”


Bob Gourley, CTOvision (@bobgourley)

Bob Gourley

The most important part of the directive, in my opinion, is the attachment with guidance on plan formulation. The thought put into that means agencies do not have to recreate the wheel when formulating their own plan. The part of the directive that we all need to watch out for abuse on: it seems to apply to all other than OMB and above. Yet history has shown those are the ones we need the most openness from.


Brian Ahier (@ahier)

Brian Ahier

“I am thrilled to see the emphasis on open government this directive represents. I hope to see government agencies able to meet the deadlines for action established by the Open Government Directive. I also want to see citizen participation in determining the high value data sets to be published. Since this directive also requires the data be published in an open format, it will be nice to have documents available where the data is not shielded within the pdf format.”


Great American Hackathon set for Dec. 12-13

Great American Hackathon

Sunlight Labs has joined with Mozilla, Google, Redhat, Fedora, Open Source for America and Code for America to promote the Great American Hackathon. The two-day event, December 12-13, aims to “to solve as many open government problems as we can with as many hackathons across the country as possible.”