Open government

Help get USDA to lead with APIs when it comes to America’s parks

Photo: USDA

Photo: USDA

I need your help with something.

I’m in the business of helping startups, all the way up to our federal government, identify valuable assets and make them more accessible for use in websites and mobile devices. As part of this work I’m always on the look out for valuable public assets across city, state and federal government, and help make sure the conversations around these assets always include application programming interfaces, so that we aren’t just building web and mobile applications in silos, and limiting the potential for public access by individuals and small businesses.

Over the last couple years, I have cultivated a small group of API evangelists in various sectors who help keep me informed of important government projects. One of my partners in crime, Alyssa Ravasio (@alyraz), brought a pre solicitation for a request for proposal to my attention from the U.S. Department of Agriculture, for a “Recreation On Stop Support Services” to my attention. You can see this service in action at Recreation.gov. In short, this is an ongoing contract that our federal government engages in with a private sector contractor to provide access to our nation’s campgrounds and campsites at our federal parks.

I would put our national parks and campgrounds at the top of treasured assets that this country possesses, and something that we need to make sure continue to be as accessible to individuals and small businesses as possible. I want to take the time to respond to the pre-solicitation from USDA, in hopes of helping them craft a better RFP before this goes out to bid.

I also want to encourage others, who are also passionate about camping and our national parks, to submit their own feedback to USDA’s solicitation number AG3187S151000–Recreation One Stop Support Services.

The current incarnation of the one stop service for recreation project has an API. Well, it has web services as part of the existing Recreation Information Database, but you also find mentions of API in the proposed RFP, under the 5.3.9 Mapping and Geospatial Capability:

A segment of the user community will not only be accessing and viewing the mapping data through the web interface at www.recreation.gov, but will also have a need to search, retrieve, and download the geospatial data via the RIDB data sharing interface(s), or APIs. The Contractor shall demonstrate utilization of programming languages, formats, data types/conventions/sources, etc. which maximize the portability and usability of maps and map related data by third-parties. This minimizes the need for third-party data consumers to procure expensive and/or proprietary software to utilize, edit or manipulate the mapping data. The contractor shall ensure that all available geospatial data within R1S is incorporated into the user interface to provide users with the most complete mapping solutions available.

There is another mention of APIs under “5.3.6 Provide Travel Planning Tools” with:

Non Federal reservable inventory may include state and local reservable and non-reservable recreation inventory which may be incorporated into the system or accessed via various application programming interfaces (APIs).

Then, towards the end you see the following needs:

  • Data Sharing and Data Management – Information originating from within RIDB and imported shall be presented in a manner such that the external origin of the data is transparent to users and consistent with presentation of information originating from within.
  • Information Sharing – The Contractor shall deliver automated and manual services for the sharing of the consolidated recreation information described herein. The sharing service shall utilize prevailing industry best practices and technologies to make the consolidated recreation information publicly available to data consumers.
  • Information Consolidation – The Contractor shall deliver automated and manual services for collecting of a wide variety of recreation information from a wide range of sources including Federal Government Agencies and Federal Partner organizations such as Historic Hotels of America.

To me, all of this infers that whoever wins the contract will need to at least maintain what is available via the RIDB, which is nice, but it is just a web service, not a modern web API, and only reflects providing access a small portion of the valuable resource that will need to be made available via Recreation.gov to partners and the public at large. We can do better.

An API needs to be a first-class citizen in this RFP, something that is not just an afterthought and a connection to the database that just needs to be maintained — an API will be central to deliver the resources needed for a “one stop services for recreation”.

Camping resources

When you look through the requirements for this project, you see all the building blocks for allowing individuals to engage with campsites across our nation, inventory, ticketing, lotteries, permitting, day use and group facilities, special events and use, equipment rentals, recreational and pass sales. All of this keeps our national parks accessible to campers.

Mapping and geospatial resources

Camping resources are meaningful to us because of where they are located. Mapping and geospatial resources will be vital to enabling users to discover the parks and campsites where they wish to stay on their camping trips. The ability to search and navigate campsites via modern mapping and providing geospatial resources is central to the one stop service for recreation.

Financial resources

A third cornerstone of the recreation one-stop support services platform are the financial resources. The ability to accept and process credit cards, cancellations, modifications and refunds will play a role in every engagement on the platform. Access to to all aspects of commerce via the platform will make or break operations at every level for the one stop service for recreation.

User resources

The fourth corner of the one stop service for recreation platform is the user. Providing secure, seamless access to user profiles and history throughout the Recreation.gov experience will be critical to it all working. User resources are central to any modern web or mobile application in 2014 and depends on camping, geo and financial resources to bring it all together.

Recreation.gov website

All of these resource need to be made available on at Recreation.gov, in the form of fully functioning web application providing a content management system (CMS) for managing the site, complete with a reservation system that allows users to discovery camping resources, via mapping geospatial resources, and pay for their reservations via financial resources—all via a secure, easy to access user profile.

Travel planning tools

Beyond the core public website and its reservation system the requirements call for travel planning tools including point-to-point itinerary planning, field sales and reservations sales, third party sales and reservable inventory management. While these tools are likely to primarily be web-based, the requirements call for remote sales and management of inventory, which requires a central API solution.

Robust reporting solutions

Reporting is prominent through the requirements requiring web analytics, canned reports, customizable reports, monthly performance reporting, and enterprise reporting systems. In short, reporting will be essential to all aspects of the one stop service for recreation for both the contract winner and operator, as well as all of the agencies involved. I would add in that end-users will also need some views into their own Recreation.gov usage, adding another layer to the conversation.

Data accessibility and portability

Another element that is present throughout the requirements is data access and portability of the camping, reservation, customer (user) and resulting financial data. Pretty much everything in the system should be accessible, right? This definitely goes beyond just what is available via the RIDB and requiring all aspects of the one-stop service for recreation to be accessible for use in any other system, or just as download.

Minimum required applications and platforms

As part of the requirements, there is a minimum requirements for browsers, and mobile platforms that the one stop service for recreation should operate on:

  • Applications:
    • Microsoft Internet Explorer
    • Mozilla Firefox
    • Google Chrome
    • Apple Safari
  • Platforms:
    • Android Mobile Devices
    • Apple iOS Mobile Devices
    • Apple Desktop OS
    • Microsoft Windows Mobile Devices
    • Microsoft Windows PC

For me, this portion of the requirements are extremely vague. I agree that the Recreation.gov and travel planning tools should work in all these browsers and be responsive on mobile browsing platforms, but these requirements seem to allude to more without actually going there. In 2014-2016, you can’t tell me that native mobile experiences won’t be something end users of Recreation.gov and the overall system will be demanding? Mobile will be the gateway to the platforms mapping and geospatial resources, and APIs are how you deliver the resources needed for developing mobile applications on all of the platforms listed.

A one-stop service for recreation requires a multi-channel approach

The one-stop service for recreation requires a multi-channel approach to deliver the camping, mapping and geo, financial and user resources to the main Recreation.gov website, supporting reservation system, travel planning tools, reporting solutions and potentially any other web or mobile applications, and round it all off with needing to provide the data accessibility and portability that is demanded of a modern federal government platform.

Transform the RIDB web service into a modern API and make the center

The only way you will engage with users Recreation.gov on the web and mobile locations they will demand, as well as deliver the reservation, travel planning and reporting solutions, while making sure everything is portable, the RIDB database and supporting web services needs to be brought up to date and transformed into a modern web API. The RFP_-_Attachment_10_-_Notional_Future_RIDB_Data_Flow_-_Sep._22_2014.pptx even reflects this vision, but the actual RFP severely falls short of describing the role an API will play in the platform.

Making security a priority across all aspects of operations

With a single API layer being the access layer for all web, mobile, reporting and data accessibility channels, security takes on a whole new meaning. APIs provide single access point for all access to platform resources, allowing security and auditing to occur in real-time, using modern approaches to API management. APIs use existing web technology and don’t require any special technology, but do provide a single layer to enable, monitor and deny access to all resources, across the one-stop services for recreation.

A centralized metrics, analytics and reporting layer

Married with security, an API-centric approach allows for a platform-wide analytics layer that can drive security, but also provide the data points required for the overall platform reporting system. With an API analytics layer this data can be delivered to platform, government, developer and end-user reporting solutions. A comprehensive metrics, analytics, and reporting strategy is essential to a modern one stop services for recreation platform.

APIs let in the sunlight, enforcing transparency and accountability

One critical aspect of a modern, API-centric approach is the transparency and accountability it provides. All database connections and web or mobile applications are brokered via an API layer, requiring all applications to rely on the same pipes for reading, writing, modifying and removing data from the system. This approach provides a secure window into all aspects of platform operations, providing the accountability requirements set forth in the proposed RFP.

Some constructive criticism for language in the proposed RFP

There are some points in the proposed RFP that, in my opinion, move the conversation backwards rather than looking to the future and fulfilling a vision of what the “one stop services for recreation platform” could be.

One example is “5.3.6.3 Third Party Sales”:

The Government anticipates that during the potential 10-year period of performance after go-live there may be an interest by commercial travel and recreation planning companies (such as Travelocity, Expedia, etc.) to utilize R1S inventory and availability data to facilitate R1S transactions. The Government may be open to such Third Party Sales arrangements, provided the business, financial and legal terms of the arrangement are in the Government’s best interests. At the Government’s request, the Contractor shall lead exploratory meetings, including Government and applicable industry representatives, to determine commercial interest in such a service and also explore the technical and fiscal feasibility of such Third Party Sales arrangements. Upon the conclusion of all necessary exploratory meetings, and should the Government decide to implement Third Party Sales, the Government will issue a formal modification to the R1S Support Services contract to incorporate the service. There is no requirement for the contractor to provide Third Party Sales services until a post-award modification is issued requiring them.

“There may be an interest by commercial travel and recreation planning companies (such as Travelocity, Expedia, etc.)?” In the current online environment, I am pretty sure that this shouldn’t be “may,” this should be “will,” and that the pace of business today moves quite a bit faster than this section describes. APIs enable a new approach to business development that was coined 10 years ago by the co-founder of Flickr, the popular sharing platform.

I understand that you will have to certify developers, but this should not get in the way of small business developers experimenting and innovating with the resources available via the platform. Business development via an API has been going on in the private sector for 10 years and isn’t something USDA should be putting off until the next procurement cycle for the platform.

The proposed RFP states:

The current contract is a Variable Quantity contract with Fixed Price per-transaction pricing such that the Contractor earns a fixed fee for each transaction processed via the www.recreation.gov website, the telephone customer support line, or in-person at participating Federal recreation locations.  The fixed fees under the current contract vary depending on the type of inventory being reserved, the sales channel, etc.

In an era where we have over 10,000 API resources to use in applications, where cloud-based utility pricing is successfully being applied, and you pay for what you use across numerous industries, you can’t have a business model like this for a public-private sector partnership, and not put APIs to use, and enabling trusted partners to build a business model around the commerce opportunities available via a platform. Right now, this is exclusive to the current contract holder, Active Network, d.b.a. ReserveAmerica, and for the next round should not be something that is exclusive to the contract winner. It should be open to the thousands of businesses that serve the parks and recreation space.

Another item I have an issue with is ownership over the source code of the platform:

The Government owns all of the inventory, transaction and customer data associated with Recreation.Gov; however the reservation system remains the sole proprietary property of the current service provider.

I know this is the current situation, but I strongly urge that for the next relationship, this is not the case. The government should own all of the inventory, transaction and customer data and the reservation system should be open source. Period. The chances that there will be innovation via the platform if it remains a proprietary solution is slim and requiring that the core reservation system and supporting API platform remain open source will stimulate innovation around Recreation.gov. Any business, including the winner of the contract, can still build proprietary solutions that use the API to deliver specific applications, or end-user solutions and premium features, but the core software should be open this time.

Some praise for what is in the proposed RFP

I am not one to just criticize without providing at least some praise, acknowledging where USDA is looking towards the future. One area I have to note is their inclusion of the U.S. Digital Services Playbook, where #3 on the checklist for deploying in a flexible hosting environment is “resources are provisioned through an API,” and #2 on the checklist for defaulting to open says that you should provide data sets to the public, in their entirety, through bulk downloads and APIs (application programming interfaces). I think the U.S. Digital Services Playbook should be included with ALL federal government RFPs, and it makes me happy to see it in here.

I also understand the number of agencies involved in the project include USDA Forest Service, U.S. Army Corps of Engineers, U.S. Department of the Interior (including at least the National Park Service, Bureau of Land Management, and Bureau of Reclamation, Fish and Wildlife Service), the National Archives and Records Administration and other potential federal partners to be determined at a later date.

Agencies involved in the trip planning component include the agencies named above, as well as the Department of Transportation; Federal Highway Administration, the Department of Commerce; National Oceanic and Atmospheric Administration, the Tennessee Valley Authority and Smithsonian Institution. This is a huge undertaking, and I commend them on the effort, but can’t also help myself in throwing in that this is all the more reason the one stop-services for recreation has to have an API. ;-)

In closing

I think I’m finished with my response to the Department of Agricultures’s solicitation number AG3187S151000–Recreation One Stop Support Services. I would like to request that they extend the deadline for response for another 30 days, to allow me to drum up more feedback, as well as put some more thought into the RFP myself. Following your own lead of including the U.S. Digital Services Playbook, I would also recommend bringing in:

  • 18F – Building on the U.S. Digital Services Playbook, 18F has an amazing set of resources to help USDA move forward with this RFP in a way that is in line with other leading platform efforts in the federal government—please make sure 18F is involved.
  • Round 3 PIFs – This is probably already in the works, but make sure there is someone from the recent round of Presidential Innovation Fellows are on board, helping make sure this project is headed in the right direction.

Ultimately, I think all the right parts and pieces are present for this project, but when it comes to  finding the right language, and vision for the RFP, it needs a lot of help, and I’m confident that 18F and PIFs can help bring it home. The most important element for me is that web service from the RIDB needs to be transformed into a modern web API and brought front and center to fuel every aspect of the platform. I’m confident that if the RFP speaks the right language, the winning contractor will be able to translate the RFP into the platform it needs to be, and serve the expectations of the modern consumers who will be using the platform in coming years.

If this RFP goes out the door without an API vision, planning of the family camping trip will done in a silo, at Recreation.gov, and not on the mobile phones that are ubiquitous in our everyday life. User do not need a web experience that is translated to function in mobile browsers, they need a web experience when it makes sense, and a native mobile experience, and even offline when necessary.

Planning the family vacation will not happen from just Recreation.gov. It needs to be part of a larger vacation planning experience, occurring via the online platforms we already use in our daily lives. If USDA focuses on an API-first vision for delivering the one stop service for recreation, it will live up to its name—truly being a one-stop service that can be used for planning your recreation from any platform or device.

Call to action

Visit USDA’s solicitation number AG3187S151000–Recreation One Stop Support Services and ask them to extend the comment period, and let them know how important it is that the platform is API-centric. Feel free to send me a link to your of feedback, and I’ll add a list of relevant stories to the bottom of this post as I find them.

Thank you, Ellen Miller

Ellen Miller (Photo: Sunlight Foundation)

Ellen Miller (Photo: Sunlight Foundation)

Today, Sunlight Foundation announced Chris Gates will take over as its new president in October after co-founder and executive director Ellen Miller said she would step down from eight years at the helm.

“I truly believe that open and equal access to information is the bedrock of democracy,” Ellen wrote in February announcing her departure. “Without it, citizens cannot make informed decisions. With it, citizens learn who and what they can trust. This belief has always been the passion of my life as it will always be the Sunlight Foundation’s goal.”

Because of this belief, everyone in the modern open government movement — from the civic hackers to the federal C-suite to everyone in between who champions the importance of open data — can thank Ellen for being instrumental in driving what is fundamental to civic innovation as we practice and celebrate it today.

I still remember when I first launched GovFresh and, within days, Ellen blogged about it. Having left Washington, D.C. years before for California, I had grown increasingly disenchanted and removed from what happened inside the Beltway. My work with GovFresh has changed that sentiment over the years, and Ellen’s small post was part of the spark that made me think perhaps Washington was getting the disruption it needed, backed less by political mudslinging, and more by a simple, straightforward path to transparency and economic innovation.

Many incredible people in the civic movement have worked under Ellen at Sunlight. From policy to technology, it has become the incubator of open innovation and innovators in Washington, and has produced many people I’ve come to admire beyond just their work there.

The first time I met Ellen was at TransparencyCamp West at Google’s Mountain View headquarters, where I interviewed her about what we then called “Gov 2.0.” I still remember that day well, meeting the Sunlight team, attending my first unconference, and realizing there were now people in Washington “just like me.” If DC was then like it was today, I would probably still be there. For those who consider themselves civic and government innovators within the Beltway, Ellen and Sunlight helped make that happen.

I’m excited about what’s to come with Chris as its new leader, but it’s also bittersweet to see Ellen step down. It’s hard to imagine the open government movement without her but, no matter what she does after, her legacy will continue to inspire the next generation of civic idealists the same way she’s inspired me.

Thank you, Ellen Miller.

GovFresh guide to openFDA

Photo: FDA / Michael J. Ermarth

Photo: FDA / Michael J. Ermarth

The U.S. Food and Drug Administration’s openFDA initiative aims to “make it easier for web developers, researchers, and the public to access large, important public health datasets.”

openFDAInitial areas of focus include adverse event and recall enforcement reports and product labeling data (see “Drugs,” “Devices,” “Foods“). It currently provides an application programming interface of information that was previously unavailable or easily-accessible to the public, including millions of reports submitted to the FDA from 2004 to 2013.

History

The U.S. Department of Health and Human Services identified “Open FDA” as a priority in its open government “Plan Version 3.0.” Led by the FDA’s Office of Informatics and Technology Innovation, the project began in March 2013 and launched a public beta in June 2014.

Timeline

  • March 2013: Taha Kass-Hout named FDA chief health informatics officer.
  • March 2013: Work on openFDA begins.
  • July 2013: White House Presidential Innovation Fellows join to support openFDA development.
  • December 2013: FDA creates the Office of Informatics and Technology Innovation lead by Kass-Hout.
  • June 2014: openFDA launches in beta.

Quotable

“The openFDA initiative leverages new technologies and methods to unlock the tremendous public data and resources available from the FDA in a user-friendly way. OpenFDA is a valuable resource that will help those in the private and public sectors use FDA public data to spur innovation, advance academic research, educate the public, and protect public health.”

– FDA Chief Operating Officer and Acting Chief Information Officer Walter S. Harris

“In the past, these vast datasets could be difficult for industry to access and to use. Pharmaceutical companies, for example, send hundreds of Freedom of Information Act (FOIA) requests to FDA every year because that has been one of the ways they could get this data. Other methods called for downloading large amounts of files encoded in a variety of formats or not fully documented, or using a website to point-and-click and browse through a database – all slow and labor-intensive processes.”

– FDA Chief Health Informatics Officer Taha Kass-Hout

Developers

Connect

References

Open source headlines from the Open Government plans

The Obama Administration’s Open Government Directive ordered Federal agencies to produce open government plans by April 7th, and while some advocates are disappointed, we have before us a bewildering number of initiatives to improve transparency, collaboration, and participation across the Government. It will not surprise you to learn that I spent some time looking for places where open source is being used in these plans.

I’m not sure I can recommend reading all of the plans cover to cover, but if you’re an advocate or have a vested interest in the future of a Federal agency, these plans are fascinating peek into each agency’s interior life. It’s not just the content of the plans, which run from exciting to comical to mundane. You can also learn a great deal about how agencies view themselves from the way these plans are presented and marketed. It will come as no surprise that the Department of Justice’s rather unlovely document spends a lot of time thinking about reducing its FOIA backlog. The Department of Energy clearly understands itself to be a first a research organization, based on its flagship data sets. The Department of Defense plan is crisp, to the point, and focuses on getting the behemoth to better collaborate and interact with other agencies, rather than the public.

The organizational psychology betrayed by these plans is for another post. My interest is in where agencies found open source. I’ve long advocated for open source as a concrete, tangible way to encourage collaboration between agencies and between the government and its citizens. I was pleasantly surprised, frankly, to see how many agencies agree. Here’s what I found, in no particular order.

US Agency for International Development

The USAID plan was a total surprise. I had no idea how many open source initiatives were being conducted by USAID. Page 30 contains this gem on their Global Development Commons work:

With over four billion subscribers in the world, the mobile phone is often the key to connecting and exchanging information with people in developing countries. The 2008 USAID Development 2.0 Challenge, implemented by the Global Development Commons, invited innovators and entrepreneurs from around the world to participate in a global competition to seek access to information and build new connections to the global community. Crowdsourcing and Open Innovation have become increasingly important engines of innovation globally, leveraged by the commercial, non-profit, academic and government sectors to identify opportunities and solve problems. USAID’s Development 2.0 Challenge yielded 115 submissions using high impact, low-cost, open source solutions.

The winner among the 115 submissions was the RapidSMS Child Malnutrition Surveillance system, which “enables health practitioners to share and track children’s nutritional information with the touch of a cell phone.”

The agency also operates the Intra-Health OPEN Initiative, which is “a suite of free open source solutions to supply health sector leaders and managers with a collection of new tools.”

Social Security Administration

The Social Security Administration is another open source underdog. Imagine all the pent-up innovation they can unlock once this project is underway:

We are in the process of creating internal capacity to host websites and applications based on open-source software solutions and we look forward to a lively exchange of ideas and program code within the growing Federal open–source software development community;

As part of SSA’s fifth goal, “making government more sustainable”, they see open source software as an essential tool:

We are a Federal leader in the use of Health Information Technology. Our work with the private sector may yield transferable ideas and tools. We will share our results and products as appropriate. For example:

  • We look forward to sharing the products of our open–source platform efforts across the growing Federal open–source development community, as well as partnering with other agencies in future endeavors; and
  • We are in the process of designing and developing an Electronic Technology Repository for communities of innovation. We expect this repository to employ open–source social networking and other tools to permit users to better manage agency knowledge, avoid unproductive duplication of effort, and share experiences. The repository will support the storage of shared materials and project artifacts, discussion boards, wikis, blogs, subscription feeds, and other pertinent information. We envision sharing these resources with other Federal organizations as well.

Others have criticized open source as being irrelevant to the open government movement, but I think interagency collaboration doesn’t happen anywhere near as often as it should, it can be made easier with open source, and it’s outstanding that the SSA seems to agree.

Department of Defense

The DOD has been using open source software for years. Though I was a bit surprised that it wasn’t explicitly mentioned in the DOD’s open government plan, and even more surprised that the much-hyped forge.mil project isn’t referenced at all, the plan includes a number of initiatives that happen to take advantage of open source tools:

The Wikified Army Field Guide, based on Mediawiki, will allow warfighters to collaboratively edit the Army’s field manuals, allowing the documents to be more accurate and responsive:

…as the battlefield changes rapidly, field manuals must keep pace.  Under the traditional process – in which a select few were charged with drafting and updating field manuals –manuals often failed to reflect the latest knowledge of Soldiers on the ground.

Using the same free software behind Wikipedia, the Army’s “wikified” field manuals invite military personnel – from private to general – to collaboratively update the Army Tactics, Techniques and Procedures Manuals in real time.  In so doing, the Army provides a secure means for battle-tested Soldiers to share their experience and advice from the field.  Wikified Army Field Manuals ensure the men and women who serve our Nation have access to the best possible information when they need it.

This is a very exciting opportunity to capture all the innovation happening “at the edge” and quickly incorporate it into useful, official documentation. It makes so much sense, I’m surprised it hasn’t already been done.

The plan also highlights XMPP, which is a tremendously popular instant-messaging protocol that runs, among others, Google Talk. It may surprise you to learn just how much XMPP’s most popular implementation, Jabber, is already being used inside the DOD. The Defense Connect Online program uses Jabber to provide secure IMs inside the DOD, and they announced in November that this would be opened up to the outside world. Because they standardized on an open standard with robust open source implementations, literally dozens of different chat clients are now available to these non-DOD DCO users.

It’s interesting how both Jabber and the Wiki Field Manual projects aim to improve collaboration, and do so on highly collaborative open source platforms. I don’t think that’s an accident.

Department of Homeland Security

VirtualUSA is DHS’ flagship initiative, which couldn’t be more appropriate. From page 23 of the DHS Open Government plan:

On December 8, 2009, Secretary Janet Napolitano publicly launched Virtual USA (vUSA), an innovative information-sharing initiative that draws on practitioner input to help Federal, State, local and Tribal first responders collaborate to make fast, well-informed decisions. vUSA integrates existing frameworks and investments to provide real-time access to operational information—such as weather conditions; traffic; the location and operational status of critical infrastructure; fuel supplies; availability of emergency shelters and medical facilities; and other critical information—that allows users to improve situational awareness and to respond quickly in emergencies.

vUSA currently operates as two pilots – one in eight southeastern states: Alabama, Georgia, Florida, Louisiana, Mississippi, Texas, Virginia and Tennessee; and the other in five states in the northwest: Alaska, Montana, Idaho, Oregon, and Washington. In Virginia alone, vUSA reduced response times to incidents involving hazardous materials by 70 percent.

VirtualUSA is much more revolutionary than this lets on. It’s a very disruptive piece of software in its space. FCW has a succinct overview of the project’s history and what it means for first responders. DHS is funding this project, based on open source and open standards, in part because it wants to encourage collaborative toolbuilding and cooperation among the states, and also because this capability is too important to be in the hands of a single GIS provider, like Google or ESRI. Because it is an open source project, and uses open standards, VirtualUSA ensures that critical assets are not locked into a single vendor, and simultaneously lower the barrier to entry for new GIS vendors.

Department of Commerce

On page 18 of the Department of Commerce plan, under “Open Source Information Technology”, we find some familiar prose:

Also emerging from Commerce’s Open Government Ideascale community was a suggestion to “become more open through the increased use of open source software.” The Department has already begun using the open source tool, Drupal, for a number of its new websites and plans to increase this use in the future. Using open source technology will allow Commerce to develop new technologies and collaborate more readily with the public and other government agencies, and within the Department itself.

To make this happen, the Office of the Chief Information Officer and the Office of Acquisition Management will be consulted to ensure that open source offerings are fully considered during procurement processes. That consideration will include the value that the Department can receive through increased collaboration with the public and as a contributor to open source communities.

Nothing short of victory at Commerce for Open Source of America, whose suggestion this was. Congratulations!

Department of Labor

You’ll find open source in the strangest places. Until I read Labor’s plan, I didn’t appreciate how much data the Department of Labor is responsible for analyzing and disseminating. With that in mind, it makes perfect sense to find this initiative on page 29:

Create a “Developer’s Corner”

We plan to establish a “Developer Corner” on www.dol.gov/open that specifically targets and engages developers. Our goal is to make it as easy as possible for developers to re-purpose our data, provide feedback, get technical help, bring developers with similar interests together and, ultimately inspire the best possible uses of our data for the benefit of the public. Ideas under consideration include a bug tracking system, RSS feeds for dataset changes, dataset versioning, public code competitions, data authentication, and an ideation platform to prioritize developer needs.

I think every department and agency that distributes data to the public (which is to say, all of them) should follow Labor’s lead and establish their own Developer sites. There’s no better way to stay engaged with this very powerful community.

Health and Human Services

The outstanding NHIN CONNECT project, which has a thriving open source community, got a mention on page 56 of HHS’s plan:

Nationwide Health Information Network – Direct

A key component of the Nation’s emerging health information technology infrastructure is the Nationwide Health Information Network (NHIN) – a set of standards, policies, and services that enable the secure exchange of health information over the Internet. “NHIN Direct” is the latest development in the evolution of the NHIN. It’s an important effort to develop a “lightweight on-ramp” to the NHIN that will enable simple, direct exchanges of information between providers, labs, pharmacies, and consumers — and which will be easy to adopt and implement. In a process that launched on March 1, NHIN Direct is being designed in close collaboration with the community of potential users, with the entire process taking place in the open, in public, on a NHIN Direct wikispace. NHIN Direct will then be implemented in real-world tests and deployments by members of the community – with HHS’s Office of the National Coordinator for Health Information Technology (ONC) playing a coordinating and convening role. The gist of the NHIN Direct strategy is to utilize a community-driven approach to ramp up and power NHIN Direct-powered health information exchange.

I should mention that my employer, Red Hat, is involved in this project.

National Science Foundation

There was nothing explicitly about open source in the NSF’s open government plan, but their plan is worth mentioning anyway, as the NSF already does a tremendous amount of work in the open source community. Here’s a search for “open source” on their web site, which yielded over 5,000 hits when I last tried it:

http://search.nsf.gov/search?access=p&output=xml_no_dtd&sort=date:D:L:d1&ie=UTF-8&btnG=Google+Search&client=NSF&oe=UTF-8&proxystylesheet=NSF2&site=NSF&q=open+source

NSF regularly awards grants under the condition that software developed under those grants is given an open source license. Some very progressive thinking, and shrewd IP stewardship from the NSF folks, so we’ll forgive them for not mentioning open source directly in their plan.

Department of the Treasury

Yet another revelation. Treasury plans to cultivate open source projects to facilitate collaboration between agencies and between Treasury and the public. I’ve written about exactly this kind of collaboration before, back in December, so I’m enormously pleased to see that Treasury agrees. I’ve emphasized my favorite passages here:

In the areas of transparency, participation, collaboration, and flagship initiative, Treasury strives to share its efforts across Government to avoid duplication across agencies and to improve value/outcome of efforts. Treasury seeks to manifest cross-agency transferability in at least two of the following ways:

  • Make training available to other agencies by opening up classes/webcasts to other agencies; providing slides, video and/or audio after the training; and posting on an e-learning platform.
  • Name an advocate who gets the word out about what the agency has to share and invites other agencies to contact that person to learn from him or her.
  • Design procurements for enterprise (where government is the enterprise) or in such a way that what is created can be shared across government at no cost.
  • Develop and post code so it can be shared with other agencies (open source or the contract written such that the government owns the code.)
  • Share platforms utilized by the agency with other agencies at no cost.
  • Create participatory events across agencies with related missions.
  • Collaborate on projects and challenges with the public and with the private sector in partnership with other federal agencies that have similar missions.
  • Share all materials, results, tools, and training that could be transferable to other agencies with the Interagency Working Group as an efficient central dissemination mechanism.

Veteran’s Administration

The VA is an enormous consumer of information technology, and gained early recognition from the open source community for its public domain VISTA electronic health record platform. On page 22 of the VA plan, it becomes clear that the VA is expanding its use of open source to lower the barrier to entry for developers who want to help the agency:

AViVA

A Virtual Installation of VistA Architecture (AViVA) is a recent innovation that we are using to support collaboration. AViVA creates a universal user interface for the electronic health record and includes prototyping of data connectors in order to securely link the AViVA platform to patient data from any source. The AViVA project incorporates HealtheVet as an update of the VistA legacy database system.

VA’s current electronic hospital management system uses a graphical user interface known as the Clinical Patient Record System (CPRS). CPRS data is stored in the legacy data system known as VistA. CPRS requires installation on each machine that operates the program rendering the program difficult to scale and expensive to maintain and update. AViVA’s implementation improves this model in two ways. First, AViVA creates a modular, web-enabled electronic health record system that can be easily and remotely maintained. Second, Medical Data Web Services (MDWS), which can be accessed by the Department of Defense, will allow the creation of applications for any data source to be plugged into the system.

AViVA is a very exciting program for the collaboration portion of our Open Government Plan and because we are committed to creating systems that allow health care providers to collaborate to provide the best care for Veterans. AViVA’s web based presentation layer will allow our doctors and nurses around the country to search patient records as simply and succinctly as you can search for pizza on Google Maps and as securely as the best retail financial service businesses. Additionally, AViVA creates collaboration between VA and DoD, our partner in caring for our nation’s heroes. Finally AViVA creates an open source platform that allows software to be shared with entities outside of VA, creating opportunities for further innovation and development beyond the agency.

National Aeronautic and Space Administration

“NASA is working to make open source software development more collaborative at NASA to benefit both the Agency and the public,” it says right on the first page of the NASA open government plan. Here’s an agency which has always relied on a vibrant research community, software developers, and a culture of innovation. I’m not surprised by their focus on open source, but I am delighted. Among other things, NASA will be sponsoring an open source code competition, has an entire section of their plan devoted to open source development, and will be developing their Nebula cloud computing system on open source software.

It’s fair to say that NASA’s plan is the strongest I’ve seen for the open source community.

Your Turn

Who did I miss? What other opportunities for open source have you found in the open government plans? Leave a comment and let us all know!

Flagship Initiatives Shine in Open Government Plans

Ever since Open Government Day – the 120 day deadline in the Open Government Directive when agencies had to release Open Government Plans – I’ve been pouring over them hoping to get a better understanding of how openness is going to be implemented. If we are to judge government openness by the barrage of documents we received last Wednesday, then we open government advocates ought to be very happy! But what are these documents made of, anyway? A word cloud illustrates it quite well – all the buzzwords that you would expect: Information, government, data, open, public.

(via Wordle)

However, most of the content within the Open Government Plans (at least the ones that I’ve had time to review) are largely aspirational. Each of them reviews the tenets of the OGD, and commit to transparency, collaboration, and participation, but lack the substance and details that will achieve each of these goals. Buzzwords dominate promises to “explore” or “experiment” towards reaching a goal. Agencies are working to plan openness, hoping to explore ideas with the public, and ready to become more transparent – but we’re not seeing the follow-through that we had hoped for in terms of actionable plans or lists of useful data sets the agency will be releasing to the public. Still, one part of each plan tended to be more fleshed out than the rest of the plan: the flagship initiative.

The OGD required agencies to describe at least one new initiative that they would begin implementing soon, and some agencies have already started their efforts towards these flagships. Because of the immediacy of the initiatives, they provide a glimpse of how each agency sees openness within their mission and how they plan to get there. Some are very substantial, providing the timelines and planning necessary to ensure that the project is ready to get underway – or in some cases, are already launched (link).

Flagship initiatives are probably the best indicator of how an agency is approaching the mandates of the Open Government Directive. They are intended to be the most concrete and specific part of the Plans – and typically, they are the only specific and developed plan for new steps towards openness. This is the road that will take agencies from idea to goal.

Some of the flagships are quite impressive, like the DOJ’s FOIA Dashboard, which will centralize and present data about FOIA, while others seem aspirational rather than ready to implement. For example, the Treasury Department hopes to become mostly paperless [link] but doesn’t provide an executable plan or indications of how they hope to achieve that goal. The four paragraphs on their open government flagship barely manage to explain the scope of the project.

Many flagship initiatives follow the general dashboard model: making data about government spending more accessible to the public using web tools and graphical presentation. OSTP (link) will be launching an R&D dashboard, and DOJ (link) has proposed centralizing FOIA data from 92 agencies as a dashboard. Centralizing this data into a dashboard allows easy comparisons between agencies and from year to year. Similarly, the Department of Energy (link) will be releasing energy information for the public in a wiki-style format, alongside educational initiatives.

Others propose opening processes that have not been open to the public, especially in technological ways. The NASA initiative (link) to collaborate with the public in developing their software, as part of an Open Source Software Development Program, will address a problem that’s widely addressed in the geek-oriented side of the open government community – it’s inordinately difficult to try to help out in technology oriented ways. Similarly, OSTP will be starting a “Geeks for Wonks” program to connect technology students and agencies with projects. Creating these avenues for skilled engagement – rather than simply asking for ideas – has the potential to engage those hoping to help who have thus far been shut out.

The most notable success? Agencies are starting to build career champions for transparency and for the use of technology within agencies. Technology can enable the transparency, participation, and collaboration that will make government more effective.

Each of these initiatives has the potential to change the way that agencies do business. That is the intention of the OGD – to change the way that the government interacts with citizens, how agencies collaborate internally, and how much information is available to the public. Clearly, this culture change has been embraced in many agencies, but not everyone has had the time to develop substantive plans. The most promising ones are well developed and are going to make information available, streamline performance, or engage citizens. I have every hope that even the most ambitious of flagships can become a reality, as agencies move forward with their Open Government Plans. I hope that next time around, the plans will have more substance – more concrete, achievable initiatives to go with the grand plans full of buzzwords.

Quick links: Federal agency open government plans

Put down the best-seller and cancel the rest of Spring Break, the White House announced today federal agencies have released their open government plans. The plans are part of the Administration’s Open Government Initiative and should make for great beach reading.

From the White House:

The Plans will make operations and data more transparent, and expand opportunities for citizen participation, collaboration, and oversight. These steps will strengthen our democracy and promote accountability, efficiency and effectiveness across the government.

Here are direct links to all plans (those not linked are not available or not easily visible, so let me know if you find them):

Meet the hackers behind OpenGov Tracker

The federal government may have closed during #snowmageddon 2010, but Jessy Cowan-Sharp and Robbie Schingler didn’t. They created OpenGov Tracker, a Website that tracks citizen ideas for federal agencies related to the Open Government Directive.

Cowan-Sharp shares what inspired them and how they did it.

Why did you create OpenGov Tracker?

In its own way, the public consultation process happening on IdeaScale right now is a historic activity, but so few people know about it. We thought that a single access point would give a sense of the participation on all the different sites, a window into the discussions happening, build some excitement, and inspire people to participate. We also thought maybe a bit of healthy competition would emerge between the different agencies, spurring additional participation. Finally, we wanted to call out and celebrate the ideas of those people who have made valuable contributions, so we promoted the most popular ideas across all agencies.

What’s the development story behind it?

When we realized the IdeaScale site had an API, we grabbed the ideas for the NASA site and started playing around. Seeing that each idea object included counts of comments, votes and lots of other information, we realized it would be easy to pull out those basic stats, calculate a few additional ones and aggregate them for all the agencies. So, we started building. The way the sites are set up, you have to register separately for an API key for each of the agencies, which wasn’t so bad– but of course then it turned out that although each agency has the same set of nominal categories, each is represented by a different category ID in the backend. This makes sense when you realize that IdeaScale is used to supporting multiple, completely stove-piped clients. But that was a fun hour or so of tediously building an index to match up the category names with each agency’s numeric category IDs.

As the number of ideas started going up, we realized that our numbers looked wrong. Upon closer inspection it turned out that the API was truncating result sets at 50. We were worried that as soon as any agency had a category that went above 50 ideas, the site would basically be useless. But IdeaScale was really helpful, and lifted the limit for us. We really appreciate that.

Of course a few agencies chose their own route instead of IdeaScale, so we haven’t included them. I’m of two minds on this. I think it’s great if agencies have their own vision for things and do something different and unique to them, since it shows they’re interested. At the same time, as a developer, it really helps us promote your stuff when there’s a common interface for accessing it. It would be neat to see us collectively put some thought into common interfaces, where feasible, for data objects on government sites and projects.

I always fail to appreciate how time consuming presentation is. Pulling out the data was much easier and faster than tweaking the layout and style. But it’s important you do that well, or obviously no one will stay on the site long enough to look at those numbers. Thankfully, Robbie’s pretty good at that part!

The site is built in using Python with Tornado as the web framework. We’re in the process of adding in MongoDB as a backend data store. It took about two evenings and two full days before we deployed it.

What features do you plan on adding in the future?

Right now the site focuses a lot on numbers. We’re working on a few additions that will bring out more of the actual content to highlight the diversity of contributions. I love looking over the tags and the titles, and appreciating how different the ideas are, how different the focus of each agency is, and how each one has its own microcosm of terminology, challenges and touchy issues. It’s actually really educational to scan the lists of ideas and learn what’s happening in the different agencies.

When we first released the site, it was just what we call a “tinyhack,” a quick and dirty project to get something useful up and running. We weren’t even saving the data. But a lot of people have asked for the ability to look at contributions over time, so now we’re growing up the code a little bit, adding a proper data store on the backend. That will also enable us to easily display trend lines, pull out more content, etc.

But there’s only another 25 days to go, so we need to optimize value provided and time to deployment. That said, we’ll make sure the data continues to be available after the consultation process is over so more fun stuff can be done by those who want to.

Gigglegate: What was so funny?

John Stewart’s Daily Show coverage of the White House Open Government Directive hones in on what everyone’s asking:

‘What was so funny?’

Quotable:

Huge event yesterday. ‘Open for Questions,’ a new experiment in open government debuted on the whitehouse.gov Website, hosted by Kevin Smith, Mike D, and Indian George Clooney. It was, as you can imagine, HIGH-larious.

In the spirit of open government, we’d like an official response from the White House.

Guesses on Gigglegate?

Open gov, Gov 2.0 leaders react to White House Open Government Directive

Here’s what open government and Gov 2.0 leaders are saying about the new White House Open Government Directive.

What’s your take?

Carl Malamud, Public.Resource.Org (@CarlMalamud)

Carl Malamud

“This is great. No equivocating, vacillating, hemming, or hawing. This is all good, big thumbs up to the folks that made this happen.”


Ellen Miller, Sunlight Foundation (@EllnMllr)

Ellen Miller

“The Open Government Directive demonstrates how the Obama administration is matching its aspirational goals with concrete policies and accountability measures. I expect it will create a sea change in how the government and public interact, what information we as citizens have at our fingertips, and that it will redefine that public information means that its online. It’s going to be up to all of us to participate and monitor how well government meets these goals.”


Craig Newmark, Craigslist (@craignewmark)

Craig Newmark

“The Open Government Initiative is a huge commitment to:

  • listening to all Americans, hearing what they have to say
  • telling people what’s going on in government, like where the money goes

The results will create effective large-scale grassroots democracy and far greater fiscal responsibility.

I feel that these efforts are complementary to the adoption of the US Consititution.”


Chris Vein, City and County of San Francisco (@Veinesque)

Chris Vein

The President’s Directive is a tremendous step forward. It not only further explains the President’s vision, but it provides an aggressive roadmap and timeline for getting Federal, State and local governments to improve transparency, increase participation and collaboration. San Francisco is proud to have responded early to the President’s call for open government with our Open Data Directive and DataSF initiatives. The President’s Directive will help San Francisco improve and extend our goal of a more transparent and open City.

Dustin Haisler, City of Manor, TX (@dustinhaisler)

Dustin Haisler

“The Open Government Directive is a great starting point for the open-gov movement in the federal government; however, one thing to consider is whether open data is truly “usable” data for our constituents. Instead of just putting datasets online for mashup artists, we should also focus on the interface our citizens will use to get the information. In addition, multi-agency collaboration starting on the local level will be a very important key to the overall initiative’s success. Overall, I think the directive is good move in the right direction for the federal government.”


Peter Corbett, iStrategyLabs (@corbett3000)

Peter Corbett

“We’ve all been eagerly awaiting the OGD and it’s not a let down by any stretch. It will lend support and clarification to what is a complex issue for our government: how to become more open, transparent and participatory. What we’re seeing here is the innovative use of technology and smart policy to unleash the talent of the American people. I’m most excited about how the work we’ve done on Apps for Democracy will soon be institutionalized throughout federal agencies when OMB releases guidance for how to use challenges, prizes and other incentives for stimulating citizen driven innovation.”


Andrew Wilson, Health & Human Services (@AndrewPWilson)

Andrew Wilson

“This directive represents a significant step toward the president’s goals of transparency, public participation and collaboration. One element that I would like to see emphasized as part of the implementation is a concerted, systemic effort to improve the tools government employees have available to collaborate internally. For me, improved internal collaboration is an essential element to developing the framework for a more fully engaged and responsive government. Imagine a world where cross-departmental information flow was so robust that citizens could interact with ANY agency on ANY issue and could get a timely, complete and helpful response.”


Steve Ressler, GovLoop (@govloop)

Steve Ressler

“Open Government Directive is a great first step in the open gov/Gov 2.0 movement. While the data and transparency piece is important, I’m most interested in how agencies create their own open gov plans and what actions they take from their planning exercise. I believe most of the movement for open gov starts when it is done at the agency level and solving true mission needs.”


Clay Johnson, Sunlight Labs (@cjoh)

Clay Johnson

“This is a great and ambitious plan that’s particularly challenging in terms of both logistics and technology. It is the equivalent of the “putting a man on the moon” of the Transparency movement in the federal government. Challenging, awe-inspiring and risky.”


Adriel Hampton, Gov 2.0 Radio (@adrielhampton)

Adriel Hampton

“I am concerned that some may use the document and its compliance deadlines as a simple checklist. However, as did the president’s January open government memo, this document empowers the growing ranks of Gov 2.0 innovators. Its guidance on data release and standards is also valuable and needed.”


Steve Lunceford, GovTwit (@dslunceford)

Steve Lunceford

“I think this is a great step to formalize a process and “movement” that has already been spreading throughout government. I would have like to have seen more guidance around transparency, participation and collaboration from an interagency standpoint versus just citizen interaction, but believe that could be a natural output as agencies strive to meet the various deadlines. It will also be interesting to see how quickly and enthusiastically agencies respond to a directive which lays out new unfunded mandates given the many priorities they are already juggling.”


Bob Gourley, CTOvision (@bobgourley)

Bob Gourley

The most important part of the directive, in my opinion, is the attachment with guidance on plan formulation. The thought put into that means agencies do not have to recreate the wheel when formulating their own plan. The part of the directive that we all need to watch out for abuse on: it seems to apply to all other than OMB and above. Yet history has shown those are the ones we need the most openness from.


Brian Ahier (@ahier)

Brian Ahier

“I am thrilled to see the emphasis on open government this directive represents. I hope to see government agencies able to meet the deadlines for action established by the Open Government Directive. I also want to see citizen participation in determining the high value data sets to be published. Since this directive also requires the data be published in an open format, it will be nice to have documents available where the data is not shielded within the pdf format.”