Featured open data

California seeks chief data officer

Posted on by Luke Fretwell
California State Capitol Building (Photo: Jeff Turner)

California State Capitol Building (Photo: Jeff Turner)

The state of California is looking for a chief data officer to “promote the availability and use of data in state government.”

The position resides within California Government Operations Agency and will report directly to GovOps Secretary Marybel Batjer.

From GovOps:

This is a Governor’s appointment and review and assessment of applicants will be handled by the Governor’s appointments office. All questions should be referred to the appointments office.

How to apply:

The actual process of applying involves going to the appointments page at gov.ca.gov, and at the bottom of the window, clicking on the “Begin Application” button. At Question #3, “Positions Sought,” scroll to the Gov Ops Agency section, where the position is listed at Gov Ops Agency, Chief Data Officer.

Position description:

Reporting to the Agency Secretary, the Chief Data Officer will have statewide responsibility for three key initiatives based on data collected in the normal course of state business to improve transparency, efficiency and accountability in state operations.

These initiatives are:

  • Developing the statewide open data portal and related governance and policy on standards, storage and privacy, as well as a statewide open data strategic plan and programs to promote civic engagement and innovation.
  • Fostering and promoting a culture of data use by enabling and encouraging departments to share data to collaborate on common issues and related programs.
  • Employing and analyzing operational data to improve program performance.

The Chief Data Officer is the primary steward of the data portal for the state’s public data, which enables public access to data in a variety of formats. The CDO also is responsible for working with state boards, departments and offices to ensure that state data is accessible through the portal. The CDO oversees development of the standards and structure to support these efforts, as well as incorporation of the state’s geo-portal, in consultation with the Department of Technology, into the statewide open data portal. The Chief Data Officer will maintain and expand the state’s data inventory and establish procedures for adding new data sets and regularly updating existing data sets.

The Chief Data Officer will promote opportunities to demonstrate the value of data in decision making; support and encourage events to encourage public use of open data for innovation, and support and encourage activities to enhance collaboration among departments and agencies through shared data.

Desired Skills and Experience:

  • Well-versed in the principles of open data, open government and Government 2.0.
  • Understanding of state government processes and practices (legislative process, budget process, etc.).
  • Technically knowledgeable, with some familiarity with using and building software applications that employ open data.
  • Strong communicator with internal and external stakeholders on deeper technical issues.
  • Understanding of different of open data formats and the pros & cons of different data formats.
  • Understanding of APIs, GIS systems and mapping concepts.
  • Familiarity with different options for open data portals (both commercial & open source).
  • Understanding of process reengineering.
  • Understanding and experience in change management, organizational performance measurement and organizational performance management.
  • Understanding of big data analysis techniques and their application in government setting.
  • Experience in quantitative analysis.

Ideal qualities:

  • Awareness of and experience with open data tools, such as those available through GitHub, and a variety of different data storage technologies.
  • Understanding of specific areas of state activity that are data intensive – such as energy and water regulation, health, public safety, etc.
  • Excellent writing and public speaking skills.

What should governments require for their open data portals?

Posted on by Luke Fretwell

Johns Hopkins University’s new Center for Government Excellence is developing a much-needed open data portal requirements resource to serve as a “set of sample requirements to help governments evaluate, develop (or procure), deploy, and launch an open data web site (portal).”

As many governments ramp up their open data initiatives, this is an important project in that we often see open data platform decisions being made without a holistic approach and awareness of what government should purchase (or have the flexibility to develop on its own).

“The idea here is that any interested city can use this as a baseline and make their own adjustments before proceeding,” said GovEx Director of Open Data Andrew Nicklin via email. “Perhaps with this we can create some common denominators amongst open data portals and eventually push the whole movement forwards.”

My fundamental suggestion is that government-run open data platforms be fully open source. There are a number of technical and financial reasons for this, which I will address in the future, but I believe strongly that if the platform you’re hosting data on doesn’t adhere to the same licensing standards you hold for your data, you’re only doing open data half right.

With both CKAN and DKAN continuing to grow in adoption, we’re seeing an emergence of reliable solutions that adequately meet the same technical and procurement requirements as propriety options (full disclosure: I work with NuCivic on DKAN and NuCivic Data).

Learn more about the GovEx open data portal standards project and post your suggestions.

San Francisco publishes year two plan, continues to lead on open data

Posted on by Luke Fretwell

San Francisco City Hall

San Francisco’s DataSF team continues to quietly and effectively demonstrate what an efficient, holistic and personable approach to open data looks like with the announcement of its year two plan and retrospective of the past year.

SF Chief Data Officer Joy Bonaguro and Open Data Program Manager Jason Lally are a dynamic duo building the blueprint for government open data offices and initiatives. According to the city, DataSF has 264 published datasets and 740 inventoried datasets, but the number isn’t as important as the approach they’ve taken since Bonaguro was appointed a little over a year ago.

Key to their success is having a long-game plan, building internal community, education and awareness and bringing a sense of aesthetic and uniformity to SF’s open data initiative.

Also, I love the “love” messaging throughout the collateral (“Written with LOVE in San Francisco.”, “Made with <3 in San Francisco”), as it humanizes the efforts more, making data less about the technical and more about the people.

Key links for those who should be watching:

It’s time for a national chief data officers council

Posted on by Luke Fretwell
U.S. Chief Data Scientist DJ Patil (Photo: <a href="https://www.flickr.com/photos/oreillyconf/16583022141">O'Reilly Conferences</a>)

U.S. Chief Data Scientist DJ Patil (Photo: O’Reilly Conferences)

As momentum around appointing public sector chief data officers grows, it’s time for the federal government to get ahead of the curve and create a formal chief data officers council similar to, but more inclusive, proactive and public than the already-established U.S. Chief Information Officers Council.

We’ve recently seen a number of federal agencies appoint formal CDO positions, including the Departments of Transportation, Energy and, just last week, Commerce. Even the White House upped the ante and validated the importance of data by naming DJ Patil as the nation’s first chief data scientist.

“Given the substantial benefits that responsibly and creatively deployed data can provide to us and our nation, it is essential that we work together to push the frontiers of data science,” wrote Patil upon his appointment.

While this momentum, including that at the state and local levels, coupled with the work being done through Project Open Data, is inspiring, there is still a lack of national community, purpose and public visibility on a unified direction and momentum.

As public data becomes a larger and important component of government’s service to citizens, it’s imperative that our federal technology leadership take a proactive role in bringing together public data leaders across all levels of government to share best practices and create support and momentum for a national data movement.

The U.S. Chief Data Officers Council, building on objectives established by the CIO Council, should execute on, but not limit itself to, the following:

  • Set implementation, security, privacy and policy guidelines that can be re-purposed across all federal, state and local agencies.
  • Provide metadata standards frameworks that can serve as a guide for governments at federal, state and local levels.
  • Create higher expectations and quarterly reviews of how well Data.gov is successfully delivering on its mission.
  • Publish quarterly report cards on how well federal agencies, states and cities are implementing open data initiatives.
  • Openly and actively communicate, engage and collaborate with the open data community at large on the above.

With a formal U.S. Chief Data Officers Council, we can establish unified, national leadership on the importance and bigger picture of data and expedite its power to truly impact every aspect of our lives.

Per President Obama, Patil says “data science is a team sport.”

Let’s build that team.

U.S. Department of Energy has a new chief data officer

Posted on by Luke Fretwell
Photo: U.S. Department of Energy

Photo: U.S. Department of Energy

According to a U.S. Project Open Data GitHub pull request, it appears the U.S. Department of Energy has named Dave Dutton as its chief data officer.

Dutton has officially held the title since January, and previously worked in data management related positions at Freddie Mac, McDonald’s and EDS.

From his LinkedIn profile related to his new role:

We will offer enterprise data services to eliminate silos, improve the reliability of data, and reduce security risks. We will also provide public-facing data services that are open for all consumers to use. Moreover, we will use digital and traditional mediums to deliver and receive high-value data and information in an easily discoverable, retrievable, and recordable format that promotes transparency with appropriate consumers. We will continue to strengthen our electronic and paper records management capability. All of these efforts will be supplemented by providing training and communications to the DOE workforce on the importance of records management and sharing data and information in a secure manner.

Much like new U.S. Department of Transportation Chief Data Officer Dan Morgan, Dutton has an important and enormous task of managing and making public large amounts of data and leveraging this to impact environmental, energy and sustainability changes throughout the world.

A new way to write to the White House

Posted on by Luke Fretwell
Official White House Photo by Pete Souza

Official White House Photo by Pete Souza

The White House has officially released the write version of the “We the People” application programming interface that now allows developers to feed data back into the petition platform via third-party applications.

“Starting today, people can sign We the People petitions even when they’re not on WhiteHouse.gov,” writes White House Director of New Media Technologies Leigh Heyman on the White House blog. “Now, users can also use third-party platforms, including other petitions services, or even their own websites or blogs. All of those signatures, once validated, will count towards a petition’s objective of meeting the 100,000-signature threshold needed for an official White House response.”

According to Heyman, more than 16 million users have created and/or signed more than 360,000 petitions.

Apply for access to the Petitions API.

Help get USDA to lead with APIs when it comes to America’s parks

Posted on by Kin Lane
Photo: USDA

Photo: USDA

I need your help with something.

I’m in the business of helping startups, all the way up to our federal government, identify valuable assets and make them more accessible for use in websites and mobile devices. As part of this work I’m always on the look out for valuable public assets across city, state and federal government, and help make sure the conversations around these assets always include application programming interfaces, so that we aren’t just building web and mobile applications in silos, and limiting the potential for public access by individuals and small businesses.

Over the last couple years, I have cultivated a small group of API evangelists in various sectors who help keep me informed of important government projects. One of my partners in crime, Alyssa Ravasio (@alyraz), brought a pre solicitation for a request for proposal to my attention from the U.S. Department of Agriculture, for a “Recreation On Stop Support Services” to my attention. You can see this service in action at Recreation.gov. In short, this is an ongoing contract that our federal government engages in with a private sector contractor to provide access to our nation’s campgrounds and campsites at our federal parks.

I would put our national parks and campgrounds at the top of treasured assets that this country possesses, and something that we need to make sure continue to be as accessible to individuals and small businesses as possible. I want to take the time to respond to the pre-solicitation from USDA, in hopes of helping them craft a better RFP before this goes out to bid.

I also want to encourage others, who are also passionate about camping and our national parks, to submit their own feedback to USDA’s solicitation number AG3187S151000–Recreation One Stop Support Services.

The current incarnation of the one stop service for recreation project has an API. Well, it has web services as part of the existing Recreation Information Database, but you also find mentions of API in the proposed RFP, under the 5.3.9 Mapping and Geospatial Capability:

A segment of the user community will not only be accessing and viewing the mapping data through the web interface at www.recreation.gov, but will also have a need to search, retrieve, and download the geospatial data via the RIDB data sharing interface(s), or APIs. The Contractor shall demonstrate utilization of programming languages, formats, data types/conventions/sources, etc. which maximize the portability and usability of maps and map related data by third-parties. This minimizes the need for third-party data consumers to procure expensive and/or proprietary software to utilize, edit or manipulate the mapping data. The contractor shall ensure that all available geospatial data within R1S is incorporated into the user interface to provide users with the most complete mapping solutions available.

There is another mention of APIs under “5.3.6 Provide Travel Planning Tools” with:

Non Federal reservable inventory may include state and local reservable and non-reservable recreation inventory which may be incorporated into the system or accessed via various application programming interfaces (APIs).

Then, towards the end you see the following needs:

  • Data Sharing and Data Management – Information originating from within RIDB and imported shall be presented in a manner such that the external origin of the data is transparent to users and consistent with presentation of information originating from within.
  • Information Sharing – The Contractor shall deliver automated and manual services for the sharing of the consolidated recreation information described herein. The sharing service shall utilize prevailing industry best practices and technologies to make the consolidated recreation information publicly available to data consumers.
  • Information Consolidation – The Contractor shall deliver automated and manual services for collecting of a wide variety of recreation information from a wide range of sources including Federal Government Agencies and Federal Partner organizations such as Historic Hotels of America.

To me, all of this infers that whoever wins the contract will need to at least maintain what is available via the RIDB, which is nice, but it is just a web service, not a modern web API, and only reflects providing access a small portion of the valuable resource that will need to be made available via Recreation.gov to partners and the public at large. We can do better.

An API needs to be a first-class citizen in this RFP, something that is not just an afterthought and a connection to the database that just needs to be maintained — an API will be central to deliver the resources needed for a “one stop services for recreation”.

Camping resources

When you look through the requirements for this project, you see all the building blocks for allowing individuals to engage with campsites across our nation, inventory, ticketing, lotteries, permitting, day use and group facilities, special events and use, equipment rentals, recreational and pass sales. All of this keeps our national parks accessible to campers.

Mapping and geospatial resources

Camping resources are meaningful to us because of where they are located. Mapping and geospatial resources will be vital to enabling users to discover the parks and campsites where they wish to stay on their camping trips. The ability to search and navigate campsites via modern mapping and providing geospatial resources is central to the one stop service for recreation.

Financial resources

A third cornerstone of the recreation one-stop support services platform are the financial resources. The ability to accept and process credit cards, cancellations, modifications and refunds will play a role in every engagement on the platform. Access to to all aspects of commerce via the platform will make or break operations at every level for the one stop service for recreation.

User resources

The fourth corner of the one stop service for recreation platform is the user. Providing secure, seamless access to user profiles and history throughout the Recreation.gov experience will be critical to it all working. User resources are central to any modern web or mobile application in 2014 and depends on camping, geo and financial resources to bring it all together.

Recreation.gov website

All of these resource need to be made available on at Recreation.gov, in the form of fully functioning web application providing a content management system (CMS) for managing the site, complete with a reservation system that allows users to discovery camping resources, via mapping geospatial resources, and pay for their reservations via financial resources—all via a secure, easy to access user profile.

Travel planning tools

Beyond the core public website and its reservation system the requirements call for travel planning tools including point-to-point itinerary planning, field sales and reservations sales, third party sales and reservable inventory management. While these tools are likely to primarily be web-based, the requirements call for remote sales and management of inventory, which requires a central API solution.

Robust reporting solutions

Reporting is prominent through the requirements requiring web analytics, canned reports, customizable reports, monthly performance reporting, and enterprise reporting systems. In short, reporting will be essential to all aspects of the one stop service for recreation for both the contract winner and operator, as well as all of the agencies involved. I would add in that end-users will also need some views into their own Recreation.gov usage, adding another layer to the conversation.

Data accessibility and portability

Another element that is present throughout the requirements is data access and portability of the camping, reservation, customer (user) and resulting financial data. Pretty much everything in the system should be accessible, right? This definitely goes beyond just what is available via the RIDB and requiring all aspects of the one-stop service for recreation to be accessible for use in any other system, or just as download.

Minimum required applications and platforms

As part of the requirements, there is a minimum requirements for browsers, and mobile platforms that the one stop service for recreation should operate on:

  • Applications:
    • Microsoft Internet Explorer
    • Mozilla Firefox
    • Google Chrome
    • Apple Safari
  • Platforms:
    • Android Mobile Devices
    • Apple iOS Mobile Devices
    • Apple Desktop OS
    • Microsoft Windows Mobile Devices
    • Microsoft Windows PC

For me, this portion of the requirements are extremely vague. I agree that the Recreation.gov and travel planning tools should work in all these browsers and be responsive on mobile browsing platforms, but these requirements seem to allude to more without actually going there. In 2014-2016, you can’t tell me that native mobile experiences won’t be something end users of Recreation.gov and the overall system will be demanding? Mobile will be the gateway to the platforms mapping and geospatial resources, and APIs are how you deliver the resources needed for developing mobile applications on all of the platforms listed.

A one-stop service for recreation requires a multi-channel approach

The one-stop service for recreation requires a multi-channel approach to deliver the camping, mapping and geo, financial and user resources to the main Recreation.gov website, supporting reservation system, travel planning tools, reporting solutions and potentially any other web or mobile applications, and round it all off with needing to provide the data accessibility and portability that is demanded of a modern federal government platform.

Transform the RIDB web service into a modern API and make the center

The only way you will engage with users Recreation.gov on the web and mobile locations they will demand, as well as deliver the reservation, travel planning and reporting solutions, while making sure everything is portable, the RIDB database and supporting web services needs to be brought up to date and transformed into a modern web API. The RFP_-_Attachment_10_-_Notional_Future_RIDB_Data_Flow_-_Sep._22_2014.pptx even reflects this vision, but the actual RFP severely falls short of describing the role an API will play in the platform.

Making security a priority across all aspects of operations

With a single API layer being the access layer for all web, mobile, reporting and data accessibility channels, security takes on a whole new meaning. APIs provide single access point for all access to platform resources, allowing security and auditing to occur in real-time, using modern approaches to API management. APIs use existing web technology and don’t require any special technology, but do provide a single layer to enable, monitor and deny access to all resources, across the one-stop services for recreation.

A centralized metrics, analytics and reporting layer

Married with security, an API-centric approach allows for a platform-wide analytics layer that can drive security, but also provide the data points required for the overall platform reporting system. With an API analytics layer this data can be delivered to platform, government, developer and end-user reporting solutions. A comprehensive metrics, analytics, and reporting strategy is essential to a modern one stop services for recreation platform.

APIs let in the sunlight, enforcing transparency and accountability

One critical aspect of a modern, API-centric approach is the transparency and accountability it provides. All database connections and web or mobile applications are brokered via an API layer, requiring all applications to rely on the same pipes for reading, writing, modifying and removing data from the system. This approach provides a secure window into all aspects of platform operations, providing the accountability requirements set forth in the proposed RFP.

Some constructive criticism for language in the proposed RFP

There are some points in the proposed RFP that, in my opinion, move the conversation backwards rather than looking to the future and fulfilling a vision of what the “one stop services for recreation platform” could be.

One example is “5.3.6.3 Third Party Sales”:

The Government anticipates that during the potential 10-year period of performance after go-live there may be an interest by commercial travel and recreation planning companies (such as Travelocity, Expedia, etc.) to utilize R1S inventory and availability data to facilitate R1S transactions. The Government may be open to such Third Party Sales arrangements, provided the business, financial and legal terms of the arrangement are in the Government’s best interests. At the Government’s request, the Contractor shall lead exploratory meetings, including Government and applicable industry representatives, to determine commercial interest in such a service and also explore the technical and fiscal feasibility of such Third Party Sales arrangements. Upon the conclusion of all necessary exploratory meetings, and should the Government decide to implement Third Party Sales, the Government will issue a formal modification to the R1S Support Services contract to incorporate the service. There is no requirement for the contractor to provide Third Party Sales services until a post-award modification is issued requiring them.

“There may be an interest by commercial travel and recreation planning companies (such as Travelocity, Expedia, etc.)?” In the current online environment, I am pretty sure that this shouldn’t be “may,” this should be “will,” and that the pace of business today moves quite a bit faster than this section describes. APIs enable a new approach to business development that was coined 10 years ago by the co-founder of Flickr, the popular sharing platform.

I understand that you will have to certify developers, but this should not get in the way of small business developers experimenting and innovating with the resources available via the platform. Business development via an API has been going on in the private sector for 10 years and isn’t something USDA should be putting off until the next procurement cycle for the platform.

The proposed RFP states:

The current contract is a Variable Quantity contract with Fixed Price per-transaction pricing such that the Contractor earns a fixed fee for each transaction processed via the www.recreation.gov website, the telephone customer support line, or in-person at participating Federal recreation locations.  The fixed fees under the current contract vary depending on the type of inventory being reserved, the sales channel, etc.

In an era where we have over 10,000 API resources to use in applications, where cloud-based utility pricing is successfully being applied, and you pay for what you use across numerous industries, you can’t have a business model like this for a public-private sector partnership, and not put APIs to use, and enabling trusted partners to build a business model around the commerce opportunities available via a platform. Right now, this is exclusive to the current contract holder, Active Network, d.b.a. ReserveAmerica, and for the next round should not be something that is exclusive to the contract winner. It should be open to the thousands of businesses that serve the parks and recreation space.

Another item I have an issue with is ownership over the source code of the platform:

The Government owns all of the inventory, transaction and customer data associated with Recreation.Gov; however the reservation system remains the sole proprietary property of the current service provider.

I know this is the current situation, but I strongly urge that for the next relationship, this is not the case. The government should own all of the inventory, transaction and customer data and the reservation system should be open source. Period. The chances that there will be innovation via the platform if it remains a proprietary solution is slim and requiring that the core reservation system and supporting API platform remain open source will stimulate innovation around Recreation.gov. Any business, including the winner of the contract, can still build proprietary solutions that use the API to deliver specific applications, or end-user solutions and premium features, but the core software should be open this time.

Some praise for what is in the proposed RFP

I am not one to just criticize without providing at least some praise, acknowledging where USDA is looking towards the future. One area I have to note is their inclusion of the U.S. Digital Services Playbook, where #3 on the checklist for deploying in a flexible hosting environment is “resources are provisioned through an API,” and #2 on the checklist for defaulting to open says that you should provide data sets to the public, in their entirety, through bulk downloads and APIs (application programming interfaces). I think the U.S. Digital Services Playbook should be included with ALL federal government RFPs, and it makes me happy to see it in here.

I also understand the number of agencies involved in the project include USDA Forest Service, U.S. Army Corps of Engineers, U.S. Department of the Interior (including at least the National Park Service, Bureau of Land Management, and Bureau of Reclamation, Fish and Wildlife Service), the National Archives and Records Administration and other potential federal partners to be determined at a later date.

Agencies involved in the trip planning component include the agencies named above, as well as the Department of Transportation; Federal Highway Administration, the Department of Commerce; National Oceanic and Atmospheric Administration, the Tennessee Valley Authority and Smithsonian Institution. This is a huge undertaking, and I commend them on the effort, but can’t also help myself in throwing in that this is all the more reason the one stop-services for recreation has to have an API. ;-)

In closing

I think I’m finished with my response to the Department of Agricultures’s solicitation number AG3187S151000–Recreation One Stop Support Services. I would like to request that they extend the deadline for response for another 30 days, to allow me to drum up more feedback, as well as put some more thought into the RFP myself. Following your own lead of including the U.S. Digital Services Playbook, I would also recommend bringing in:

  • 18F – Building on the U.S. Digital Services Playbook, 18F has an amazing set of resources to help USDA move forward with this RFP in a way that is in line with other leading platform efforts in the federal government—please make sure 18F is involved.
  • Round 3 PIFs – This is probably already in the works, but make sure there is someone from the recent round of Presidential Innovation Fellows are on board, helping make sure this project is headed in the right direction.

Ultimately, I think all the right parts and pieces are present for this project, but when it comes to  finding the right language, and vision for the RFP, it needs a lot of help, and I’m confident that 18F and PIFs can help bring it home. The most important element for me is that web service from the RIDB needs to be transformed into a modern web API and brought front and center to fuel every aspect of the platform. I’m confident that if the RFP speaks the right language, the winning contractor will be able to translate the RFP into the platform it needs to be, and serve the expectations of the modern consumers who will be using the platform in coming years.

If this RFP goes out the door without an API vision, planning of the family camping trip will done in a silo, at Recreation.gov, and not on the mobile phones that are ubiquitous in our everyday life. User do not need a web experience that is translated to function in mobile browsers, they need a web experience when it makes sense, and a native mobile experience, and even offline when necessary.

Planning the family vacation will not happen from just Recreation.gov. It needs to be part of a larger vacation planning experience, occurring via the online platforms we already use in our daily lives. If USDA focuses on an API-first vision for delivering the one stop service for recreation, it will live up to its name—truly being a one-stop service that can be used for planning your recreation from any platform or device.

Call to action

Visit USDA’s solicitation number AG3187S151000–Recreation One Stop Support Services and ask them to extend the comment period, and let them know how important it is that the platform is API-centric. Feel free to send me a link to your of feedback, and I’ll add a list of relevant stories to the bottom of this post as I find them.

Register for Data Transparency 2014

Posted on by Luke Fretwell

Data Transparency 2014The Data Transparency Coalition will host Data Transparency 2014 on Tuesday, September 30, in Washington, D.C.

Speakers include U.S. Deputy Chief Technology Officer Nick Sinai, U.S. Comptroller General Gene Dodaro, Office of Management and Budget Controller David Mader and Department of Treasury Data Transparency Executive Director Christina Ho.

Official Twitter hashtag for the event is #DT2014. There’s also an iPhone app available to attendees.

Register here.

An open data blueprint for the U.S. Department of Commerce

Posted on by Kin Lane
U.S. Secretary of Commerce Penny Pritzker announcing the agency's new chief data officer position. (Photo: U.S. Secretary of Commerce)

U.S. Secretary of Commerce Penny Pritzker announcing the agency’s new chief data officer position. (Photo: U.S. Secretary of Commerce)

Re-published from API Evangelist

U.S. Secretary of Commerce Penny Pritzker recently announced the Department of Commerce will hire its first-ever chief data officer. I wanted to make sure that when this new and extremely important individual assumes their role, they have my latest thoughts on how to make the Department of Commerce developer portal the best it possibly can be, because this will be the driving force behind the rapidly expanding API driven economy.

Secretary Pritzker does a pretty good job of summing up the scope of resources that are available at Commerce:

Secretary Pritzker described how the Department of Commerce’s data collection – which literally reaches from the depths of the ocean to the surface of the sun – not only informs trillions of dollars of private and public investments each year and plants the seeds of economic growth, but also saves lives.

I think she also does a fine job of describing the urgency behind making sure Commerce resources are available:

Because of Commerce Department data, Secretary Pritzker explained, communities vulnerable to tornadoes have seen warning times triple and tornado warning accuracy double over the past 25 years, giving residents greater time to search for shelter in the event of an emergency.

To understand the importance of content, data and other resources that are coming out the Department of Commerce, you just have to look at the list of agencies under its purview that already have API initiatives:

Then take a look at the other half, who have not launched APIs:

The data and other resources available through these agencies reflect the heart of not just the U.S. economy, but the global economy, which is rapidly being driven by APIs powering stock markets, finance, payment providers, cloud computing and many other cornerstones of our increasingly online economy.

Look through those 13 agencies. The resource they manage are vital to all aspects of the economy: telecommunications, patents, weather, oceans, census, to other areas that have a direct influence on how markets work (or don’t).

I’m all behind the Commerce hiring a CDO, but my first question is, “what will this person do?”

This leader, Secretary Pritzker explained, will oversee improvements to data collection and dissemination in order to ensure that Commerce’s data programs are coordinated, comprehensive, and strategic.

Yes! I can get behind this. In my opinion, in order for the new CDO to do this, they will have to quickly bring all of the agencies /developer programs up to a modern level of operation. There is a lot of work to be done, so let’s get to work exploring what needs to happen.

A central Commerce developer portal to rule them all

Right now, the Commerce developer portal, commerce.gov/developer, is just a landing page. An after thought, to help you find some APIs–not a portal.

The new CDO needs to establish this real estate as the one true portal, which provides the resources other agencies will need for success, while also providing a modern, leading location for developers of web, mobile, Internet of things applications and data journalists or analysts to find the data they need.

If you need a reference point, look at Amazon Web Services,SalesForce, eBay or Googe’s developers areas—you should see this type of activity at commerce.gov/developer.

Each agency must have its own kick-ass developer portal

Following patterns set forth by Commerce, each sub-agency needs to possess their own best-of-breed developer portal, providing the data, APIs, code and other resources that public and private sector consumers will need. I just finished looking through all the available developer portals for commerce agencies, and there is no consistency between them in user experience, API design or resources available. The new CDO will have to immediately get to work on taking existing patterns from the private sector, as well as what has been developed by 18F, and set a establish common patterns that other agencies can follow when designing, developing and managing their own agencies developer portal.

High-quality, machine-readable open data by default

The new CDO needs to quickly build on existing data inventory efforts that has been going on at Commerce, making sure any existing projects, are producing machine-readable data by default, making sure all data inventory is available within their agency’s portal, as well as at data.gov. This will not be a one-time effort. The new CDO needs to make sure all program and project managers, also get the data steward training they will need, to ensure that all future work at Commerce, associated agencies and private sector partners produces high-quality, machine-readable data by default.

Open source tooling to support the public and private sector

Within each of the Commerce and associate agency developer portals, there needs to be a wealth of open source code samples, libraries and SDKs for working with data and APIs. This open source philosophy, also needs to be applied to any web or mobile applications, analysis or visualization that are part of Commerce funded projects and programs, whether they are from the public or private sector. All software developed around Commerce data, and receive public funding should be open source by default, allowing the rest of the developer ecosystem, and ultimately the wider economy to benefit and build on top of existing work.

Machine-readable API definitions for all resources

This is an area that is a little bit leading edge, even for the private sector, but is rapidly emerging to play a central role in how APIs are designed, deployed, managed, discovered, tested, monitored and ultimately integrated into other systems and applications. Machine-readable API definitions are being used as a sort of central truth, defining how and what an API does, in a machine-readable, but common format, that any developer, and potentially other system can understand. Commerce needs to ensure that all existing, as well as future APIs developed around Commerce data, possess a machine-readable API definition, which will allow for all data resources to be plug and play in the API economy.

Establish an assortment of blueprints for other agencies to follow

The new Commerce CDO will have to be extremely efficient at establishing successful patterns that other agencies, projects and programs can follow. This starts with developer portal blueprints they can follow when designing, deploying and managing their own developer programs, but should not stop there, and Commerce will need a wealth of blueprints for open source software, APIs, system connectors and much, much more. Establishing common blueprints, and sharing these widely across government will be critical for consistency and interoperability–reducing the chances that agencies, or private sector partners will be re-inventing the wheel, while also reducing development costs.

Establish trusted partner access for public and private sector

Open data and APIs do not always mean publicly available by default. Private sector API leaders have developed trusted partner layers to their open data and API developer ecosystems, allowing for select, trusted partners greater access to resources. An existing model for this in the federal government is within the IRS modernized e-file ecosystem, and the trusted relationships they have with private sector tax preparation partners like H&R Block or Jackson Hewitt. Trusted partners will be critical in Commerce operations, acting as private sector connectors to the API economy, enabling higher levels of access from the private sector, but in a secure and controlled way that protects the public interest.

Army of domain expert evangelists providing a human face

As the name says, Commerce spans all business sectors, and to properly “oversee improvements to data collection and dissemination in order to ensure that Commerce’s data programs are coordinated, comprehensive, and strategic,” the CDO will need another human layer to help increase awareness of Commerce data and APIs, while also supporting existing partners and integrators. An army of evangelists will be needed, possessing some extremely important domain expertise, across all business sectors, that Commerce data and resources will touch. Evangelism is the essential human variable, that makes the whole open data and API algorithm work, the new CDO needs to get to work writing a job description, and hiring for this army—you will need an 18F, but one that is dedicated to Commerce.

Department of Commerce as the portal at the center of the API economy

The establishment of an official CDO at the Department of Commerce is very serious business, and is a role that will be central to how the global economy evolves in the coming years. The content, data, and digital resources that should, and will be made available at commerce.gov/developer and associated agencies, will be central to the health of the API driven economy.

Think of what major seaports have done for the the economy over the last 1,000 years, and what role Wall Street has played in the economy over the last century. This is the scope of the commerce.gov/developer portal, which is ultimately the responsibility of this new role.

When the new CDO gets started, I hope they reach out to 18F, who will have much of what you need to get going. Then sit down, read this post, as well my other one on, An API strategy for the U.S. government, and once you get going, if you need any help, just let me know—as my readers know, I’m full of a lot of ideas on APIs.