An open data blueprint for the U.S. Department of Commerce

[caption id=”attachment_17921” align=”alignnone” width=”800”] U.S. Secretary of Commerce Penny Pritzker announcing the agency’s new chief data officer position. (Photo: U.S. Secretary of Commerce)[/caption]

Re-published from API Evangelist

U.S. Secretary of Commerce Penny Pritzker recently announced the Department of Commerce will hire its first-ever chief data officer. I wanted to make sure that when this new and extremely important individual assumes their role, they have my latest thoughts on how to make the Department of Commerce developer portal the best it possibly can be, because this will be the driving force behind the rapidly expanding API driven economy.

Secretary Pritzker does a pretty good job of summing up the scope of resources that are available at Commerce:

Secretary Pritzker described how the Department of Commerce’s data collection – which literally reaches from the depths of the ocean to the surface of the sun – not only informs trillions of dollars of private and public investments each year and plants the seeds of economic growth, but also saves lives.

I think she also does a fine job of describing the urgency behind making sure Commerce resources are available:

Because of Commerce Department data, Secretary Pritzker explained, communities vulnerable to tornadoes have seen warning times triple and tornado warning accuracy double over the past 25 years, giving residents greater time to search for shelter in the event of an emergency.

To understand the importance of content, data and other resources that are coming out the Department of Commerce, you just have to look at the list of agencies under its purview that already have API initiatives:

• Bureau of Economic Analysis
• Bureau of the Census
• International Trade Administration (ITA)
• National Institute of Standards and Technology (NIST)
• National Oceanic and Atmospheric Administration (NOAA)
• National Telecommunications and Information Administration
• Patent and Trademark Office

Then take a look at the other half, who have not launched APIs:

• Bureau of Industry and Securityy
• Committee for the Implementation of Textile Agreements
• Economics and Statistics Administration
• Economic Development Administration
• Minority Business Development Agency
• National Technical Information Service

The data and other resources available through these agencies reflect the heart of not just the U.S. economy, but the global economy, which is rapidly being driven by APIs powering stock markets, finance, payment providers, cloud computing and many other cornerstones of our increasingly online economy.

Look through those 13 agencies. The resource they manage are vital to all aspects of the economy: telecommunications, patents, weather, oceans, census, to other areas that have a direct influence on how markets work (or don’t).

I’m all behind the Commerce hiring a CDO, but my first question is, “what will this person do?”

This leader, Secretary Pritzker explained, will oversee improvements to data collection and dissemination in order to ensure that Commerce’s data programs are coordinated, comprehensive, and strategic.

Yes! I can get behind this. In my opinion, in order for the new CDO to do this, they will have to quickly bring all of the agencies /developer programs up to a modern level of operation. There is a lot of work to be done, so let’s get to work exploring what needs to happen.

A central Commerce developer portal to rule them all

Right now, the Commerce developer portal, commerce.gov/developer, is just a landing page. An after thought, to help you find some APIs–not a portal.

The new CDO needs to establish this real estate as the one true portal, which provides the resources other agencies will need for success, while also providing a modern, leading location for developers of web, mobile, Internet of things applications and data journalists or analysts to find the data they need.

If you need a reference point, look at Amazon Web Services,SalesForce, eBay or Googe’s developers areas—you should see this type of activity at commerce.gov/developer.

Each agency must have its own kick-ass developer portal

Following patterns set forth by Commerce, each sub-agency needs to possess their own best-of-breed developer portal, providing the data, APIs, code and other resources that public and private sector consumers will need. I just finished looking through all the available developer portals for commerce agencies, and there is no consistency between them in user experience, API design or resources available. The new CDO will have to immediately get to work on taking existing patterns from the private sector, as well as what has been developed by 18F, and set a establish common patterns that other agencies can follow when designing, developing and managing their own agencies developer portal.

High-quality, machine-readable open data by default

The new CDO needs to quickly build on existing data inventory efforts that has been going on at Commerce, making sure any existing projects, are producing machine-readable data by default, making sure all data inventory is available within their agency’s portal, as well as at data.gov. This will not be a one-time effort. The new CDO needs to make sure all program and project managers, also get the data steward training they will need, to ensure that all future work at Commerce, associated agencies and private sector partners produces high-quality, machine-readable data by default.

Open source tooling to support the public and private sector

Within each of the Commerce and associate agency developer portals, there needs to be a wealth of open source code samples, libraries and SDKs for working with data and APIs. This open source philosophy, also needs to be applied to any web or mobile applications, analysis or visualization that are part of Commerce funded projects and programs, whether they are from the public or private sector. All software developed around Commerce data, and receive public funding should be open source by default, allowing the rest of the developer ecosystem, and ultimately the wider economy to benefit and build on top of existing work.

Machine-readable API definitions for all resources

This is an area that is a little bit leading edge, even for the private sector, but is rapidly emerging to play a central role in how APIs are designed, deployed, managed, discovered, tested, monitored and ultimately integrated into other systems and applications. Machine-readable API definitions are being used as a sort of central truth, defining how and what an API does, in a machine-readable, but common format, that any developer, and potentially other system can understand. Commerce needs to ensure that all existing, as well as future APIs developed around Commerce data, possess a machine-readable API definition, which will allow for all data resources to be plug and play in the API economy.

Establish an assortment of blueprints for other agencies to follow

The new Commerce CDO will have to be extremely efficient at establishing successful patterns that other agencies, projects and programs can follow. This starts with developer portal blueprints they can follow when designing, deploying and managing their own developer programs, but should not stop there, and Commerce will need a wealth of blueprints for open source software, APIs, system connectors and much, much more. Establishing common blueprints, and sharing these widely across government will be critical for consistency and interoperability–reducing the chances that agencies, or private sector partners will be re-inventing the wheel, while also reducing development costs.

Establish trusted partner access for public and private sector

Open data and APIs do not always mean publicly available by default. Private sector API leaders have developed trusted partner layers to their open data and API developer ecosystems, allowing for select, trusted partners greater access to resources. An existing model for this in the federal government is within the IRS modernized e-file ecosystem, and the trusted relationships they have with private sector tax preparation partners like H&R Block or Jackson Hewitt. Trusted partners will be critical in Commerce operations, acting as private sector connectors to the API economy, enabling higher levels of access from the private sector, but in a secure and controlled way that protects the public interest.

Army of domain expert evangelists providing a human face

As the name says, Commerce spans all business sectors, and to properly “oversee improvements to data collection and dissemination in order to ensure that Commerce’s data programs are coordinated, comprehensive, and strategic,” the CDO will need another human layer to help increase awareness of Commerce data and APIs, while also supporting existing partners and integrators. An army of evangelists will be needed, possessing some extremely important domain expertise, across all business sectors, that Commerce data and resources will touch. Evangelism is the essential human variable, that makes the whole open data and API algorithm work, the new CDO needs to get to work writing a job description, and hiring for this army—you will need an 18F, but one that is dedicated to Commerce.

Department of Commerce as the portal at the center of the API economy

The establishment of an official CDO at the Department of Commerce is very serious business, and is a role that will be central to how the global economy evolves in the coming years. The content, data, and digital resources that should, and will be made available at commerce.gov/developer and associated agencies, will be central to the health of the API driven economy.

Think of what major seaports have done for the the economy over the last 1,000 years, and what role Wall Street has played in the economy over the last century. This is the scope of the commerce.gov/developer portal, which is ultimately the responsibility of this new role.

When the new CDO gets started, I hope they reach out to 18F, who will have much of what you need to get going. Then sit down, read this post, as well my other one on, An API strategy for the U.S. government, and once you get going, if you need any help, just let me know—as my readers know, I’m full of a lot of ideas on APIs.