Month: September 2015

Participation and the cult of catalogs

“Anonymous access to the data must be allowed for public data, including access through anonymous proxies. Data should not be hidden behind ‘walled gardens.’”
8 Principles of Open Government Data

In the world of open data, there are few things that carry more weight than the original 8 principles of open data.

Drafted by a group of influential leaders on open data that came together in Sebastopol, Calif., in 2007, this set of guidelines is the defacto standard for evaluating the quality of data released by governments, and is used by activists regularly to prod public organizations to become more open.

With this in mind, it was intriguing to hear a well-known champion of open data at the Sunlight Foundation’s recent TransparencyCamp in Washington, D.C. raise some interesting questions about one of these principles, typically considered sacrosanct in the open data community.

Andrew Nicklin (formerly at the helm of open data efforts for both the City and State of New York, and now Open Data Director for the Center for Government Excellence at Johns Hopkins University) asked TransparencyCamp attendees to consider some of the implications of the sixth principle on open data – which calls for non-discriminatory access to data. This principle is generally taken to mean that users of open data should be able to access it anonymously and that governments should not require users to identify who they are or what they plan to do with the data as a condition of accessing it.

While there is obvious merit to this principle, Andrew observed that when governments know who is using their data and how they are using it, there are enormous opportunities to enhance the data and make it more useful for data consumers. If governments don’t understand what user’s want, providing useful data that can meet their needs is difficult – strictly enforcing anonymous access to data may end up being be an impediment to better understanding what data users actually need.

Without being directly critical of the principle or the original intentions behind it, Andrew made a thoughtful suggestion for open data advocates at TransparencyCamp to consider. To me, these comments highlight an important issue facing the civic technology community and governments themselves – one that almost no one is talking about.

When it comes to building the infrastructure of open data – putting in place the pieces of technology that users will leverage to find and use government open data – very little thought seems to be given to what users – data consumers – want or need.

The idea of “build with, not for” has become a central tenant to how civic technology solutions are designed and implemented. Yet this idea seldom applies to the platforms that governments use to make open data available, which form the foundation of many civic technology solutions.

Costs and benefits

“Funding is the most cited barrier to implementing or expanding open data initiatives.”
Empowering the Public Through Open Data

A recent collaborative effort between the University of Southern California’s Annenberg Center on Communication Leadership & Policy and the USC Price School of Public Policy produced a hugely valuable report on the current state of open data in the 88 incorporated cities comprising Los Angeles County.

Based on surveys and interviews with city officials on their open data efforts, this report provides unique insights into the ways that government leaders view open data. Among the findings – government officials surveyed for the report consider funding to be the most significant barrier to expanding work on open data. This isn’t a surprise, and this sentiment is likely not unique to the Los Angeles County area.

But when taken together with other findings, it can seem counterintuitive. Along with citing funding as a constraint, government officials expressed a preference for commercial open data catalogs over open source (or free) alternatives. These commercial solutions – some of which impose non-trivial costs on local governments – appear to meet a perceived need on the part of government officials in that they are viewed as making it “easier to publish [data] and put it in the hands of the citizens.”

Commercial software generally tends to fare better in the government procurement process than open source software, so this outcome isn’t all that shocking. But it’s worth noting this contradiction in the findings of the USC report between the cost constraints limiting more progress on open data and the reported preference for (sometimes pricy) commercial open data catalogs.

Cost aside, there are a few reasons why upfront investment in a commercial open data catalog may not be the best way to start a new open data effort.

Architecting participation 

The web … took the idea of participation to a new level, because it opened participation not just to software developers but to all users of the system.
– Tim O’Reilly, The Architecture of Participation

First, and somewhat ironically, public information on the cost of commercial open data portals can be hard to come by. Another report on municipal open data efforts in southern California found a wide disparity in what different governments – some just a few miles apart, and almost identical in population – pay for commercial open data catalogs. This can make it difficult for governments to know if they are getting good value for the price being paid.

In addition, commercial open data catalogs often come with visualization, mapping and charting tools out of the box. This can make it easier for governments to augment open data offerings by showing what can be done with it. Though these offerings may come at an additional price, some may view them as a way to help advocate open data to internal skeptics – a picture (or a graph, or a chart) is worth a thousand words as the saying goes.

From a user needs perspective, this approach feels very unidirectional – this is government telling the data community what it believes is important, not the other way around. There are a host of examples of sophisticated visualizations and applications being built with government data by outside data users. And while this approach requires outreach and engagement, there is an ever-increasing abundance of tools available for members of the data community to use to create maps, visualizations and new applications.

These two approaches – out of the box vs. community built – are not mutually exclusive. We can see a number of examples of governments using commercial open data catalogs to engage with external data users that produce useful, valuable visualizations and apps – New York City, the City of Los Angeles, Chicago and San Francisco are all great examples of this dual approach.

However, open data efforts in all of those cities have benefited from robust technology and startup communities and often visionary leadership. Almost all of these cities have a long tradition of civic hacking. For cities that don’t have these assets (or have them in smaller quantities), outreach and engagement to nurture and build a data community will be a crucial factor in the long-term success of an open data program. These cities – many of them smaller and with more limited resources – may also feel the cost constraints of implementing an open data effort more acutely than larger cities.

It’s fair to say that the next wave of cities that adopt open data programs may face a very different set of challenges than the cities that have come before them.

Putting Users First

“The procurement model of government digital services generally leads to services that satisfy policy needs, not user needs.”
Government Technology Procurement Playbook, Code for America

The time feels right to rethink how cities put in place the basic infrastructure of open data.

At last year’s Code for America Summit, I gave a talk on how open data was being adopted in small to midsized cities in the U.S. In researching my talk, I found that while larger cities have almost all implemented some form of open data program, less than 20% of the 256 incorporated places in this U.S. with populations between 100,000 and 500,000 have an open data program.

Open data in this country is still – almost exclusively – a big city phenomenon.

Efforts to address this imbalance are underway – the What Works Cities initiative (of which the Center for Government Excellence at Johns Hopkins is a key part) is now working to bring open data and data-driven decision making to 100 mid-sized cities. More and more, small and mid-sized cities are starting to look at open data as a key driver of government innovation.

We are now at a juncture where we can not only help a new cohort of cities adopt open data, but to help ensure that these efforts embrace the principle of “build with, not for” from the ground up. If we’re going to be successful, it’s important that we question long-held beliefs – like the original 8 principles of open data – to ensure our efforts are most efficiently aligned with the outcomes we desire.

It’s worth considering whether commercial open data catalogs provide the best option for the next wave of cities that are embracing open data to succeed and build a healthy data culture, both inside and outside of government.

But whatever foundation we choose to lay for the next phase of open data, we’ll need to make sure we’re putting user’s needs first.

(Note – the term “cult of catalogs” is not my own. I first heard it used by Friedrich Lindenberg, though others may have used it as well.)

USDS publishes design standards for federal government websites

U.S. Web Design Standards

U.S. Web Design Standards

The U.S. Digital Service has soft-launched new U.S. Web Design Standards to “create consistency and beautiful user experiences across U.S. federal government websites.”

The standards, similar to popular design frameworks like Bootstrap and Foundation, include a style guide and downloadable code assets aimed at creating a unified, mobile-friendly citizen experience and making it easier for developers to deploy across all government websites.

With the new standards comes a slight modification to play three of the U.S. Digital Services Playbook (“Make it simple and intuitive”) from “Create or use an existing, simple, and flexible design style guide for the service” to “Use a simple and flexible design style guide for the service. Use the U.S. Web Design Standards as a default.”

Work on the standards began earlier this year with the pattern library and, as I’ve mentioned before, the federal government could realize millions of dollars in savings by simply re-purposing a unified framework instead of re-inventing the design wheel. Much like what Bootstrap and Foundation have done to exponentially expedite development and go-to-market launches for countless startups and their products, so too could the U.S. Web Design Standards for federal government digital services.

Update from 18F:

Thank you, Jake

Jake Brewer speaking at PDF 2014. (Photo: Personal Democracy Media)

Jake Brewer speaking at PDF 2014. (Photo: Personal Democracy Media)

I’m still stunned and heartbroken by the news that we’ve lost Jake Brewer.

Jake and I met six years ago at TransparencyCamp West, where I interviewed him and was able to capture a little bit of how he saw the world, and what needed to be done to make it better.

The last time we communicated was via text on his first day in his new role with the White House and, in between those years, as he has with so many others, he always inspired me with his graceful approach to change and impact.

After I got the news Sunday morning, I started to watch the interview and broke down in disbelief and sadness that I still feel as I write this.

There have been countless, beautiful anecdotes on Jake’s compassion, humility and contributions, and there’s nothing I can add that would do justice to honor the influence he’s had on me other than to say, Jake, I miss you so much, and you will be with me always as I try to live up to the standards you set for those of us still here.

Related: Micah Sifry has a great tribute and excellent anecdote. President Obama issued a statement, and U.S. Chief Technology Officer Megan Smith expanded with a White House blog post. Friends have set up a memorial education fund for his children.

Retrospecting agile government

Last week, I was in New Orleans for the CivicActions corporate retreat, and presented to the team on the work done around Agile Government Leadership, and I wanted to share some of this here because, while there’s still much to do around changing government project management practices, we’ve accomplished a great deal over the past 18 months since its inception.

Here’s an overview:


  • Momentum: This isn’t necessarily a reflection on the work we’re doing, but more the reality on the time it takes to bring awareness and influence culture change. We’ve seen this with government adoption of open source and open data, and changing project management practices, especially when procurement policies play a big role, will take time.

Next steps:

We’re currently conducting a survey on how AGL can best address the community needs, so please take the survey.

Additional thoughts:

When you’re in the weeds building something, you don’t always take the time to fully step back and take a holistic look at what’s been accomplished, but I’m really proud of the work that’s been done by so many people. Big shout out and thank you to the AGL steering committee and my colleagues Elizabeth Raley, Henry Poole, Aaron Pava, Bill Ogilvie and the entire CivicActions team for supporting this initiative.

AGL is proof that true civic innovation is about collaboration across the public and private sectors, focusing on impact, adding value and, when it comes to transforming government, believing in the long game.

What should governments require for their open data portals?

Johns Hopkins University’s new Center for Government Excellence is developing a much-needed open data portal requirements resource to serve as a “set of sample requirements to help governments evaluate, develop (or procure), deploy, and launch an open data web site (portal).”

As many governments ramp up their open data initiatives, this is an important project in that we often see open data platform decisions being made without a holistic approach and awareness of what government should purchase (or have the flexibility to develop on its own).

“The idea here is that any interested city can use this as a baseline and make their own adjustments before proceeding,” said GovEx Director of Open Data Andrew Nicklin via email. “Perhaps with this we can create some common denominators amongst open data portals and eventually push the whole movement forwards.”

My fundamental suggestion is that government-run open data platforms be fully open source. There are a number of technical and financial reasons for this, which I will address in the future, but I believe strongly that if the platform you’re hosting data on doesn’t adhere to the same licensing standards you hold for your data, you’re only doing open data half right.

With both CKAN and DKAN continuing to grow in adoption, we’re seeing an emergence of reliable solutions that adequately meet the same technical and procurement requirements as propriety options (full disclosure: I work with NuCivic on DKAN and NuCivic Data).

Learn more about the GovEx open data portal standards project and post your suggestions.

Take the 2015 Agile Government Survey

Agile Government LeadershipAgile Government Leadership wants to learn more about the state of agile project management in government, its challenges and successes, and how AGL can better support the community.

The 2015 Agile Government Survey is now open to public sector employees and private sector vendors across all professional areas of expertise (executive, procurement, project managers, developers).

Please take a few minutes to participate in the survey and share with your colleagues.