Apps for Democracy

Open government hackathons matter

Open government hackathons matter

The civic hackathon – a gathering (either virtual or physical) of technologists for a few days or weeks to build civic-themed software – remains one of the more durable manifestations of the open government movement.

Hardly a week passes without the announcement of a new event or contest – sometimes more than one. As I’ll explain more fully in a moment, this is a good thing.

The civic hackathon is also, increasingly, one of more analyzed facets of the open government movement.

There are more and more smart, engaged people talking about ways to make civic hackathons better – to help ensure that the software these events produce is of higher quality and has a longer lasting effect. This is also a good thing.

Some of the more enlightened analyses on methods/strategies for improving civic hackathons that have crossed my radar of late (by no means a complete list) are the following:

Also worth a read is a recent post on TechPresident by Nick Judd (always a thoughtful contributor on this topic).

In reading much of what is written on the subject of civic hackathons lately, it’s easy to take away a feeling of concern – even skepticism – about their real value.

The constant lament I hear is that civic hackathons don’t work (or don’t work well enough) because many of the apps that are developed as part of these events are not sustained long-term. Some don’t survive the weekend.

I have for some time tried to dispel the notion that this is the only measure (or even one of the most important) of a civic hackathon’s success. And in this post, I will try again.

I <3 Hackathons

I’ve got a thing for civic hackathons.

I was a competitor in the very first Apps for Democracy that took place under Vivek Kundra in Washington, DC, and I was also a competitor in the first Apps for America contest put on by the Sunlight Foundation.

Since then, I’ve been a participant in lots of other civic hackathons and coding events as either a participant, organizer and sponsor (sometimes as more than one).

I’m currently organizing a Philadelphia civic hackathon and helping to organize another in Baltimore. I am a part of not one, but two entries in the FCC’s Apps for Communities competition.

Yeah, I like hackathons.

This doesn’t always make me the most objective person in discussions about whether civic hackathons “work,” but I believe my multifaceted experience with these events has given me insight into other factors that can be used to evaluate their success.

I think civic hackathons can be bigger than the apps the generate. With some forethought and planning, these events can generate benefits that resonate well beyond the end of the award ceremony.

I think it’s a mistake to judge the success of a hackathon solely on how long the apps it produces “live” afterwards.

It’s also a mistake to try and improve hackathons by focusing exclusively on strategies for sustaining apps in the long term. This misses some of the most important benefits that can be generated by these events.

Whether we’re judging past success of civic hackathons or trying to improve future performance, it’s time to get beyond the apps.

You Get What You Plan For

I’m by no means suggesting that striving for long-term adoption of apps generated at civic hackathons is a trivial or unimportant thing. Far from it.

I’m currently working with a group in Philadelphia that developed an app as part of a recent Random Hacks of Kindness event, to identify funding and supporters to help support operation of the app long-term.

My contention here is that this is but one of the benefits to come from this civic hacking event generally, and from this software application specifically.

Not only did the efforts of my team result in an app – they resulted in a previously unavailable data set being published for others to use.  The app my team worked on helps people in Philadelphia locate farmer’s markets and food retailers that accept Supplemental Nutrition Assistance Program (SNAP) reimbursement through text messaging. The data behind this app is now available for anyone that wants it, either through an API that supports geo-spatial queries or as a downloadable file in a commonly used format.

The data our app needed to operate was “liberated” in the process of building the app. It is now available for anyone else to use, tweak, modify or expand.

That was our plan, and whether we are able to secure longer-term support for our app, and receive assistance in promoting it, this liberated data will live on.

I’m not the only person that has made this argument. Clay Johnson – formerly of Sunlight Labs – has emphasized repeatedly the need to build a community around app contests. This is another positive outcome that can have long term benefits that is not directly related to how many apps are actively being used six months after a civic hacking event.

I noted with some excitement the number of elected officials and political candidates that attended the recent Summer of Smart hackathons in San Francisco. This is a great way to expose public sector employees and officials to the power of civic hacking.

It’s an approach I am using in the upcoming Apps for SEPTA coding event I’m helping organize in Philadelphia, where officials from the Mayor’s office (who’ve never been to a hackathon before) will be in attendance.

I’ve argued in the past that one of the key benefits of civic hackathons is that they stretch traditional notions of public service delivery and show governments what is possible to do with their data. I can’t think of a more effective way to do this than through a civic hacking event.

There is also the very real potential for these events to generate reusable components – open source software that can be used by other developers or governments to build civic applications down the road.

Nick Judd of TechPresident said this much more eloquently than I:

“With each hackathon, some of the detritus — bits of code, training videos, documentation, the right people trading email addresses — becomes scaffolding for the attendees of later ones.”

The benefits that are achievable through civic hackathons go far beyond just the collection of apps that get developed in the course of a weekend.

But the impetus is on organizers and supporters of such events to plan for these benefits, and to nurture them after the event is concluded. You get what you plan for, and if event organizers don’t plan past the end of the weekend then the potential for a missed opportunity is real.

Civic hackathons are bigger than the apps they generate – they always have been.

Many, though, are now just realizing how far the benefits of these weekends of caffeine-fueled hacking extend.

A ‘glass half full’ view of government app contests

An increasing number of people are starting to suggest that the concept of the “app contest” (where governments challenge developers to build civic applications) is getting a bit long in the tooth.

There have been lots of musings lately about the payoff for governments that hold such contests and the long term viability of individual entries developed for these contests. Even Washington DC – the birthplace of the current government app contest craze – seems the be moving beyond the framework it has employed not once, but twice to engage local developers:

“I don’t think we’re going to be running any more Apps for Democracy competitions quite in that way,” says Bryan Sivak, who became the district’s chief technology officer in 2009. Sivak calls Apps for Democracy a “great idea” for getting citizen software developers involved with government, but he also hints that the applications spun up by these contests tend to be more “cool” than useful to the average city resident.

App contests abound

This view is starting to crystallize against the backdrop of an ever greater number of app contests being held. At the recent Gov 2.0 Expo in Washington DC, Peter Corbett of iStrategy Labs (who helped launch the first government app contest in DC) gave a presentation that listed several dozen governments around the globe that had recently completed an app contest or were scheduled to soon start one.

And the biggest app contest to date – being sponsored by the State of California – is slated to begin soon. (Two fringe technology companies that you’ve probably never heard of – Google and Microsoft – are set to partner with the Golden State for this 800 pound gorilla of government app contests.)

So if app contests are being used in more and more places, and the size and scope of these contests keeps growing, what’s with all the hand wringing of late?

Lessons learned from app contests

My take on app contests is not an unbiased one. I’ve been a competitor in three different app contests (the original Apps for Democracy, the original Apps for America, and the NYC Big Apps competition) and was recognized for my work in them. Outside of contests, I’ve build applications using open government data and APIs for the cities of Toronto and San Francisco, and for the New York State Senate.

Clearly I am a supporter of the concept of the government app contest.

Having said that, though, I do think that those taking a more skeptical view of app contests are asking some important questions. The government app contest has come a long way since Vivek Kundra was in the driver’s seat in the DC technology office. It’s time to start asking how app contests can be improved.

But before we move on to that discussion, it is worth noting the lessons that have been learned over the last two years or so from government app contests.

First, governments and citizens benefit when high value, high quality data sets are released by governments that are in machine readable formats, easily consumed by third party applications. Believe it or not, there is still debate in many places on this point. App contests prove the theory that publishing open government data provides tangible benefits.

Second, app contests prove that it is possible to engage and excite both developers and high level elected officials about open government data. The cause of open government can’t be anything but well served when these two groups are excited about it, and appealing to both successfully in equal measure is usually very challenging.

Third, and maybe most importantly, government app contests provide sort of a “petri dish” for government officials to see how government data might be used. They let governments solicit ideas from the private sector about the different ways that open data can be used in a manner that is low risk and low cost. Some of the proposed uses of government data that emerge from these contests – whether its tweeting a recorded message to your Congressman, or using an IM client to browse campaign finance data – might never be considered by governments but for them running an app contest.

These lessons aside, there are those who contend that the existence of app contest entries that have languished (or even been abandoned altogether) after a contest is over suggests that an app contest didn’t work well (or as well as it should have). I don’t think this is necessarily the case.

Look at it this way; once a government has decided to publish open data sets and enable the development of one single app by an outside developer, the marginal cost of the next app (from the perspective of government) is essentially zero.

Once a data set has been put into a machine readable format and staged for download so that it can be used by a developer or third party, what is the cost of the next download? Or the next 50, or 100? Essentially nothing.

The road to tech startup profitability and success is a long and hard one, and it’s littered with the hollowed out husks of ideas (some very bad, some very good) that for one reason or another just don’t make it.

Should we be overly concerned that the dynamic of government app contest entries is essentially the same as it is for any other sort of technology startup project? Personally, I don’t think so.

Making app contests better

I do however, think there are some things that government app contests organizers can do a better job on.

Most notably, government engagement with app developers over the long-term has proved to be somewhat challenging. Gunnar Hellekson of Red Hat has observed the same phenomenon:

“..I would think that one of the desired outcomes [of an app contest] was an ongoing community of developers that are producing and maintaining applications like this — whether it’s for love, money, or fame. It would be a shame to see hard work like this die on the vine because we’ve lost the carrot of a cash prize.”

I don’t think this is an issue with developers necessarily – I know there is still lots of excitement around the data sets that have served as the foundation for app contents that are now over. I think the issue is that governments do not always have a plan for post-contest developer engagement.

Once the prizes are given out, and the award ceremony is over, there are no plans or strategies in place to keep developers engaged over the long haul. I do not believe this is an issue of money – not every developer is looking for a cash prize, and there are some good examples of government agencies (MassDOT and BART among them) who do a pretty good job of keeping developers engaged without contests.

I also think that a greater emphasis could be placed in app contests on developing reusable components (as opposed to user-facing solutions) that can be released as open source software and used by anyone to consume data or interact with a government API. I’m talking specifically about things like open source libraries for interacting with the Open311 API – tools and libraries specifically designed to make it easier to use open government data.

The easier it is to use government data and APIs the more people will do it, and the more development of reusable components as a by product of app contest, the less angst there will be about projects that don’t remain viable long-term. If one of the requirements of entry is the use (or reuse) of common components, even contest entries that fizzle out down the road will have made a tangible contribution to the open data effort.

I think with a few simple changes, app contests can continue to be used as an effective tool by governments to encourage the development of cutting edge applications powered by “democratized” government data.

Open source matters to open government. Really.

“Open source and open government are not the same,” I’ve been reading recently. When discussing the role of open standards in open government transparency projects, Bob Caudill at Adobe, is concerned that open source and open standards are being conflated. He likes open standards just fine, but:

“Open standards are driving for interoperability between systems or applications, while, the goal of open source is to make high-quality software available to the market free of charge.”

As an open source advocate, I’m surprised. What, I have to wonder, is so threatening about open source? Why the effort to take open source off the table? I’ve written on the topic before, and I didn’t think this was controversial — but apparently I was wrong. Andrea DiMaio at Gartner is more pointed:

“For those who have been following some of the vintage discussions about government and open source, this will probably sound like a déjà vu. I honestly thought that people had finally given up pushing the confusion between open source and open standards or open formats, but here we are again.”

While they both agree on the importance of open standards (although transparency also seems to annoy DiMaio), they also remind us that tools, proprietary or open source, are a means to an end. An open standard is an open standard, whether implemented by an open source project or a proprietary one. What’s important, they insist, is more transparency, collaboration, and participation. Open source is immaterial at best, and a distraction at worst.

They’re right, of course, that open standards are crucial to ensuring meaningful transparency in government. It does not follow, however, that this precludes a role for open source.  Open source software is an invaluable tool — one of many — to approach all three goals (transparency, collaboration, participation) of the Open Government Directive. It’s not about open source software specifically, although the software helps. It’s about the process that open source projects use to create good software. Because the open source development process requires real collaboration, tangible progress towards a goal, and the participation of a broad community of users and developers, it’s an excellent mechanism for getting citizens involved in the work of government.

DiMaio couldn’t disagree more. Referring to Nat Torkington’s idea of using the open source development model to improve transparency projects:

“…there is a fundamental flaw in this line of thought. Open source projects cluster a number of developers who collaborate on an equal footing to develop a product they are jointly responsible for, as a community.

“Government does not have the luxury of doing so. An agency publishing crime statistics or weather forecast or traffic information is ultimately accountable for what it publishes.”

I couldn’t disagree more. Again, DiMaio and Caudill misunderstand how the open source process works and what it can contribute. The trouble, I think, is with a too-narrow understanding of what participation and collaboration might mean, and a similarly narrow view of what the open source development process has to offer.

The goal of open source is much more than just making no-cost software, as Caudill suggests. It’s about producing better software through a process of inclusion and rough consensus. The source code is free of charge largely because that is the best way to create a large community around the project, it’s not the final goal. And while some open source projects function better than others, they are not, as a rule, unaccountable. In order for the projects to succeed, they must be highly accountable to their community.  Further, many open source projects have commercial ventures (like my company, Red Hat) that live or die by their success, which makes them extremely accountable. So to say that the government cannot rely on open source software or the open source process because it is unaccountable is just not true. We know this to be the case because you can find the government using open source software in the Army, the NSA, the Census, the White House, and just about everywhere else. So there’s no reason to think that open source process cannot inform and support an open data project, as DiMaio suggests.

Setting accountability to the side, the more interesting conversation is how open source can bring some unique benefits to open government, unavailable any other way.

If you look at the outstanding work of pro-transparency organizations like the Sunlight Foundation, govtrack.us, RECAP, and others, nearly all are using open source and the open source development model. It’s not, as DiMaio and Caudill suggest, because they’re naive ideologues who are confused as to the meaning of “open”. These are smart people doing serious work. They’re using open source because it’s the best way to collect a large number of contributors around a common problem. They’re using open source because the transparency of the process and software makes their work credible. They’re using open source because they believe that free access to government data means free access to the tools that make that data useful.

The alternative is closed, proprietary tools, which do little to further the transparency goals. RECAP, for example, had a difficult time understanding the US Courts’ closed PACER system, and had to do a lot of difficult reverse-engineering to work with it effectively. The job would have been significantly easier if they had access to the PACER software source code. Fortunately, because RECAP is an open source project, their hard work making PACER usable is now available to everyone. So to dismiss open source as irrelevant to the crucial work of making government data available and valuable to the private citizen, and the even more important work of encouraging a collaboration between government and its citizen, is deeply misguided.

Again, even though data transparency seems to annoy DiMaio, I think there’s good reason for the tremendous transparency effort the administration and the private sector have brought to bear. First, data transparency is a relatively simple problem to solve. It’s easy to publish data on the Internet, and there’s a tremendous amount of value to be extracted. So while it’s only a part of the challenge — indeed, is only one leg of the Open Government Directive — it’s an easy win for both government and its citizens.

But DiMaio is correct that open government is about much more than just data, so let’s generalize this further. We could understand open government as an opportunity to increase the quality of interaction between citizens and their government through collaboration. “The government is not a vending machine,” as Tim O’Reilly paraphased Frank DiGiammarino of the National Academy of Public Administration, “which we kick when it doesn’t work right.” Instead of treating government as a black box, we should treat our government as the place where we, in the public and private sector, come together, to solve problems as a group. This is why we refer to “government as a platform.” Yes, as DiMaio says, each agency is responsible for its own output. But that doesn’t mean the public has no stake. Precisely because we want to hold agencies to a higher standard, we must provide a means of collaboration and participation.

The trouble is, there’s a lot more of us than there is of them. How can one agency effectively collaborate with 300 million constituents? Likewise, how can an agency effectively communicate with that many people? One of the reasons the open government movement is so preoccupied with technology and the Internet is that they represent a solution to this problem. For the first time, the government and its citizens have the means to work effectively at this scale. There are all kinds of tools for this: social networking, blogging, data.gov, the Ideascale Open Government sites, and so on. One of those tools, the one that is most interesting to me, is the open source development process.

Note that I didn’t say open source software. Although I love the software, and could talk for days about why the government should be using more of it, it’s the process that creates this software that is most valuable to the goals of collaboration and participation.

In the last 40 years, open source software communities have learned how to effectively solve complex tasks with large, far-flung, geographically dispersed communities. Why wouldn’t we take these methods, and apply them to the task of creating a better government? As I mentioned earlier, Nat Torkington suggested using the open source process to improve data quality. The NASA CoLab project uses open source software and the open source development process alongside other collaborative tools to get researchers from the public and private sector to work together. The Defense Information Systems Agency is using the forge.mil project to encourage collaboration between the DOD and its contractors — not just for software, but for testing, certification, and project management. The Apps for Democracy, Apps for Army, and Apps for America contests are all attempts to harness the collective intelligence of citizens and government to solve common problems using the open source model — not just building tools, but building the means to collaborate on top of open tools, like Open 311 and DataMasher.

So when DiMaio bemoans the lack of government employee engagement and the lack of community data, it may be because he doesn’t realize that this work is happening, and it’s happening using open source and (more generally) collaborative innovation models.

Both DiMaio and Caudill make the mistake of believing that open source is about making cheap bits. Instead, it’s a blueprint for effective collaboration on a massive scale. Advocates for open source in government, like me and my friends at Open Source for America, aren’t just talking about open source tools, although those are also useful. We believe that the open source development model has a concrete contribution to make to the open government movement — and those who dismiss open source as irrelevant don’t realize just how open a government can be.

Developers for Glory

Although it may be simple to conflate the Apps for Democracy and Apps for America contests with the exciting new Apps for Army contest, they really couldn’t be more different. Together they represent an exciting experiment in what it takes to pull communities together around a problem. Though they all offer cash prizes to the winners, they each took a slightly different approach, with different results.

Cash incentives are somewhat controversial in open source circles. Most old-school advocates for open source development strongly prefer developers who are personally invested — famously, those that “scratch their own itch.” Developers who are paid a salary to work on software are also invested, but perhaps less zealously than those who are solving a problem they are afflicted with themselves. Developers who are working for glory and cash prizes, the model used by the “Apps for…”  competitions, is yet another class of developer, and despite the excellent submissions to the previous contests, there are valid concerns that the quality and sustainability of the code is not as good as it could be with a different set of incentives. Time will tell, of course.

If I’m a developer for glory, I may compete for the cash prize, or for altruistic reasons, but I’m also competing for the notoriety I’ll get if I win. If I don’t win, what will I do with the code I’ve developed? Even if I win, what are my incentives to continue working on the project? Put another way: how can we ensure that all of this good work and goodwill turns into viable, and active software projects once the contest is over?

Apps for Democracy is instructive.  The contest encouraged developers to provide services on top of the “platform” of Washington, D.C.’s IT infrastructure. This platform includes 270 public data feeds and the city’s newly unveiled 311 API.  47 submissions were collected in 30 days, and the winner was an iPhone and Facebook application that enabled users to take snapshots of potholes, broken windows, and so forth, have them tagged with GPS coordinates, and submitted to the city’s 311 service. Very handy. Unfortunately, the ongoing care and feeding for the application doesn’t seem to be there. The Washington City Paper found in a January 25th, 2010 followup on the contest:

The “Touch City’s Heart” Social DC 311 Web site seems to have been abandoned—it hasn’t been updated for months—saying the “IPhone” app is still waiting for approval from Apple (Apple approved it long ago). Some members of the D.C. 311 team had never laid eyes on the Web site until City Desk asked about it. “I’ve never even heard of it,” said one 311 operator. It has only 27 active monthly users on its Facebook Fan page and 40 followers on Twitter.

I’ll also note that after some cursory research, the source code doesn’t seem to be disclosed to the public yet, which I understand was one of the intents of the contest. Now, to be fair, there seem to be bigger plans afoot:

The dismal following is not a sign of failure, Sivak says. The District intends to take Social DC 311 and revamp the current model into an app that’s “enterprise-ready and robust for a large volume of users,” Sivak says. “Think of this first step as a pilot.”

Fair enough, but I would think that one of the desired outcomes was an ongoing community of developers that are producing and maintaining applications like this — whether it’s for love, money, or fame. It would be a shame to see hard work like this die on the vine because we’ve lost the carrot of a cash prize.

The first Apps for America contest winner was Filibusted, a tool for outing Senate obstructionists. It measures obstruction by the Senator’s votes on cloture motions. You can find the source on GitHub, but there doesn’t seem to be much activity since the initial checkin. One bug was opened 8 months ago, and doesn’t appear to have been addressed. The last blog post was in December. At the same time, there’s not much to work on — the site has a single purpose, which it seems to fulfil even without much of a community around it. It doesn’t really need a large community, I’d guess, because it’s “done.”

The second Apps for America yielded DataMasher. This tools allows you to compare Federal data sets with each other. Once you have the data and visualization you like, you can share it with others on the site. The source code was released, per the terms of the contest, but doesn’t seem to have much of a community around it. In fact, the DataMasher website doesn’t seem to link to the code from their own site. That hasn’t made the application less popular, though — the community isn’t working on the code, it’s working on the datasets. There’s a steady stream of new mashups that other users rate and comment on. In all, a healthy community that relies on user-generated content to ensure it remains a useful tool.

The second Apps for America contest also produced the strikingly elegant govpulse.us. It’s a vastly improved interfact to the Federal Register developed by the gifted team at GravyCones. The code for this application is available to the public, and seems actively developed to this day. This is, I think, exactly what the organizers had in mind when they started this contest: the tool is popular, the development community is active, and the project continues to improve.

Which brings us to Apps for Army, which is a serious departure from the other contests. First, it’s available only to Army soldiers and civilian employees, nobody from the public — not even reservists. In fact, you need a DoD ID card to go to the official contest website. Second, it seems that only the first 100 teams can participate. From a community standpoint, the project is wading into very unfamiliar territory. Rather than gathering the collective wisdom an initiative of thousands of interested developers, they’ll be picking 100 volunteers, seemingly at random.

The Apps for Army contest further diminishes its potential reach by dictating the tools developers will use: the DISA RACE environment to host the project, and the forge.mil repository for code. Since these resources are being paid for by the Army’s CIO, who is sponsoring the contest, what will happen to the competitors once the competition’s over? There are, of course, excellent reasons for asking folks to use the existing DoD infrastructure, but I can’t help but wonder what would happen if the doors were flung open, and the bar was lowered for participation.

This isn’t to say that I’m less enthusiastic about these experiments. I’m very excited at the idea of encouraging employees — in the Army, or anywhere else — to solve their own problems. That’s a goodness in and of itself. We just can’t forget that software isn’t a product — it’s a process that requires nurturing. The best way to nurture is to build a community, and that requires transparency and a low barrier to entry for participants. The larger and more active the community, the more likely the software will be better. The more closed, prescriptive, and limited the project, I think, the less likely that it will be viable in the long-term.

So these “Apps for…” competitions are instructive. Each project is building its own kind of community, and I’m eager to see how these projects fare in the months and years ahead.