Apps for Army

Open source matters to open government. Really.

“Open source and open government are not the same,” I’ve been reading recently. When discussing the role of open standards in open government transparency projects, Bob Caudill at Adobe, is concerned that open source and open standards are being conflated. He likes open standards just fine, but:

“Open standards are driving for interoperability between systems or applications, while, the goal of open source is to make high-quality software available to the market free of charge.”

As an open source advocate, I’m surprised. What, I have to wonder, is so threatening about open source? Why the effort to take open source off the table? I’ve written on the topic before, and I didn’t think this was controversial — but apparently I was wrong. Andrea DiMaio at Gartner is more pointed:

“For those who have been following some of the vintage discussions about government and open source, this will probably sound like a déjà vu. I honestly thought that people had finally given up pushing the confusion between open source and open standards or open formats, but here we are again.”

While they both agree on the importance of open standards (although transparency also seems to annoy DiMaio), they also remind us that tools, proprietary or open source, are a means to an end. An open standard is an open standard, whether implemented by an open source project or a proprietary one. What’s important, they insist, is more transparency, collaboration, and participation. Open source is immaterial at best, and a distraction at worst.

They’re right, of course, that open standards are crucial to ensuring meaningful transparency in government. It does not follow, however, that this precludes a role for open source.  Open source software is an invaluable tool — one of many — to approach all three goals (transparency, collaboration, participation) of the Open Government Directive. It’s not about open source software specifically, although the software helps. It’s about the process that open source projects use to create good software. Because the open source development process requires real collaboration, tangible progress towards a goal, and the participation of a broad community of users and developers, it’s an excellent mechanism for getting citizens involved in the work of government.

DiMaio couldn’t disagree more. Referring to Nat Torkington’s idea of using the open source development model to improve transparency projects:

“…there is a fundamental flaw in this line of thought. Open source projects cluster a number of developers who collaborate on an equal footing to develop a product they are jointly responsible for, as a community.

“Government does not have the luxury of doing so. An agency publishing crime statistics or weather forecast or traffic information is ultimately accountable for what it publishes.”

I couldn’t disagree more. Again, DiMaio and Caudill misunderstand how the open source process works and what it can contribute. The trouble, I think, is with a too-narrow understanding of what participation and collaboration might mean, and a similarly narrow view of what the open source development process has to offer.

The goal of open source is much more than just making no-cost software, as Caudill suggests. It’s about producing better software through a process of inclusion and rough consensus. The source code is free of charge largely because that is the best way to create a large community around the project, it’s not the final goal. And while some open source projects function better than others, they are not, as a rule, unaccountable. In order for the projects to succeed, they must be highly accountable to their community.  Further, many open source projects have commercial ventures (like my company, Red Hat) that live or die by their success, which makes them extremely accountable. So to say that the government cannot rely on open source software or the open source process because it is unaccountable is just not true. We know this to be the case because you can find the government using open source software in the Army, the NSA, the Census, the White House, and just about everywhere else. So there’s no reason to think that open source process cannot inform and support an open data project, as DiMaio suggests.

Setting accountability to the side, the more interesting conversation is how open source can bring some unique benefits to open government, unavailable any other way.

If you look at the outstanding work of pro-transparency organizations like the Sunlight Foundation, govtrack.us, RECAP, and others, nearly all are using open source and the open source development model. It’s not, as DiMaio and Caudill suggest, because they’re naive ideologues who are confused as to the meaning of “open”. These are smart people doing serious work. They’re using open source because it’s the best way to collect a large number of contributors around a common problem. They’re using open source because the transparency of the process and software makes their work credible. They’re using open source because they believe that free access to government data means free access to the tools that make that data useful.

The alternative is closed, proprietary tools, which do little to further the transparency goals. RECAP, for example, had a difficult time understanding the US Courts’ closed PACER system, and had to do a lot of difficult reverse-engineering to work with it effectively. The job would have been significantly easier if they had access to the PACER software source code. Fortunately, because RECAP is an open source project, their hard work making PACER usable is now available to everyone. So to dismiss open source as irrelevant to the crucial work of making government data available and valuable to the private citizen, and the even more important work of encouraging a collaboration between government and its citizen, is deeply misguided.

Again, even though data transparency seems to annoy DiMaio, I think there’s good reason for the tremendous transparency effort the administration and the private sector have brought to bear. First, data transparency is a relatively simple problem to solve. It’s easy to publish data on the Internet, and there’s a tremendous amount of value to be extracted. So while it’s only a part of the challenge — indeed, is only one leg of the Open Government Directive — it’s an easy win for both government and its citizens.

But DiMaio is correct that open government is about much more than just data, so let’s generalize this further. We could understand open government as an opportunity to increase the quality of interaction between citizens and their government through collaboration. “The government is not a vending machine,” as Tim O’Reilly paraphased Frank DiGiammarino of the National Academy of Public Administration, “which we kick when it doesn’t work right.” Instead of treating government as a black box, we should treat our government as the place where we, in the public and private sector, come together, to solve problems as a group. This is why we refer to “government as a platform.” Yes, as DiMaio says, each agency is responsible for its own output. But that doesn’t mean the public has no stake. Precisely because we want to hold agencies to a higher standard, we must provide a means of collaboration and participation.

The trouble is, there’s a lot more of us than there is of them. How can one agency effectively collaborate with 300 million constituents? Likewise, how can an agency effectively communicate with that many people? One of the reasons the open government movement is so preoccupied with technology and the Internet is that they represent a solution to this problem. For the first time, the government and its citizens have the means to work effectively at this scale. There are all kinds of tools for this: social networking, blogging, data.gov, the Ideascale Open Government sites, and so on. One of those tools, the one that is most interesting to me, is the open source development process.

Note that I didn’t say open source software. Although I love the software, and could talk for days about why the government should be using more of it, it’s the process that creates this software that is most valuable to the goals of collaboration and participation.

In the last 40 years, open source software communities have learned how to effectively solve complex tasks with large, far-flung, geographically dispersed communities. Why wouldn’t we take these methods, and apply them to the task of creating a better government? As I mentioned earlier, Nat Torkington suggested using the open source process to improve data quality. The NASA CoLab project uses open source software and the open source development process alongside other collaborative tools to get researchers from the public and private sector to work together. The Defense Information Systems Agency is using the forge.mil project to encourage collaboration between the DOD and its contractors — not just for software, but for testing, certification, and project management. The Apps for Democracy, Apps for Army, and Apps for America contests are all attempts to harness the collective intelligence of citizens and government to solve common problems using the open source model — not just building tools, but building the means to collaborate on top of open tools, like Open 311 and DataMasher.

So when DiMaio bemoans the lack of government employee engagement and the lack of community data, it may be because he doesn’t realize that this work is happening, and it’s happening using open source and (more generally) collaborative innovation models.

Both DiMaio and Caudill make the mistake of believing that open source is about making cheap bits. Instead, it’s a blueprint for effective collaboration on a massive scale. Advocates for open source in government, like me and my friends at Open Source for America, aren’t just talking about open source tools, although those are also useful. We believe that the open source development model has a concrete contribution to make to the open government movement — and those who dismiss open source as irrelevant don’t realize just how open a government can be.

Developers for Glory

Although it may be simple to conflate the Apps for Democracy and Apps for America contests with the exciting new Apps for Army contest, they really couldn’t be more different. Together they represent an exciting experiment in what it takes to pull communities together around a problem. Though they all offer cash prizes to the winners, they each took a slightly different approach, with different results.

Cash incentives are somewhat controversial in open source circles. Most old-school advocates for open source development strongly prefer developers who are personally invested — famously, those that “scratch their own itch.” Developers who are paid a salary to work on software are also invested, but perhaps less zealously than those who are solving a problem they are afflicted with themselves. Developers who are working for glory and cash prizes, the model used by the “Apps for…”  competitions, is yet another class of developer, and despite the excellent submissions to the previous contests, there are valid concerns that the quality and sustainability of the code is not as good as it could be with a different set of incentives. Time will tell, of course.

If I’m a developer for glory, I may compete for the cash prize, or for altruistic reasons, but I’m also competing for the notoriety I’ll get if I win. If I don’t win, what will I do with the code I’ve developed? Even if I win, what are my incentives to continue working on the project? Put another way: how can we ensure that all of this good work and goodwill turns into viable, and active software projects once the contest is over?

Apps for Democracy is instructive.  The contest encouraged developers to provide services on top of the “platform” of Washington, D.C.’s IT infrastructure. This platform includes 270 public data feeds and the city’s newly unveiled 311 API.  47 submissions were collected in 30 days, and the winner was an iPhone and Facebook application that enabled users to take snapshots of potholes, broken windows, and so forth, have them tagged with GPS coordinates, and submitted to the city’s 311 service. Very handy. Unfortunately, the ongoing care and feeding for the application doesn’t seem to be there. The Washington City Paper found in a January 25th, 2010 followup on the contest:

The “Touch City’s Heart” Social DC 311 Web site seems to have been abandoned—it hasn’t been updated for months—saying the “IPhone” app is still waiting for approval from Apple (Apple approved it long ago). Some members of the D.C. 311 team had never laid eyes on the Web site until City Desk asked about it. “I’ve never even heard of it,” said one 311 operator. It has only 27 active monthly users on its Facebook Fan page and 40 followers on Twitter.

I’ll also note that after some cursory research, the source code doesn’t seem to be disclosed to the public yet, which I understand was one of the intents of the contest. Now, to be fair, there seem to be bigger plans afoot:

The dismal following is not a sign of failure, Sivak says. The District intends to take Social DC 311 and revamp the current model into an app that’s “enterprise-ready and robust for a large volume of users,” Sivak says. “Think of this first step as a pilot.”

Fair enough, but I would think that one of the desired outcomes was an ongoing community of developers that are producing and maintaining applications like this — whether it’s for love, money, or fame. It would be a shame to see hard work like this die on the vine because we’ve lost the carrot of a cash prize.

The first Apps for America contest winner was Filibusted, a tool for outing Senate obstructionists. It measures obstruction by the Senator’s votes on cloture motions. You can find the source on GitHub, but there doesn’t seem to be much activity since the initial checkin. One bug was opened 8 months ago, and doesn’t appear to have been addressed. The last blog post was in December. At the same time, there’s not much to work on — the site has a single purpose, which it seems to fulfil even without much of a community around it. It doesn’t really need a large community, I’d guess, because it’s “done.”

The second Apps for America yielded DataMasher. This tools allows you to compare Federal data sets with each other. Once you have the data and visualization you like, you can share it with others on the site. The source code was released, per the terms of the contest, but doesn’t seem to have much of a community around it. In fact, the DataMasher website doesn’t seem to link to the code from their own site. That hasn’t made the application less popular, though — the community isn’t working on the code, it’s working on the datasets. There’s a steady stream of new mashups that other users rate and comment on. In all, a healthy community that relies on user-generated content to ensure it remains a useful tool.

The second Apps for America contest also produced the strikingly elegant govpulse.us. It’s a vastly improved interfact to the Federal Register developed by the gifted team at GravyCones. The code for this application is available to the public, and seems actively developed to this day. This is, I think, exactly what the organizers had in mind when they started this contest: the tool is popular, the development community is active, and the project continues to improve.

Which brings us to Apps for Army, which is a serious departure from the other contests. First, it’s available only to Army soldiers and civilian employees, nobody from the public — not even reservists. In fact, you need a DoD ID card to go to the official contest website. Second, it seems that only the first 100 teams can participate. From a community standpoint, the project is wading into very unfamiliar territory. Rather than gathering the collective wisdom an initiative of thousands of interested developers, they’ll be picking 100 volunteers, seemingly at random.

The Apps for Army contest further diminishes its potential reach by dictating the tools developers will use: the DISA RACE environment to host the project, and the forge.mil repository for code. Since these resources are being paid for by the Army’s CIO, who is sponsoring the contest, what will happen to the competitors once the competition’s over? There are, of course, excellent reasons for asking folks to use the existing DoD infrastructure, but I can’t help but wonder what would happen if the doors were flung open, and the bar was lowered for participation.

This isn’t to say that I’m less enthusiastic about these experiments. I’m very excited at the idea of encouraging employees — in the Army, or anywhere else — to solve their own problems. That’s a goodness in and of itself. We just can’t forget that software isn’t a product — it’s a process that requires nurturing. The best way to nurture is to build a community, and that requires transparency and a low barrier to entry for participants. The larger and more active the community, the more likely the software will be better. The more closed, prescriptive, and limited the project, I think, the less likely that it will be viable in the long-term.

So these “Apps for…” competitions are instructive. Each project is building its own kind of community, and I’m eager to see how these projects fare in the months and years ahead.