The GSA is currently planning forge.gov, which is widely assumed to be based on forge.mil, the much-discussed collaboration platform from the Defense Information Systems Agency, or DISA. forge.mil is a pretty incredible idea: a single destination for testing, certification, and software development in the Defense Department.
It sounds obvious, but the idea remains revolutionary. For the first time, there would be a single repository for source code that could be shared between the hundreds of agencies, commands, and programs in DOD. Developers would be able to share their work in a familiar, web-based environment. A previous version of forge.mil was pulled for unknown reasons, but the current iteration is based on the TeamForge product from CollabNet. If you’ve used SourceForge, you get the idea. The DOD is the largest consumer, and one of the largest developers of software in the world. Much of this software is redundant, locked up by vendors and integrators, can’t work with other software, and nobody remembers how to maintain it. There’s no doubt forge.mil was long overdue.
It’s dangerous, though, to assume that forge.mil is a useful template for forge.gov. I think forge.mil could lead forge.gov down the same road as core.gov and other failed attempts to encourage source code reuse in government. To understand why forge.mil can be useful and simultaneously poisonous to forge.gov, you have to first understand how the DOD does software.
COTS vs. GOTS
Before, say, the mid-1990s, much of the DOD’s software was owned by the government. GOTS, or “Government Off-the-Shelf” as it’s now called, was built and maintained by the DOD and its contractors. This was appropriate for some military-specific systems, but the strategy outlived its usefulness when the government could no longer keep up with commercial enterprises. For many pieces of common software, like operating systems, spreadsheets and web browsers, the open market produced more innovative and higher-quality products. So down came the order: use commercial software. COTS (“Commercial Off-the-Shelf”) was ascendant.
Boeing, Lockheed Martin, General Dynamics, and the other integrators no longer write as much software from scratch. Instead, their business model has changed. They now glue commercial software together, and wrote code to fill in the gaps. Their business model, however, remains the same. Instead of seeking rent on the proprietary software they’ve written on behalf of the government, they seek rent on the integration work they’ve done.
This is a lucrative business: the latest UAV is composed of hundreds of software systems, some commercially available, some written by the integrator. If I’m the integrator, I’m the only one who knows how all the pieces go together and I can charge a handsome sum to anyone who’d like to field their technology on my platform. Think of Apple’s locked-up App Store, but flying through the air with a missile strapped to its belly.
There’s nothing nefarious about this, of course. Integrators are doing what the market commands, and controlling access to platforms is an perfectly legitimate business model. It discourages reuse, though, which means that it’s difficult for the DOD to effectively use the software it purchased or developed. You can read more about this strange market and its consequences in the excellent “Losing the Softwar(e)” by my friend and fellow Open Source for America member, John Scott.
Openness as a Desperate Act
So Rob Vietmeyer of DISA decided to borrow from the principles of the open source community and Internet governance. A more open and transparent development process at DISA could remove barriers to reuse, encourage collaboration, and discourage proprietary or closed systems. COTS software is still king, but where the government needs to control its own integration, set its own standards, and exercise stewardship over its own infrastructure, it can still develop its own GOTS solutions — this time, in an open, collaborative manner. Where GOTS was once insular, slow-moving, and highly proprietary, it can now be produced at lower cost and with lower barriers to entry for new innovations. Because this “Open GOTS” is built using familiar open source methods, the projects have a fighting chance of working together.
The Walled Garden and the Moral Hazard
So far, the DOD software problem sounds similar to the government software problem. Don’t misunderstand forge.mil, though. It’s not your typical open source development environment. If you talk with the forge.mil project team, like Guy Martin of CollabNet, he’s quick to correct you if you compare forge.mil to SourceForge. forge.mil is something very different. With all the challenges of this profoundly broken market, forge.mil had to make some serious sacrifices.
The first thing you’ll notice about forge.mil is that you can’t get to it. Access to forge.mil is severely restricted. To get access, you must have an official DOD Common Access Card (CAC) or have a DOD employee sponsor you for an “ECA Certificate”. Dave Wheeler of the Institute for Defense Analysis describes it as “gated development.” On the mil-oss mailing list, this is referred to as the “CAC Wall”.
The CAC Wall has some unintended consequences, and raises some very difficult questions. First, it prevents many well-meaning developers who don’t work for the Defense Industrial Base from helping the projects inside. There are literally tens of thousands of developers behind that wall, and forge.mil keeps that group partitioned from both the public and government employees without CAC or ECA credentials. The bargain is that by keeping these undesirables out, they carve out a sense of safety so skittish program managers are more likely to host their code. The cost of that sense of safety is a much smaller audience than they could muster hosting on more public platforms.
The CAC Wall also creates a dangerous an incentive to split communities. What happens if someone in the DOD wants to hack on a piece of open source code? They’ll host the hack at forge.mil, and the public could never see those hacks again. If I’ve licensed my project under an open source license, it’s because I want others to contribute. If that code disappears behind the CAC Wall, I’m cut off from tens of thousands of DOD developers. This “forge.mil fork” scenario is serious business. It’s a scenario where everyone loses.
Finally, the CAC Wall may create a moral hazard for the developers who live behind it. In the open source community, folks are very careful about what code they commit, and since they’re never really sure who’s a friend and who’s a foe. In a community where everyone’s “trusted,” developers can become complacent, making them more vulnerable to poorly written or hostile code.
Despite these real concerns, the CAC Wall still makes sense for forge.mil. DISA has some very legitimate security concerns about the code that’s being developed. They’ve weighed that risk against the advantages of public scrutiny, cooperation with external projects, and they’ve made the informed decision to keep the code behind the “CAC wall”, and to their credit they encourage forge.mil developers to contribute patches upstream wherever possible instead of hosting on forge.mil.
forge.mil as Role Model
So we have a kind of Judgement of Solomon in the forge.mil platform. Because of its very unique market dynamics, the DOD needs to take advantage of open source projects, the open source development model, encourage collaboration, and reduce its reliance on proprietary platforms. But for that to happen, DISA had to put all the work behind the CAC wall.
forge.mil has attracted the attention of other agencies. How could it not? The DOD has the worst-case scenario: the gravest problem, the most complex market conditions, and the most dire consequences. So other CIOs take notice and the press on this forge.mil experiment has been relentless.
This brings brings us to the news that the GSA’s Dave McClure is planning forge.gov, a civilian counterpart to forge.mil. This is exciting. This is also terrifying, because they seem to be following the forge.mil model by restricting access to only US citizens. Guy Martin says:
I realize that putting up a barrier to entry in the form of positive identification of US citizenship and a vetting process will irk some who believe that everything should be free and open…
Consider me irked. I want to be perfectly clear about this: the compromises made in forge.mil are dangerous for forge.gov. If forge.gov were to follow the forge.mil “CAC Wall” approach, it will permanently damage the “Open GOTS” movement.
forge.gov cannot be forge.mil
forge.mil may be instructive and inspiring, but it’s a corner-case and fraught with compromises that have diminished its utility. In the case of forge.gov, it would be hosting unclassified code for civilian agencies. There’s no need to create a “trusted” environment. There’s no need to verify the citizenship or security clearance of its participants. The standard open source mechanisms are more than sufficient: only project leaders can commit code to the repository, a semiformal review procedure for patches, and so forth. In any case, I’m struggling to imagine why the repository would be better secured by allowed access to 300 million people. Let’s agree that making US citizenship a prerequisite is counterproductive, unworkable, unnecessary and most important: it’s un-American.
The entire country of Jordan has adopted the VA’s VistA software for their national healthcare system. Countless overseas researchers collaborate with their US counterparts through open source projects. When we share our source code with the world, it improves the quality of the software and is, in fact, a uniquely practical kind of diplomacy.
Since we don’t need to control access to the projects as we do in forge.mil, I have to wonder why we would need a prescribed set of tools for hosting each project. TeamForge is a fine piece of software, but there are literally dozens of viable alternatives. Developers are very picky about their tools, and extremely picky about the version control systems that are at the heart of these code repositories. If we presumptuously select tools on their behalf, we create an unnecessary barrier to entry.
The ideal forge.gov is two forge.govs.
With this in mind, we should break the forge.gov project in two.
First, forge.gov is useful as a catalog of open source projects that are used and created by the civilian government, many of which happily reside on agency websites or public repositories already. Forge.gov could be the way that projects and developers easily find each other. The Freshmeat or Ohloh of government, if you like. Let’s call this the forge.gov Catalog. The forge.gov Catalog would be as inclusive as possible, tracking the progress of every Open GOTS project we can find.
There are still projects that need an infrastructure, of course, and forge.gov could provide that. The forge.gov Repository would be provided by the GSA as a service to agencies, and provide a complete development environment. Here, I’m thinking of github or SourceForge.
I think by splitting these roles, and avoiding the dreaded CAC Wall, we can include the largest possible group of contributors and take best advantage of the excellent open source work that’s already underway.
Am I missing something here? Is there a national security concern that I’m overlooking? Is there an advantage to a homogeneous set of developer tools that I don’t understand? Let me know.