I had a chance to talk with Matthew Burton, the former intelligence analyst turned open source cause celebre who just launched a tool that helps frame and understand arguments with imperfect evidence. It’s based on method called Analysis of Competing Hypotheses (ACH), which has been around for quite some time. Matthew and his friend Josh Knowles, though, have a tool that allows the ACH method to be used by multiple participants simultaneously. It’s fascinating stuff, so I’m grateful that he took the time to talk with me.
On a personal note: I’m delighted to see that Matthew is a fellow emdash enthusiast, as you’ll see below.
First, tell me a little about ACH and how you first became interested in the method.
In the fall of 2005, Dick Heuer, the creator of ACH, contacted me after reading an article I’d written for Studies in Intelligence. The article was about how Intelink could benefit by being more like the Web. Dick had been wanting for some time to build a Web-based, multi-user tool for ACH, so he asked me to build it. I spent the following summer at DS&T, interviewing ACH practitioners and trainers.
Intellectually, the most fascinating aspect of this project has been its applicability to groupthink and dissenting viewpoints. When I started, the Intelligence Community was still feeling the effects of Iraq WMD blowback. Dick referred me to a book, “Groupthink: Psychological Studies of Policy Decisions and Fiascoes,” by Irving Janis (ISBN: 0395317045). It’s a fascinating book. Janis evaluates several US policy failures from the 20th century. He not only makes it clear that a groupthink tendency had a hand in misguiding groups of otherwise brilliant men (they were all men); he also pinpoints the moments where much-needed dissent and skepticism were quashed by the desire not to disrupt the camaraderie that comes with consensus.
In my summer at DS&T, I learned that this was a problem with ACH as well. The existing software couldn’t record multiple viewpoints, meaning dissenting opinions evaporated; and analysts had to state their opinions in front of everyone else instead of from the pressure-free environment of their desks. I saw a real opportunity to make a dent, however small, in the problems that lead to intelligence failures.
Are you dogfooding? Has the hyper-rationality of ACH slipped into your real life? Did you apply ACH when you asked your wife to marry you? :)
Ha! Fortunately, I can weasel my way out of that question just by explaining the purpose of ACH. It’s a tool for discovering facts–either has already happened, or what will happen in the future. When it comes to questions that involve personal preferences, it’s not going to work so well, because its goal is to keep you from thinking subjectively about objective matters, not help you objectify the subjective.
Were you a developer first, or an analyst first?
I actually wouldn’t call myself a developer even now. I maintain the ACH code, but most of it was written by my friend Josh Knowles, a classmate from ITP who’s collaborated with me on several projects. But I’ve been a geek for a long time, having been neck-deep in the Web since I was 15. I took that knowledge–and my expectations for the national security structure’s technological prowess — into my job at DIA. That led to disappointment, so I channeled that negative energy into a desire to change how this place works.
Does your analyst interest complement your developer interests, or are they two separate things?
Most of the development projects I work on are meant to fix a problem that is consuming me, whether it be intelligence analysis, legislative transparency, or political campaigning. I mentioned I’ve been a geek for a long time. But I consciously avoided studying computer science in college because I didn’t want to code for it for its own sake, as an end; I didn’t want to end up building Web sites for online pet stores. Very soon after joining the government, I realized that my interest in technology could be used to further a bigger goal, and I’ve been doing that ever since.
If you developed this tool while a contractor, how did you end up with this code? What kind of hurdles did you have getting a copyright?
Josh and I developed the code ourselves, so that’s why I have the copyright. We did it in collaboration with both Dick and a DC consultancy called Pherson Associates. The Phersons—two retired CIA veterans—have been training analysts to use more structured analysis techniques like ACH for years, so they and Dick were there to make sure the software doesn’t betray the methodology. We built this for an intelligence community platform called A-Space, which was supposed to cater to outside developers, much like Facebook opened its platform to third-party app developers. But it’s been two years since A-Space launched, and neither the development specifications nor the purchasing mechanics have been defined. In that time, our own lives have changed: I’m an entrepreneur handling multiple projects in New York, and the Phersons are neck-deep in their growing training business, so it makes sense for us to take the ACH software in a direction that keeps us from having to maintain dozens of licenses while still allowing us to shepherd it. That means open source.
I know you’ve spent a lot of time thinking about licensing. Can you walk me through your thinking, and how it’s evolved?
It hasn’t evolved so much as led me in infinite loops. My goals for open sourcing the code are in some ways conflicting: I think ACH can help the world make better, more fact-based decisions, so I want it to be as widely adopted as possible. But I’ve been on a personal crusade to change the way the government buys software—namely, they should spend less on it and avoid vendor lock-in. So these two goals pull me simultaneously away from and toward copyleft. Beyond copyleft, the intricacies are so fine that I find myself unable to really comprehend the eventualities that my various options would lead to. When I reach that point in my thinking, I realize that I’m probably overthinking it, so I stop. Then I do it all over again.
It sounds like you feel as though open sourcing the code would lead to less use in the government? Can you unpack that for me a bit?
I feel as though certain FOSS licenses would lead to less government adoption than would other FOSS licenses. Specifically, the government relies on lots of proprietary legacy systems that would be contaminated by GPL code. And any intelligence agency that modifies the code would be loathe to share those modifications back to the world. So according to some people I talk to, the GPL is a non-starter. However, other people say that the copyleft provision would not be invoked simply by providing GPL code to government users, as such code is only being used internally. Different people have different opinions about the GPL’s acceptability in government, which makes me think that the fate of the software depends less on my licensing decision than it does on the worldview of the prospective user. This realization takes me back to the end of that thought loop, where I feel like I’m overthinking it.
Who do you imagine using this software, beyond government? If I work in a group of analysts, it makes sense — but do you imagine other, commercial uses? Non-commercial?
On the commercial side, anyone whose profession requires them to use the facts at hand to figure out a puzzle: pathologists, investigative journalists, detectives, investors. I’m really excited about the non-commercial side. It’s been frustrating to watch political discourse in our country devolve into nonsense that is less about the facts and more about what “feels” right. This attitude is beginning to take hold of the policy process as well, and that’s bad news. I’m under no illusions that ACH matrices will start appearing on the Senate floor, but the idea of using ACH-backed arguments to explain policy proposals to lawmakers is intriguing. I think this falls in line with the mission of Expert Labs, and I plan to reach out to them and see if we can do something cool together. I also think universities are a good fit for ACH, as college is all about challenging your preconceived notions and teaching you the meaning of rigorous research.
If I’m a developer who’s intrigued by the ACH idea, how can I help?
Go to http://competinghypotheses.org. There, you’ll find the repository, the code documentation, and information on how to join the community.
[The site will be live in about two weeks, Matthew and Josh are working furiously on it, I'm sure. For now, you can sign up to get notified when it's up.]
So what’s next for you and Josh? What other projects are you working on?
We’re big fans of Stack Overflow, the Q&A community for developers. We think it could be a great platform for answering citizens’ questions about government: navigating bureaucracies and legal codes is very daunting, and finding answers to simple questions—how do I amend my tax return, how do I fight a parking ticket, how do I get a permit—can be extremely frustrating. Government agencies are experts at making you wait in lines and on the phone. We know that they answer the same questions many times a day, and that private citizens can usually add valuable information (“Don’t go at 1 pm, it’s really busy”) that the government employees can’t or won’t provide. We’d like to form a community of confused citizens and people who are willing to help them, so in between ongoing projects, we’ve been sketching out how we want to accomplish this — whether it should be done through Stack Exchange, whether we should build our own, etc.
[You can learn more about the project at http://govdecoder.com/.]
Finally, what’s your favorite government open source project?
I’ve never seen it in action, but DHS’s Virtual USA project sounds remarkable. On top of using open source software to build it, the objective of the project is to break another government taboo: sharing information with other agencies and levels of government. Having been an intelligence analyst who relied a lot on mapping tools and was constantly frustrated by the inability to share geographic data even within your own building, it’s apparent that if Virtual USA delivers, it’s going to dramatically change how first responders work.
# # #
I want to thank Matthew for taking the time to speak with me. I think the project is fascinating, and I can’t wait for it to launch.