A recent VentureBeat headline misleadingly suggests agile development practices were the cause of Healthcare.gov’s “failure.”
The article is in reference to a new U.S. Government Accountability Office report released Thursday: “Healthcare.Gov: Ineffective Planning and Oversight Practices Underscore the Need for Improved Contract Management.”
Because there’s been more and more attention given to IT project management methodologies, particularly agile, in the aftermath of the Healthcare.gov launch, and most people don’t read beyond the headlines, it’s important to shed some light on how agile was actually referenced in the report.
The story, “Feds say agile development contributed to Healthcare.gov failure,” makes a two-sentence reference to agile, neither of which suggest agile methodologies were the issue. The issue, as indicated in the article, was how the Centers for Medicare & Medicaid Services and its contractors implemented agile.
From the post:
The CMS allowed the contractors to use an “agile” approach to developing its data hub and website, when the CMS had already admitted that it had little experience developing in that way, the report says.
Here are excerpts of the report’s agile references:
To help manage compressed time frames for FFM and data hub development, CMS program officials adopted an iterative IT development approach called Agile that was new to CMS. Agile development is a modular and iterative approach that calls for producing usable software in small increments, sometimes referred to as sprints, rather than producing a complete product in longer sequential phases. However, we found that the quality assurance surveillance plans were not used to inform oversight. For example, contracting and program officials, including the COR and contracting officer, were not sure if the quality assurance surveillance plan had been provided as required by the FFM and data hub task orders. Although a copy was found by CMS staff in June 2014, officials said they were not aware that the document had been used to review the quality of the contractor’s work. Instead, CMS program officials said they relied on their personal judgment and experience to determine quality. The Office of Management and Budget issued guidance in 2010 that advocated the use of shorter delivery time frames for federal IT projects, an approach consistent with Agile. However, CMS program officials acknowledged that when FFM and data hub development began in September 2011, they had limited experience applying an Agile approach to CMS IT projects. In 2011, CMS developed updated guidance to incorporate the Agile IT development approach with its IT governance model, but that model still included sequential reviews and approvals and required deliverables at pre-determined points in the project. In our July 2012 report, we found a number of challenges associated with introducing Agile in the federal environment. Specifically, we found that it was difficult to ensure that iterative projects could follow a standard, sequential approach and that deviating from traditional procedural guidance to follow Agile methods was a challenge. We also reported that new tools and training may be required, as well as updates to procurement strategies. Therefore, the new approach that CMS selected in order to speed work also carried its own implementation risks. ... Despite the revised FFM schedule, it is not clear that CMS held all of the governance reviews for the FFM and data hub or received the approvals required by the life cycle framework. The framework was developed to accommodate multiple development approaches, including Agile. A senior CMS program official said that although the framework was used as a foundation for their work, it was not always followed throughout the development process because it did not align with the modified Agile approach CMS had adopted.
So, GAO didn’t say agile was the cause of Healthcare.gov’s failures. It merely outlines the bureaucratic challenges of effectively implementing it.
In fact, GAO has been supportive of an agile approach in the past. A 2012 report, “Software Development: Effective Practices and Federal Challenges in Applying Agile Methods,” recommended the CIO Council continue to champion agile and modular practices throughout the federal government.