Like the Mariners’ string of depressing seasons and lost opportunities, this month’s botched rollout of the federal Department of Health and Human Services (DHHS) healthcare.gov website represents a huge lost opportunity for the federal government. And some of the root causes of these two fiascos are amazingly similar.
Washington state’s own, separate, health benefit exchange registered phenomenal success, enrolling over 35,000 people in three weeks.
How can one state succeed where the federal government, with all its resources, cannot?
In many ways, software development is like running a baseball club. The club will have an owner, or, if multiple owners, a board. The board hires the general manager (GM) who, in turn, hires the players, negotiates their contracts and hires the coach. The coach makes sure the players work together, trains the team and practices practices practices in preparation for game day.
The state of Washington succeeded by starting off with a solid infrastructure for the project, and by creating a separate Health Benefit Exchange – a public-private partnership – with its own board. Although the board is appointed by the governor and the Legislature, it is otherwise independent. And it hired its own CEO – veteran health executive Richard Onizuka. For their part, Onizuka and his staff made smart choices, contracting with a single vendor – Deloitte – to build its exchange.
As prime contractor, Deloitte bore sole responsibility for creating a successful exchange website. To ensure their success, Deloitte built the Washington exchange in the cloud – which allowed them to add extra server space to meet surges in demand – and tested it extensively with hundreds of different use cases. According to Michael Marchand, the exchange’s Director of Communications, they also tested the site with three waves of real, off-the-street, consumers.
During the project, Marchand reports, they hired BlueCrane, a separate quality assurance (QA) company to independently track Deloitte’s progress and deliver monthly reports to the board. “The reports showed red-yellow-green project status, and those reports are public on the Exchange website,” Marchand said.
The Washington site did have problems during its first few days, but the root cause was quickly found and fixed. Now the software is performing almost flawlessly.
In the Washington Health Benefit Exchange, Richard Onizuka is the GM, Deloitte is the coach and the team is well-practiced, with help from BlueCrane and diligent user testing. The Federal approach, with its numerous problems, provides a sharp contrast:
1. No strong GM. One of the basic principles of information technology project management is clear identification of a business owner and project sponsor – a GM – whose business is on the line for the project.
President Obama and Secretary Kathleen Sebelius, the owners of the Federal healthcare exchange, hired a bureaucracy — the Centers for Medicare and Medicaid Services (CMS) – to act as GM and oversee its technology. The administrator of that bureaucracy is Marilyn Tavenner, whose name has been almost absent from news reports and press conferences since the fiasco began.
2. No coach. Another principle of project management is having “one throat to choke”, a single person or entity responsible for making the project happen. A baseball team can buy a lot of highly skilled players and put them on the field, but unless the coach gets them to work together, the season will be a disaster.
Where the Washington state exchange hired Deloitte, the CMS bureaucracy assumed this role itself, hiring 55 contractors to build different pieces of healthcare.gov. CMS took on the role of “system integrator” - making all the pieces work together. That’s the equivalent of Mariners’ GM Jack Zduriencik and his whole front office coaching the team themselves.
3. Poor quality assurance (QA). A staple of good project management is to contract with independent consultants or agencies to oversee work and give early warnings of problems. A QA is similar to the batting coach or hitting coach hired by baseball teams, working to improve specific skills and warning the head coach of problems.
In the case of healthcare.gov, there were some agencies performing this function – the Government Accountability Office (GAO) raised major concerns in June, for example. So did many other experts inside and outside government, including CMS’ own Chief Information Officer (CIO) Henry Chao, who stated in March, “I’m pretty nervous. I don’t know about you. Let’s just make sure it’s not a third-world experience.” But DHHS itself scored the project as a perfect “5 out of 5” – no problems - as recently as August 29th. Clearly quality assurance was either not in place for healthcare.gov, or ignored, resulting in a string of successive glitches when the site went live.
4. Federal procurement is thoroughly broken. In software development, as in baseball, the goal is to hire a talented and experienced team. The primary Obamacare contractor is CGI Federal, a Canadian company created expressly for the purpose of winning federal government technology contracts – not an expert at developing complex websites.
Amazingly, not only did CMS hire CGI Federal, but they gave them an “Indefinite-Delivery, Indefinite-Quantity” (IDIQ) contract, which does not include any specific due dates or require any milestones or deliverables (such as a working website due on October 1). Why? IDIQs are easier for government agencies to issue: They can be given to pre-approved government contractors, like CGI Federal, and don’t require agencies to issue an open request for proposals, which can take a year or more under complex federal rules.
The heart of the problem, according to former O’Reilly Media Washington technology correspondent Alex Howard, is “a complex system of regulations that rewards contractors that are better at bidding on giant federal contracts than at building software.”
5. No sense of urgency; slow, not agile. Most private companies now use a software development approach known as agile, which emphasizes quick software development cycles — often in sprints of six weeks or less. Agile also often uses open source code, which means that programmers can view and critique each other’s work.
The front end of healthcare.gov (the part users see) actually was built using agile techniques, but CGI Federal built the guts (the part that does all the work) using an approach called waterfall. Waterfall projects are ponderous, lengthy and expensive, requiring the full completion of one task before the next can be started. As a result, the whole healthcare.gov system suffers from immense complexity and sloppy code – perhaps as many as 500 million lines of it – which will complicate the fixes.
So, if healthcare.gov were a baseball team, they would have a weak GM and no coach. They would have hired old, slow, and yet somehow inexperienced players, who ignore their batting and hitting coaches. All in all, this makes the Mariners look pretty good.
Perhaps this season isn’t lost for healthcare.gov. Maybe it’s only mid-season and the owner and GM can still save their win-loss record by contracting with hotshot utility players — the “tech surge” just announced by President Obama.
But it is unlikely Obamacare will be going to the World Series in 2013.
