HealthCare.gov website

The HealthCare.gov website. (Image © iStockPhoto)

By now, I’m sure you’ve heard about the debacle that is HealthCare.gov and how badly the rollout went. I’m also pretty certain that you you’ve only heard this discussed from a political perspective (who is to blame, what bureaucracies got in the way, whether the ACA is to blame or not), and not a technical one. There are a lot of lessons we can take away from HealthCare.gov, and not all of them political.

Let’s start by listening to the first two and a half minutes of this interview between MSNBC’s Chris Hayes and Clay Johnson, who was the lead programmer for Howard Dean’s 2008 presidential run as well as co-founder of Blue State Digital, which was largely responsible for the technical strategy behind Barack Obama’s 2008 presidential campaign.

So what did we learn?

  • Politicans are technically illiterate, and they certainly don’t know how to put together a website, which makes them poor candidates for determining what went wrong.
  • You need money, time, and talent to build a website.
  • The federal procurement process (how the government awards contracts to private firms) does not deliver talent, but it does gobble up money and time.

Think about your own “procurement process” and how you chose the vendor or individual to build your website. Did you understand all or most of what was in the bid? Did you feel comfortable and confident enough to ask smart and clarifying questions? If not, perhaps you need to learn a bit more about the space you’re working in. That will better enable you to communicate effectively and be sure that you’re hiring the right person for the job.

Next up, NPR’s All Tech Considered covered the fact that the website was built using the Waterfall software development model, which has fallen by the wayside because it produces messes like HealthCare.gov. While that is a little further in the weeds than I think we need to go, take a look at this quote:

[…T]he key problems we eventually saw with the rollout of HealthCare.gov last month: limited testing time, evolving requirements, over-reliance on contractors[…]. The really damaging decision, according to the consultants: launching “at scale.” The exchanges for all 50 states opened on the same day, instead of a few states at a time, gradually opening the marketplaces in phases.

Further:

Consultants noted there was no clear leader in charge of this project, which we now know contributed to its disastrous release. And there was no “end-to-end testing” of its full implementation, something we now know never happened.

In opposition to the waterfall method, the agile method stipulates that you launch large software projects in stages, not all at once (aka, “at scale”). That way, real users can test small parts of the system at a time, and you can collect feedback and bug reports that can be incorporated into your next launch phase.

Speaking of testing, what I really want to draw your attention to in those two paragraphs from NPR is that there was very little testing done. This is a common pitfall in small projects, as well as large. It’s important to spend time testing your site before it launches, and to test again when changes are made (called regression testing). According to multiple news sources, no testing had been done until 5 days before the October 1 launch date. No load testing either; the site went live with the ability to only handle 500 concurrent users.

To show a stark contrast, From Software is developing a video game called Dark Souls II, which takes advantage of online capabilities to augment gameplay. Take a look at their network testing schedule; note that they list test objectives, target geographic regions for each test, and note that these tests took place October through November of 2013. The game is set to release in March 2014. In other words, From Software has left itself plenty of time to do real user load testing.

In conclusion, the primary take-aways here are to equip yourself with the knowledge necessary to direct a website development project and to test, test, test.


Further reading:

Pin It on Pinterest

Share This