Let’s start by listening to the first two and a half minutes of this interview between MSNBC’s Chris Hayes and Clay Johnson, who was the lead programmer for Howard Dean’s 2008 presidential run as well as co-founder of Blue State Digital, which was largely responsible for the technical strategy behind Barack Obama’s 2008 presidential campaign.
So what did we learn?
- Politicans are technically illiterate, and they certainly don’t know how to put together a website, which makes them poor candidates for determining what went wrong.
- You need money, time, and talent to build a website.
- The federal procurement process (how the government awards contracts to private firms) does not deliver talent, but it does gobble up money and time.
Think about your own “procurement process” and how you chose the vendor or individual to build your website. Did you understand all or most of what was in the bid? Did you feel comfortable and confident enough to ask smart and clarifying questions? If not, perhaps you need to learn a bit more about the space you’re working in. That will better enable you to communicate effectively and be sure that you’re hiring the right person for the job.
Next up, NPR’s All Tech Considered covered the fact that the website was built using the Waterfall software development model, which has fallen by the wayside because it produces messes like HealthCare.gov. While that is a little further in the weeds than I think we need to go, take a look at this quote:
[…T]he key problems we eventually saw with the rollout of HealthCare.gov last month: limited testing time, evolving requirements, over-reliance on contractors[…]. The really damaging decision, according to the consultants: launching “at scale.” The exchanges for all 50 states opened on the same day, instead of a few states at a time, gradually opening the marketplaces in phases.
Consultants noted there was no clear leader in charge of this project, which we now know contributed to its disastrous release. And there was no “end-to-end testing” of its full implementation, something we now know never happened.
In opposition to the waterfall method, the agile method stipulates that you launch large software projects in stages, not all at once (aka, “at scale”). That way, real users can test small parts of the system at a time, and you can collect feedback and bug reports that can be incorporated into your next launch phase.
Speaking of testing, what I really want to draw your attention to in those two paragraphs from NPR is that there was very little testing done. This is a common pitfall in small projects, as well as large. It’s important to spend time testing your site before it launches, and to test again when changes are made (called regression testing). According to multiple news sources, no testing had been done until 5 days before the October 1 launch date. No load testing either; the site went live with the ability to only handle 500 concurrent users.
To show a stark contrast, From Software is developing a video game called Dark Souls II, which takes advantage of online capabilities to augment gameplay. Take a look at their network testing schedule; note that they list test objectives, target geographic regions for each test, and note that these tests took place October through November of 2013. The game is set to release in March 2014. In other words, From Software has left itself plenty of time to do real user load testing.
In conclusion, the primary take-aways here are to equip yourself with the knowledge necessary to direct a website development project and to test, test, test.
- Regarding large scale government tech projects and why they fail: Getting to the bottom of HealthCare.gov’s Flop by Clay Johnson
- Regarding Obama’s “tech surge” and why it very likely won’t help: It might take a long time to get HealthCare.gov working, no matter what Obama does by Timothy B. Lee
- Regarding waterfall vs. agile software development: Agile Vs. Waterfall: Evaluating the Pros and Cons by Kasia Mikoluk