As the New York City Velocity conference winds to a close, this year’s Web operations and performance event was highlighted by some engaging speakers and a laundry list of Web development news and releases from companies big and small.
In addition, Mikey Dickerson, a former Google engineering manager who’s now a member of the tech staff for the newly established U.S. Digital Service, gave a keynote. In “One Year After HealthCare.gov: Where Are We Now?” he gave the audience an inside look at how he got roped into overseeing the massive website cleanup and overhaul of HealthCare.gov.
What started as a favor to some desperate government friends and a multi-day trip to D.C. turned into—through what Dickerson dubbed “commitment escalation”—almost four months of 17-hour days overseeing hundreds of engineers and programmers from more than 50 different companies and contractors.
What exactly was wrong with HealthCare.gov? As Dickerson put it, the project went exactly as normal considering government IT projects fail spectacularly all the time. The only unusual factor was that failure becoming national news.
“Over 50 different companies were hired to build this site,” Dickerson said. “When a different company is responsible for each aspect of the site, you’re trapped in an eternal loop of damnation. The site was down in real time, days ticking away, but surprisingly there really was no sense of urgency.”
Because the HealthCare.gov project was completely mismanaged from the get-go, Dickerson said there were a lot of easy, basic fixes. He and the four other “experts” brought in to assess the damage created a “war room” in the Virginia office, installing TVs and desks and renting out an entire floor so that the disparate site contractors were all working out of the same place, with Dickerson often running the twice-daily operations meetings. As it said in the team’s initial report to the White House, one of the first things to do was to add monitoring.
(Related: Velocity NYC: Two myths of control from Chef’s enterprise architect)
“We sent out this report. Bad news: Everything has been done wrong. Good news: Because of that, there are a lot of easy, basic fixes. Will that be enough to make the site work? There was no monitoring, no dashboard, no nothing. No place to look to find out whether the site was up today or not besides CNN, which is how we found out about a lot of problems in the beginning. You’re dead in the water if you can’t see what’s going on on the site.”
The White House announced a revised goal of signing up 4 million people for the Affordable Care Act by Dec. 1, so Dickerson and his team of both incumbent developers and newly brought-on help went about fixing things little by little, following chains of events to discover problems like the fact that contracts were working off of separate issue ticketing systems. He explained that while it was labor-intensive, fixing HealthCare.gov didn’t require inventing any new techniques of website management.
“Dec. 1 came around and miraculously, things sort of worked,” Dickerson said. It was a very, very strange experience. Like this imposing castle everyone thought was unfixable. It was made out of sand. We didn’t think we would fix it, we just gave it a shot because we had to.”
By April, 8.1 million Americans enrolled through HealthCare.gov, more than the pre-fiasco estimate of 7 million, preventing a delay in the Affordable Cart Act’s implementation and facilitating the first major healthcare reform since the advent of Medicare and Medicaid in the 1960s.
When Dickerson finally did get back to Google, he found it hard to care. So two months later he returned to D.C. for good to head the U.S. Digital Service, fixing the hundreds of other shoddily built and broken websites vital to government infrastructure with what he described as “the highest level of government support to make removing obstacles possible” to plug the online “sinkhole of bureaucracy.”
“This is real life; this is your country,” Dickerson said. “These problems are fixable, these problems are important, but they require you to choose to work on them.”
Velocity news and notes
Here’s what else was on hand concerning Web development, operations and application performance.
Ansible: Configuration-management and IT automation company Ansible demonstrated its automation products, including the newly released Ansibile Tower team-based UI.
AppNeta: APM company AppNeta exhibited its SaaS-based full-stack solution for integrated performance visibility through application code and the network to the end user. The APM suite includes TraceView, AppView, FlowView and PathView solutions.
Catchpoint: Web performance monitoring solution Catchpoint demonstrated its real-user measurement and synthetic monitoring capabilities, and CEO Mehdi Daoudi presented on preparing for IT disasters and creating a culture of quality website performance.
Chef: IT infrastructure automation provider Chef exhibited the release candidate of Chef 12, which boasts a single codebase containing open-source and commercial versions. More information about Chef 12 is available here.
Crittercism: Mobile application performance-management solution Crittercism exhibited its mAPM solution to monitor and manage mobile app performance on a complete mobile stack.
Cumulus: Cumulus Networks, a Linux operating system for networking hardware, exhibited its Cumulus Linux bare-metal networking distribution. It also announced partnerships with Dell and VMware to bring an open networking ecosystem to software-defined data centers.
Dell: Dell Software exhibited the SaaS edition of its Foglight application performance-monitoring solution and cloud manager, which recently gained Android and iOS support.
Ghostery: Enterprise SaaS provider Ghostery exhibited its free browser plug-in for data privacy, along with a presentation from COO Emily Riley on security “blind spots” in e-commerce sites.
HP: HP announced an expansion to its Performance Testing Suite with HP StormRunner Load, a cloud-based platform focused on application quality and delivery in agile development. HP StormRunner Load will be available in October.
Keynote: Mobile and Web performance-management analytics company Keynote announced mobile QA testing for wearables. It also released a competitive rankings report comparing native mobile app performance and the addition of smartwatches to Keynote’s cloud-based mobile testing solution.
Opsmatic: Configuration monitoring company Opsmatic demonstrated its agile solution with features such as automatic drift detection, and keyword search to identify specific configuration problems.
SmartBear: Application software provider SmartBear exhibited its AlertSite UXM Application Quality and Performance Management Platform with a new JavaScript tracing feature. The platform integrates several of SmartBear’s offerings into one hub, including SoapUI and Lucierna.
SOASTA: Cloud-based testing provider SOASTA announced an update to its mPulse Real User Monitoring solution, which collects user data in real time and performs “What If” analysis. The update is focused on enhanced Big Data analytics capabilities, updated iOS support for touch-based data elements, a new FlipBook dashboard mode, and added data dimensions.
SoftLayer: Cloud computing, managed hosting and server provider SoftLayer (an IBM company), put on an arcade-style Server Challenge, timing attendees as they pull and plug server wires to load data, connect the network and “save the world.”
