Recognized as a Leader on Gartner Magic Quadrant for BCMP Solutions, Worldwide

Sympathy for HealthCare.gov

Oct 30, 2013

Back to Veoci BlogSympathy for HealthCare.gov

Comedians, commentators, and politicians are having a field day with the problems at healthcare.gov. As software developers and computer operations experts, part of us is just not laughing. We have spent years in high-volume web software and the jokes and blogs about outages and errors do not make us laugh as hard as everyone else. Scar tissue perhaps?

Obamacare - Error Page Could the government have done better? 

Yes, absolutely. There has not been a single outage or performance slowdown that we later found out we could not have handled better. Rewind and watch in slow motion, and you will always see what could have been. Fear of the potential aftermath makes working on outages a double-edged sword: even if you work wonders getting the system back online, it will be followed by a penalty for not having prevented the problem in the first place. Having been in numerous high stress outages and having worked to resolve them in the trenches, we share some sense of camaraderie with the IT team behind the Obamacare website. Popular belief assumes a straight line from disaster to the nirvana of perfectly-running systems. The reality is that the path out of an outage or performance problem requires a difficult decision between several options, all of which cause some damage and require time for recovery, and there is always uncertainty about the extent of the damage and recovery time. 

One could (a) go into full disaster recovery mode, (b) roll back all changes to some previous version when things were supposedly running well, or (c) try and find the problem and fix it on the fly. Disaster recoveries can fail and usually include loss of some data. Previous versions may lead to the same problem with the new situation; roll back could introduce other errors; or a previous version could lead to worse issues that you just don't know about. Most organizations first attempt to fix the problem on the fly, and this has its own set of issues: experts may not be available, experts may not be able to find the issue, vendors might not be responsive, or most likely, unlike disaster recovery or rollback, one usually cannot estimate the exact time in which a solution will be completed. 

 Selecting the path to follow requires a "General Patton" spine; the "fog of outage" makes knees weak. The solution is to develop a specialized team that is brought together for disasters and not only has superb technical capability, but also the instinct to make difficult decisions quickly - people who almost subconsciously reach the best answer. As we have developed Veoci over the past three years, our initial inspiration came from the people on the front line who work to resolve outages. Better communication, task management, and rapid team assembly are the basics. A complete record of what was known at what time makes the lives of our responders easier: "Well, at that time, all we knew was X." As we talk to IT managers, it is often instructive to find out if they have personally lived through high-stress outages. If they have, the Veoci demo brings out a cathartic outpouring of how useful Veoci would've been. 

With what must be an enormous load on the Obamacare site, their problems are understandable. It is a new system, and the emergency management team is still getting to know each other and starting to blow away the chaff. Combine the bureaucracy ensured by procurement and development rules with the complexities of coordinating hundreds of developers, numerous systems, and millions of users, and the current situation seems inevitable. Now that it is a crisis, healthcare.gov is probably in agile development mode.

The bureaucracy has run for the hills, and a much smaller and focused team is fixing the issues. After all, healthcare.gov is neither a massive system nor is its load any more daunting than what many web sites now handle routinely. No new technology is needed, just better execution. I would expect them to stumble through November and for everyone to forget about how bad the situation was by March 2014. -Dr. Sukh Grewal, CEO Grey Wall Software, developers of Veoci

Subscribe to the Veoci Blog

Receive all the latest emergency, crisis, and continuity management news, tips, and advice

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Related Posts

ITDR Communications: Lessons from a Middleware Bug

Anyone with the will and skill could crack the bug, which spelled serious danger for MuleSoft’s users and those users’ customers. MuleSoft needed to patch the bug immediately. And they did, all while making their customers aware of an issue that could’ve flown under the radar.

Continue reading
The “Magic” of Veoci

As the IT market is replete with contenders, it’s important for companies to stand out from the crowd - getting named by Gartner in the Magic Quadrant is a big boost.

Continue reading
Business Continuity vs ITDR: What are the Differences?

In the past, you may have heard the terms “business continuity” and “disaster recovery” used in conjunction, or even interchangeably, but what do they really mean? You probably won’t be surprised to discover that they have many similar goals when it comes to recovering from an unplanned incident and restoring essential functions, but it is the nuances of their differences that are really crucial to understand.

Continue reading

Connect with us on Social Media

Join us on our journey to improve emergency, operations, and continuity management!

Veoci Facebook PageVeoci Twitter AccountVeoci Linkedin Company Page

Face crisis and continuity challenges with expert solutions designed for you and your teams.

Learn how Veoci puts you in control