Organizations are figuring out how to deliver features faster, but they are still struggling to release without any issues. A recent report from Split Software found while a majority of organizations release new features on a bi-weekly basis, many are experiencing downtime after the new feature is introduced.
According to the company, 82% of respondents commonly uncover bugs in production, 38% have a mean time to resolution of greater than 1 day, and 41% have to roll back or hotfix more than 10% of new features.
“Our survey of the DevOps community has highlighted some troubling issues that directly result from the intense demand to release faster,” said Dave Karow, evangelist for Split. “There are inherent risks that organizations must bear, to speed these releases to market and remain competitive.”
Additionally, the company does not believe organizations are equipped to detect issues or address issues once they are detected, which can take a toll on user experience, finances and reputation. IDC has found that technical downtimes can cost Fortune 500 companies as much as $500,000 per hour.
“Once an issue is found, teams also often have to roll back code or hotfix it in production. Both of these practices can introduce additional risk,” the company wrote in a blog post.
In order to continue to move quickly, without introducing more risk into applications and services, Split recommends using feature flags for gradual rollouts; application, error, and feature monitoring to catch bugs quickly; and experimentation for avoiding unnecessary work.
Other findings of the report included: 87% of respondents release new features more than once a month to keep up with the demand for new features; 88% of DevOps teams need more than an hour to resolve detected issues; and 27% of respondents think features are poorly adopted and utilized.
The report is based on conversations with more than 100 different DevOps organizations in the US.