Years ago, comedian Steve Martin joked about the best trick to play on a 3-year-old kid: “Whenever you’re around him,” he said, “talk wrong.” On his first day at school, the kid will raise his hand and say, “May I mambo dogface to the banana patch?”
Pitching a project for implementing DevOps at your company can make you feel a lot like that kid. To you, the ask is crystal clear. To your executives, it’s mumbo jumbo. Why? Because if you’re like lots of technical folks, you’re using technical terms to make your case. However, “technical” is not the language that your management team speaks when discussing new processes and tools. When you’re asking to “mambo dogface,” they’re thinking about things like, “How many people do I need to implement this?” or “How many people will need to be trained?” and “How long will it take them to become proficient?”
Take heart! That glazed look in their eyes doesn’t necessarily mean they don’t care about DevOps. It’s just that it’s critical to establish the correct language to help them understand the benefits of the DevOps initiative. This means explaining the benefits in terms they’ll understand—impacts to business and financial costs.
In order to use the correct language, you need to understand the full cost of your application delivery pipeline. And the scary truth is, application delivery is WAY more expensive than most people realise. A simple application deployment from development to production can cost an average of $300k for just one application.
Breaking out of budget silos
The disparate language problem occurs because companies unintentionally obscure the real costs of application delivery. The reason is companies think about application costs in two ways. First, companies typically budget for applications on a per-project basis. Second, the staff for these projects are separated across organisational groups such as development, QA, and Operations. The result is that while costs are properly budgeted for application changes, the costs associated with application delivery are not properly correlated for application release from development to production.
Additionally, most companies speak in terms of cost-of-deployment (COD) by only focusing on the cost of production deployment events. Once development is done with its changes, development teams hand-off their changes to QA and move on to the next project. But what happens if problems in production force developers off their current projects to work on their previous project? Budgets are drained when staff and infrastructure have to be reassigned to “closed” projects, burying an application’s actual cost.
Let’s take a look at how you can uncover these hidden costs inside your application release process, and talk to management in a way that will make them much more receptive to your DevOps initiative. All it takes is a simple spreadsheet and a bit of effort to pull together numbers that are probably already know.
Calculating the cost of an application
In my previous role as an IT manager, I developed a method for itemising each stage of application delivery. Today, I use this method in my work helping enterprise customers implement DevOps initiatives. Let’s walk through a real-life example I created with input from the procurement and IT managers of a large insurance company. We’ll use a basic spreadsheet to do this. We’re looking for a ballpark here, so we’ll keep these costs simple. Note: all costs are in USD.
Estimate staffing costs
The best place to start is by estimating the average hourly value of staff. The good news is that most IT managers deal with these numbers all the time. You can see in the figure below that we’ve estimated the average hourly cost of IT deployment staff at $125 and Business and Test staff at $75.
Estimate the number of deployments, environments, and labor hours
Let’s move on to the cost of deployments leading up to production. The goal here is to capture the number of people actively involved in the deployment/release to an environment, as well as the number of hours to deploy the application to a particular environment.
We start by estimating the number of deployments to development per year. For this part of the exercise, I assume that all deployments to development are handled by developers, and the deployment process is fully automated. For this reason, these releases should be “quick” and require very little time from any development staff.
Next, list the environments that an application must travel through to get from development to production. Incredibly, most companies skip over these “middle” environments when budgeting for a project, even though application delivery to these environments has an enormous impact on things like costs attributed to the project and QA project schedule(s). Another reason to focus on these environments is to quantify the impacts to your business users, for reasons such as scheduling implications, project delays and missing project deadlines.
To estimate the number of deployments per year for these “middle” deployments, we multiply 1/3 of the development deployments by the number of environments (33 * 4) for a total of 132. I use this math because not all deployments to development are going to be pushed forward to production. But, we all know that there are always multiple iterations of releases from development to the middle environments. These multiple pushes are related to bug fixes, as well as staggered code/feature/enhancement delivery.
We estimate the number of hours to deploy to each one of these environments at 3 hours, since, generally speaking, the amount of automation decreases as you move from development towards production. Also, these deployments are a combination of deployment steps (both automated and manual), as well as release scheduling. You need to ensure the receiving team is ready, then deploy the code. After that, initial validation needs to take place to sign off on the release.
Finally, we look at the error rate to deploy to each environment from development through Pre-production. Here, I’ll use a number like 15% to start the conversation and adjust up or down depending on how the team describes their processes. You then want to estimate the number of hours to fix release errors. Remember, it’s important to think about all the work that needs to be done to stabilise an environment. For example, to stabilise QA after a failed release, we need to think of all the remediation steps, like sending out notifications, analysing the problem, fixing the code and redeploying the application.
Estimate the cost of production releases
Next, we estimate the cost of production itself, including the typical number of major and minor releases for an application, total labour hours, error rate and hours to resolve a bad deployment. Most companies account for the cost of a production deployment when budgeting for a project but tend to overlook the failure rates, as well as time for recovery.
Estimate production failures
Finally, there is the number that no one likes to talk about—the costs to the company of an unexpected production outage. This number represents the impact to your end users. However, failures must be accounted for to get a true estimate of the cost of an application, and can be a simple high-level estimate to help you get your point across. For example, you are a trading firm and your customers cannot log in after a failed website release. How much money will your company lose while the website is down during a rollback of the release or another release to fix the first?
The moment of truth
It’s time to calculate your totals, starting with the current cost of development deployments through to the current cost of production deployments. My method of calculating each of these numbers appears in the Notes column. Now add up everything in the Values column and
there you have it. The complete, estimated cost of the application release cycle from development to production. In this example, you can see that the cost for this one application is almost $800k. When I work through this exercise with people, this is where I typically pause and let the folks tweak the spreadsheet to lower that number. I can tell you that even when all the numbers are adjusted lower, you will still be shocked to see what a single application release from development to production is costing you.
Of course, this is just one of potentially hundreds of applications that a company works on each year. If you think the cost is hard to believe, you’re not alone. Once you’ve tweaked the spreadsheet to represent your delivery, you will have a clear picture of how budget silos are hiding the real cost of software delivery. You’ll also have insight into how much money the current release processes are draining from the business. For me, the most shocking part is that all the releases from Post-development to production comprise such significant expenses to IT projects, yet are not accounted for in many IT budgets.
Next step: Try it yourself
If you’re considering proposing a DevOps initiative to your company’s management, I encourage you to do this exercise for the team for which you’re trying to implement DevOps. Work with the teams involved with delivery of the application from development to production, and modify the spreadsheet accordingly.
The results may surprise you, but don’t be discouraged. By uncovering the true cost of application delivery, you’ll walk away not only with the business language to convincingly make your case to management, but also with ROI goals that will help guide your team to DevOps success. Instead of delivering a confusing proposition of “mambo dogface,” you’ll be speaking the language that your IT Management needs to help you implement your DevOps initiatives.
T.J. Randall, VP Customer Success, XebiaLabs
Image Credit: Profit_Image / Shutterstock
No comments:
Post a Comment