From Cloud Chaos to Financial Clarity: Leveraging FinOps
The adoption of cloud services accelerated during the COVID-19 pandemic. According to a report by International Data Corp., total worldwide spending on cloud computing, including hardware, software, and support services, will surpass $1 trillion in 2024, sustaining a compound annual growth rate of 16%.
Finance executives, previously more familiar with budgeting for server hardware and software licenses, have a new IT-related challenge: How to get a handle on and best manage the costs of burgeoning cloud services.
Sound familiar? It should…it’s strikingly similar to how the chasm between software developers writing code and the IT operations folks responsible for implementing and maintaining it led to massive complications in forecasting the cost, schedule, workload, and quality of the software. DevOps became the answer to integrating these disparate practices with outstanding results. With developers and IT operations acting as one team, organizations push products and services into customers’ hands quicker and in closer alignment with the forecasted cost and schedule goals.
FinOps is transforming the finance of cloud services the same way that DevOps transformed IT development and operations by providing greater traceability between cloud cost drivers and business outcomes.
The Size of the Problem
A 2020 study by International Data Group and IT services provider Insight revealed that public cloud cost overruns are strikingly common. The study was based on a survey of 200 U.S. business and IT executives. About 70% of the respondents said their organizations had experienced public cloud costs as much as 62% higher than initially anticipated.
A separate survey of senior enterprise IT professionals by Pepperdata found that cloud spending for 2020 was expected to be over budget by between 20% and 40%.
It’s not just studies and surveys that tell the problem. Some companies have publicly admitted their challenges:
Twilio: In 2020, cloud communications platform Twilio reported higher-than-expected cloud costs due to increased usage during the COVID-19 pandemic.
Baidu: In 2020, Chinese search engine giant Baidu reported higher-than-expected cloud costs after expanding its cloud services portfolio.
Robinhood: In 2020, the online brokerage firm Robinhood encountered unexpected cloud costs during a period of the high trading volume.
GitLab: In 2021, software development company GitLab reported unexpected cloud costs after a spike in usage during the COVID-19 pandemic.
To address their unexpected cloud costs, 3 of the four (Twilio, Robinhood, and GitLab) began leveraging FinOps principles to set budgets and optimize their cloud spending, using tools such as AWS Cost Explorer and CloudHealth to monitor and analyze their usage.
Baidu, also launched programs and processes leveraging FinOps principles to optimize their cloud usage, including monitoring and analyzing their cloud spending, identifying and eliminating waste, and negotiating better pricing with cloud providers. While they didn’t employ AWS tools, they did establish a cloud cost center and a cross-functional team responsible for managing cloud costs, consisting of representatives from finance, IT, and business units.
The challenge lies in the fact that most engineers may not fully comprehend the impact of cloud expenses on organizations. For example, one engineer migrated their website to a serverless AWS stack, primarily to take advantage of the ability to autoscale in the event of unexpected traffic spikes. However, two weeks later, the company received a notification from AWS stating that their actual costs were 41 times higher than anticipated. AWS also projected that the customer’s charges would be 12 times higher than predicted for the remainder of the month.
Leveraging FinOps can help organizations meet these challenges by achieving greater visibility and control over their cloud spending, enabling them to make informed decisions about resource allocation and investment, and ultimately optimizing their cloud infrastructure to meet their business needs while minimizing costs.
Cloud FinOps Definition
Short for Financial Operations, “FinOps is an evolving cloud financial management discipline and cultural practice that enables organizations to get maximum business value by helping engineering, finance, technology, and business teams to collaborate on data-driven spending decisions.” (Definition Updated: November 2021 by the FinOps Foundation Technical Advisory Council)
In essence, leveraging FinOps requires a cultural approach that involves team collaboration to effectively manage cloud costs. It entails taking responsibility for cloud usage, with support from a central group that adheres to best practices. Teams across different functions, such as Engineering, Finance, and Product, work together to achieve faster product delivery while also gaining greater financial control and predictability.
Six principles for effectively leveraging FinOps:
- Foster collaboration: The finance and technology teams should work closely together to continuously improve efficiency and innovation by treating the cloud as a driver of innovation.
- Focus on business value: Unit economic and value-based metrics should guide decisions as they demonstrate the business impact better than aggregate spending. Cost, quality, and speed should be balanced while considering the cloud as a driver of innovation.
- Promote ownership and accountability: Engineers should take ownership of costs from architecture design to ongoing operations, with individual teams empowered to manage their own usage of the cloud against their budget. Cost-effective architecture, resource usage, and optimization should be decentralized.
- Provide timely data: Real-time cost data should be accessible to all levels of the organization to drive better cloud utilization. Fast feedback loops lead to more efficient behavior, and real-time financial forecasting and planning help explain cost increases.
- Establish a centralized team: A centralized team should evangelize and enable best practices in a shared accountability model, similar to security. Rate, commitment, and discount optimization should be centralized to take advantage of economies of scale.
- Embrace the variable cost model of the cloud: The variable cost model of the cloud should be seen as an opportunity to deliver more value, not as a risk. Just-in-time prediction, planning, and purchasing of capacity, agile iterative planning, and proactive system design with continuous adjustments in cloud optimization should be embraced.
Promoting Good Behavior
Effective governance is also a crucial component for leveraging FinOps for cloud services. Participants must realize that analysis should not end after selecting an initial cloud provider. As business needs and strategies evolve, the cloud solution must also evolve. With the increasing complexity of workloads in the cloud, proper governance policies for public, private, and hybrid clouds are essential.
Governance should occur on two levels – organizational and solution. The organizational level manages the overall cloud optimization strategy, while the solution level manages each cloud service and service provider relationship. Sourcing or procurement departments oversee all cloud contracts and consolidate the cloud payment process at the solution level.
A tight feedback loop between application developers, cloud managers, and business leaders is crucial for successful governance efforts. Collaboration between cloud managers and app developers guides developers to the cloud platforms and services best suited for each application.
Although a central FinOps function can facilitate organizational change, effective FinOps management requires collaboration among disparate engineering, finance, and business teams. Individuals across all levels and departments of an organization can contribute to the FinOps practice, including executives, engineers, FinOps practitioners, operations, finance, and procurement.
The above diagram demonstrates how, for organizations operating on the FinOps model, a cross-functional team known as a Cloud Cost Center of Excellence (CCoE) interacts with the rest of the business to manage the cloud strategy, governance, and best practices that the rest of the organization can leverage to transform the business using the cloud.
To complete a sound cloud cost management and optimization plan, it is essential to establish regular communication and training programs. While cloud providers offer standardized services and basic metrics for monitoring, it is ultimately the responsibility of the cloud consumer to evaluate and communicate whether the services provided are meeting stakeholders’ expectations.
Final Thoughts for Leveraging FinOps
The shift to the cloud can be a complicated and ongoing process that requires constant reinvention to reap its cost-reducing and opportunity-creating benefits. The migration may involve multiple business processes, vendors, balance sheets, and regulatory compliance, with different stakeholders having varying expectations and motives. Therefore, a simple process can quickly turn into a complex web of conflicting goals, dependencies, and cost overruns. Leveraging FinOps during the transition can assist the transformation team in effectively disrupting the market without interrupting the business.
As businesses migrate to the cloud, they may experience increased costs due to duplication, such as paying for both cloud and legacy systems simultaneously, as well as data synchronization expenses. Cost planning, such as reserving instances at a discount, and implementing cost governance systems can help control expenses and avoid costly fixes resulting from rushed deployments. Businesses can use cloud value calculators to conduct cost-benefit analyses and optimize infrastructure, boost staff productivity, and increase business value.
Businesses need to determine the level of cloud expertise required, ensure that executive-level support is involved in decision-making, and implement continuous training to equip employees with the necessary skills to achieve success as the business expands.
Finally, it’s important to clarify the purpose of moving to the cloud, as unclear expectations for cloud services can add to the lack of cost control. Businesses may still believe that moving to the cloud will result in cost savings, but this is not necessarily always the case. Quite often, shifting workloads from data centers to the cloud can be a tool for innovation rather than solely for cost savings.
