In this blog post, we delve into the fascinating world of DevOps, exploring its current state, the operational transformations it’s undergoing, and the trends that are emerging. We’ll discuss everything from the rise of GitOps and platform engineering to the challenges of cost optimization and security in the DevOps space. If you want to remain competitive and relevant in the industry, take the time to read this post. You’ll find valuable insights into the evolution and future of DevOps.
I’d like to acknowledge Brad Maltz, Senior Director, and Nati Shalom, Fellow for Edge Solutions at Dell Technologies, for their insightful discussion and informative presentation in the webinar “How the Edge Breaks DevOps.” Their expertise and insights have greatly enriched the content of this blog post and provided valuable perspectives on the evolution and emerging trends in DevOps.
A. The Current State of DevOps
Let’s delve into the current state of DevOps. When we think of DevOps, we’re essentially considering an operating model mentality. DevOps has been around for a while, probably about 16 years or more, and it has matured over the years. We’re now at a point where IT operations need to become more agile as they work with their end users, be they application developers, application owners, business owners, or others.
The DevOps Spectrum
DevOps is about making IT operations agile while partnering with application development. In the world of DevOps, customers fall into a spectrum that we can break up into three categories.
1). Traditional IT Operations
The first category includes end users who are unlikely to adopt DevOps. For them, technology is not a business outcome that they prioritize. This is not common, but it happens. These users are content with their servers and storage and lean towards a more traditional way of thinking about IT operations.
2). The DevOps Elite
On the other end of the spectrum, we have a group that we refer to as the DevOps elite. These individuals and teams represent the pinnacle of DevOps adoption and expertise.
The DevOps elite are those who have taken the initiative to build their own cloud environments. This is no small feat. Building a cloud involves setting up and managing a complex network of servers, storage, and other resources, all interconnected and configured to provide scalable, on-demand computing power.
These teams are well-skilled and well-staffed, often comprising experts in various areas of IT, from system administration to software development. They are proficient in writing automation code, a critical skill in the world of DevOps. Automation code is what allows routine tasks to be performed automatically, reducing manual effort and the potential for errors.
But the DevOps elite don’t just stop at automation. They’re also building platforms - comprehensive environments that provide all the tools and services needed to develop, deploy, and manage applications. These platforms are essentially their own clouds, tailored to their specific needs and workflows.
Moreover, the DevOps elite are not confined to a single cloud environment. They have the skills and knowledge to operate across a multi-cloud space. This means they can manage and coordinate resources across multiple cloud environments, whether those are public clouds (like Amazon Web Services, Google Cloud, or Microsoft Azure), private clouds (clouds that are owned and operated by the organization itself), or a combination of both.
Being part of the DevOps elite requires a high level of expertise and a commitment to continuous learning and improvement. But for those who reach this level, the rewards - in terms of efficiency, agility, and scalability - can be significant.
3). The Middle Ground
Then there’s everybody else in the middle. This is where a lot of people fall. It ranges from what we call the DevOps beginner space all the way up through the DevOps scaling. People in this category are either starting to learn Kubernetes, playing with Ansible and Terraform, or maybe they did a little Puppet and Chef back in the day. They’re trying to understand how these tools and methodologies impact their business and IT operations.
Scaling DevOps and Building an Internal Platform
As individuals progress in their DevOps journey and move up the stack, they begin to recognize the importance and strategic value of the tools and methodologies they’re learning. However, understanding these tools is just the first step. The real challenge lies in scaling them to meet the needs of a growing organization.
Scaling DevOps tools and practices is not just about increasing their capacity, but also about integrating them seamlessly into the organization’s workflows. This requires a deep understanding of the organization’s needs and the ability to adapt and customize tools accordingly.
In addition to scaling, there’s also the challenge of building an internal platform. An internal platform can be thought of as a customized suite of tools and services tailored to meet the specific needs of an organization. Building such a platform requires a thorough understanding of the organization’s workflows, as well as the ability to select and integrate the right tools to support those workflows.
One approach to building an internal platform is to apply the principles of the “golden path.” The golden path is a set of best practices or guidelines that steer users towards the most effective and efficient use of the platform. By putting some guardrails in place, organizations can ensure that their end users are using the platform in a way that delivers well-thought-out outcomes.
However, building an internal platform and scaling DevOps practices are not one-time efforts. They require ongoing maintenance and refinement to ensure they continue to meet the evolving needs of the organization. This underscores the importance of continuous learning and improvement in the field of DevOps.
The Jack-of-all-Trades in DevOps
Another critical aspect to consider when thinking about DevOps is the diverse skill set required by professionals in this field. Often referred to as “jack-of-all-trades, master of many,” these individuals need to have a broad understanding of various areas in the tech landscape.
This includes everything from infrastructure - understanding the hardware and software components that make up an organization’s IT framework, to automation - the ability to use tools and technologies to automate tasks that were traditionally manual, reducing the risk of human error and increasing efficiency.
They also need to grasp orchestration, which involves coordinating automated tasks to create a consolidated process or workflow. Orchestration can help streamline complex processes and operations within an organization’s IT infrastructure.
In addition, they need to understand various levels of abstraction. In computing, abstraction involves managing the complexity of a system by breaking it down into smaller, more manageable parts. This can involve everything from abstracting the underlying hardware in a virtual machine, to abstracting the services in a microservices architecture.
Furthermore, they need to be familiar with developer pipelines and developer toolchains. A developer pipeline, also known as a CI/CD (Continuous Integration/Continuous Deployment) pipeline, is a set of automated processes that allow developers to compile, build, and deploy their code reliably and efficiently. A developer toolchain is the set of programming tools used to perform a specific function within the development process, such as coding, debugging, or testing.
Given the breadth and depth of knowledge required, it’s understandable why it’s sometimes challenging to find enough people with the right skill set to help build out an organization’s DevOps capabilities. This underscores the importance of continuous learning and upskilling in the tech industry, particularly in fields like DevOps.
B. Operational Transformation in DevOps: Navigating Multiple Shifts
As we define DevOps, we’re witnessing a multitude of shifts in the world around us that are driving operational transformations.
1). The Introduction and Success of Kubernetes
Firstly, the introduction and success of Kubernetes have started to match what virtualization has been doing for years. Virtualization technology has been a mainstay in the tech industry, allowing us to run multiple operating systems on a single hardware system and providing cost savings, improved server provisioning and deployment, increased IT productivity, efficiency, agility, and responsiveness.
However, Kubernetes, an open-source platform designed to automate deploying, scaling, and operating application containers, has gained significant traction in recent years. Its success lies in its ability to provide a platform for automating deployment, scaling, and operations of application containers across clusters of hosts. It works with a range of container tools and runs containers in a clustered environment to provide better infrastructure.
This means that if you’ve been working with virtualization for years in the industry, you’ll also need to pick up Kubernetes and container management over the next few years. You’ll have to manage them side by side, which presents a unique set of challenges and learning curves.
The challenge here is how to automate and operate more of that DevOps mentality, but across both abstractions and both platforms. This involves understanding how to leverage the automation capabilities of both virtual machines and containerized applications, and how to manage and orchestrate these different environments effectively. It requires a shift in mindset from managing individual servers to managing applications across a distributed network environment.
2). The Role of Public Clouds
Secondly, the rise of public clouds has presented us with a unique challenge. Public clouds like Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure have delivered a developer experience that many of our end users have become accustomed to. These platforms provide a range of services and tools that allow developers to build, deploy, and scale applications more efficiently and effectively.
This developer experience often includes features like on-demand access to compute resources, managed databases, analytics tools, machine learning services, and more. It also typically involves a user-friendly interface and comprehensive documentation that makes it easier for developers to understand and use these services.
However, this has led to a situation where a lot of our end users are starting to ask us in IT and DevOps how to deliver that same developer experience from within the enterprise. They’re looking for the same level of convenience, flexibility, and power that they get from public clouds, but within the context of their own organization’s infrastructure.
This is a challenge that many in the industry are grappling with. How do we replicate the developer experience of public clouds within our own infrastructure? How do we provide our developers with the tools and services they need to build, deploy, and scale applications effectively while also ensuring that we maintain control over our data and comply with our organization’s policies and regulations?
We’re aiming to address this challenge over the next few years. This will likely involve a combination of adopting new tools and technologies, rethinking our IT processes and workflows, and perhaps most importantly, fostering a culture of continuous learning and innovation within our organizations.
3). The Proliferation of the Term ‘Platform’
Additionally, the term ‘platform’ is really proliferating, and it’s being utilized excessively. So, we must figure out how to take all these different platforms at all these different levels and integrate them to make your own platform. That should really be your end state that you’re striving to achieve.
4). The Reality of Multi-Cloud
Lastly, multi-cloud is a reality. Multi-cloud by default is real. What we mean by that is people don’t always consciously end up in a multi-cloud space. They start with Amazon, they have some on-prem, somebody goes to Azure and spins that up, and you end up with two or three or four operating teams behind the scenes dealing across multi-cloud. And that’s not by design, which is actually a hard thing to deal with. So how do you start to get ahead of that, be more proactive in how you operate and manage across a multi-cloud space, including Edge?
5). The Emergence of Edge
The last part of this major shift is the fact that things like Edge are real now. We’ve been dealing with data centers and the public cloud forever at this point. But what’s happening is the introduction of the edge and some of the economic factors around us that are forcing people to rethink owning real estate is pushing them even into colocation conversations. So now you have colocation, public cloud, you have on-premises data centers, and you have the edge. And how do you lifecycle manage across all four of those in that more automated DevOps way? This is really what’s happening around us right now.
Making DevOps Successful
DevOps isn’t about the usage of specific tools, rather the smart leverage of tooling that allows you to automate and observe at scale. When you think about making DevOps successful, there are a few different things you need to think about. First off, you need to focus on configuration and lifecycle management. That’s the bread and butter of what the DevOps space has been driving towards: application and infrastructure level config in lifecycle management.
Secondly, as mentioned already, you need to think about what the platform means to you. How do you develop the platform for your business utilizing other platforms and technologies and tools out there? Now, to be able to get to that platform, you need to understand your use cases and requirements. How do you map out what your end users need, what the business needs, and define them in that use case mentality? And as you’re doing that, you should think about how you define the standards behind those use cases.
So, between the use cases, the standards, and the requirements, that feeds into your platform that you’re working on developing.
And then the other thing is, how are you going to handle all the complexity? To be successful, you need to abstract complexity away. This is around the developer experience. How do you deliver a much more seamless developer experience so that people don’t have to learn what I call the ‘nerd knobs’ under the covers? They really are more at the higher-level catalog experience. A lot of those are factors to help you drive DevOps to be way more successful.
C. DevOps Trends and Transformations
In the dynamic world of technology, the landscape is constantly evolving. As we look forward over the next few years, there are several major trends emerging in the DevOps space that are worth discussing.
1). GitOps
Firstly, there’s the concept of GitOps. Traditionally, we used to write scripts and have playbooks for lifecycle and configuration management. However, the introduction of GitOps has revolutionized this process. With GitOps, you can codify the versioning control of all your systems within Git. Other systems, such as Argo CD, monitor these Git repositories. As you make changes in the Git environment, tools like Argo help drive change on the other side to ensure things come up to the right versions. This is a very important technology concept within the industry that’s going to help people really scale up their platforms in the future.
2). The Emergence of Platform Engineering
Next up is the emergence of platform engineering. In the realm of DevOps, there are tons of titles: SREs, automation engineers, infrastructure automation folks, DevOps engineers, and many more. The newest one is platform engineer. This role is emerging from the notion that in DevOps, a lot of people focused on the pipeline, the CI/CD side, the developer tooling. What we’re really looking at here is how to bring the IT folks along on the journey. How do you take infrastructure components like storage, server, network, and operating systems and abstractions, and how do you build more of that platform around the Infrastructure as a Service (IaaS), Container as a Service (CaaS), and Platform as a Service (PaaS) mechanisms? That’s where platform engineering is really emerging, pulling the IT folks along for the ride.
3). Cost Optimization and the Rise of FinOps
On top of that is cost optimization. We’re seeing this huge surge in FinOps across public cloud, on-prem, and all over the place, including the edge. The notion is that as you lifecycle manage all your infrastructure assets and your applications across the deployment scenarios, you want to start to understand the cost. When deciding to deploy an app or an infrastructure resource to a certain deployment location, is that the best spot from a cost perspective based on my requirements? FinOps introduces the procurement cost side of the equation into the automation workflows to make better-informed decisions about workload placement in the future.
4). Security and the Alignment to DevSecOps
And the last thing is, of course, security. We can’t forget security. But the alignment to DevSecOps is not really a fad or a trend. It’s something that must be pervasive in everything we do, and you’re going to see security getting embedded way more going forward. From the concepts around zero-touch provisioning through zero trust, all these things start to pull in the security angle of the discussion.
Wrapping Up
The world of DevOps is dynamic and ever evolving. As we navigate through the shifts and transformations, it’s clear that the future of DevOps holds exciting possibilities. From the adoption of GitOps to the emergence of platform engineering, and from the surge in FinOps to the increasing focus on security, these trends are shaping the way we approach IT operations. As we continue this journey, it’s crucial to stay agile, keep learning, and embrace change.