Pay ¥2 to read

InfoQ Trends Report: DevOps and Cloud Computing (2019‑03‑16)

Key takeaways

1. Kubernetes has effectively won the container orchestration market and become the default cloud‑agnostic abstraction for compute. The next “hot” areas in this space are likely to be service meshes and developer experience / workflow tooling.
2. Chaos Engineering has reached the early‑adopter stage. “Resilience Engineering” was once lumped together with Chaos Engineering, but we now treat it as a separate topic in the innovator stage.
3. We are closely watching innovation around edge computing, edge ML inference, and “originless computing”.
4. The bookAccelerateby Nicole Forsgren, Jez Humble, and Gene Kim defines several themes around digital transformation and high‑performing teams; these are in the early‑adopter stage.
5. Although “AIOps” is arguably an overused buzzword, we still classify it in the innovator stage and keep tracking it—especially the value of machine‑learning‑driven operational insights and alerting.
6. We’ve realized that conflating Continuous Integration (CI) and Continuous Delivery (CD), and mixing practices with tools conceptually, is problematic. We believe CItoolsare in the late‑majority adoption phase, while CIbest practicesare still in the early‑majority phase. CD practices are also in the early‑majority phase.
7. We’re tracking the Software‑Defined Delivery (SDD) manifesto and “code‑vs‑configuration” approaches for delivery and pipeline automation.

This article mainly shares our view of DevOps and cloud computing—two areas focused on infrastructure patterns, the realization of those patterns in technology stacks, and the design processes and skills software engineers need.

In this DevOps and cloud computing trends report, we argue that Kubernetes has truly captured the container orchestration market and is now the default cloud‑agnostic compute abstraction. However, Kubernetes is not a complete Platform‑as‑a‑Service (PaaS), and most organizations still need a PaaS‑like layer to efficiently deploy and operate software. That’s why service meshes and developer‑experience / workflow tools are likely to be the next major focus areas.

Service meshes manage service‑to‑service communication and rollout control, while developer‑experience and workflow tools help engineers efficiently complete the “develop–test–deploy–observe” loop.

We see Chaos Engineering as being in the early‑adopter phase, thanks largely to the work of the Netflix team, the authors of the O’Reilly bookChaos Engineering, and the increasing number of tools such as Chaos Toolkit and Gremlin’s SaaS offering. Based on conversations with John Allspaw, Casey Rosenthal, Nora Jones, and others in the community, we have separated “Resilience Engineering” from Chaos Engineering and placed it in the innovator category.

We’re also watching edge computing, edge ML inference, and “originless computing” with great interest. Most of these technologies are still in the innovator phase, but it’s clear that public cloud providers and innovators like Cloudflare are investing heavily here.

Acceleratedefines several important themes around digital transformation and high‑performing teams. Based on our discussions with external contributors, we’re tracking topics such as measuring high performance, evidence‑based transformation, and transformational leadership. We consider these to be early‑adopter themes and hope to publish more coverage on InfoQ.

Although “AIOps” may be an overused buzzword, we still classify it in the innovator stage and keep paying attention—specifically to the potential value of ML‑based operational insights and alerting. As software systems become more complex and more distributed, these capabilities can complement full‑stack tracing, improve issue detection, and reduce the search space during incident triage.

We’ve found that conflating CI and CD, and mixing practices with tools, is problematic. We also acknowledge that our own articles and tagging around these topics have not always been ideal. In our view, CI tools (such as hosted CI services) are in the late‑majority phase, but CI best practices are still in the early‑majority phase. CD practices are likewise in the early‑majority phase.

We’re following the Software‑Defined Delivery (SDD) manifesto and “code‑vs‑configuration” approaches for delivery and pipeline automation. Teams at Atomist and Pivotal are driving much of the discussion and technology in this area.

One of the figures in the original report shows the Q1 2018 trends radar; the Q1 2019 radar is at the start of the article.

As one of the ten originators of the radar, I see Chaos Engineering moving into the early‑adopter stage, with Gremlin and AWS teams framing it as an extension or re‑imagining of traditional DR/BC practices. I also believe minimalist container images belong in the early‑majority phase, with DIY toolchains like LinuxKit in the innovator phase, distributed tracing in the early‑majority phase, general DevOps in the late‑majority phase, and containers, IaC, and continuous delivery also in the late‑majority phase.

We probably need to add service meshes to the radar, since they are a very hot topic and clearly in the early‑adopter phase. I’d like to hear everyone’s thoughts on whether we should move items around or add/remove topics.

Helen Beal (DevOps consultant, coach, trainer, speaker, and author)

My thoughts are:

– As you suggested, classify CI/CD as late‑majority.
– Add AIOps and ML for operations to the innovator stage.
– Should “DevOps toolchain” appear on the radar? Possibly in the early‑majority stage?
– Add Electric Cloud and XebiaLabs to the DevOps dashboard.
– Add “AI” to the dashboard?
– Classify ChatOps as early‑majority.

Steffen Opel (Managing Partner at Utoolity)

I agree that DevOps, containers, IaC, and CD are now in the late‑majority phase—organizations that ignore them will soon fall behind. Adding service meshes to the early‑adopter category is a good idea.

I also support adding AIOps as a topic in the innovator phase, although I view it as a broad superset of “ML for operational insight and alerting” that may eventually replace that narrower term. AIOps covers other areas as well, such as AWS EC2 predictive scaling.

While interest in ChatOps and full‑stack tracing is growing, I don’t think either has yet reached early‑majority status—at least not as the “default way of operating” for most organizations. ChatOps tools, in particular, still have a lot of maturing to do. On the other hand, “log aggregation and analysis” seems beyond the early‑majority threshold, especially as it builds on centralized log aggregation; perhaps we should merge those topics.

Finally, I’d classify “edge computing” as an innovator / early‑adopter topic (for example, services such as Lambda@Edge, AWS Snowball Edge, AWS Greengrass, and Cloudflare Workers). Once widely used public clouds like CloudFront, Lambda, and Cloudflare offer robust edge‑compute capabilities, adoption will accelerate quickly.

Chris Swan (Researcher, VP & CTO for Global Delivery at DXC Technology)

I agree with Daniel, Helen, and Steffen. What’s missing here is explicit coverage of “code vs. configuration” and the Software‑Defined Delivery manifesto in the CD space.

Regarding Steffen’s emphasis on edge computing: will “originless computing” really take off? It’s still very immature, and it’s hard to tell whether early demos are meaningful or just hype. But considering IoT and the demand for lower UI latency, I’m inclined to give it some credit.

One nit: people often conflate CI and CD because the same tools are used for both, but we know they’re not the same. CI is clearly late‑majority, since it’s relatively easy to implement even in traditional organizations. CD has not yet reached that level, because it usually requires reorganizing Dev and Ops into DevOps teams—a step many companies haven’t taken. On the innovation‑adoption curve, this creates a “chasm” that’s hard to cross, since CD adoption is about much more than tool choice.

Helen Beal

I think Chris and others have raised an important clarification.

Manuel Pais (Independent DevOps and Continuous Delivery Consultant)

On the CI/CD discussion, I worry that we’re mixing tools and practices, which are at different stages of adoption. I agree that CI/CD tools are entering the late‑majority phase, but only some practices (as defined in CI and CD books) are widely applied. Many organizations do not implement CI correctly (for example, Jenkins surveys show that the actual number of builds triggered is much lower than you would expect for “true CI”)—they’re simply using a CI server. Dave Farley’s recent criticism of long‑lived branches reinforces this point.

If we separate tools from practices, I’d put tools in the late‑majority phase and practices in the early‑majority phase. I’d also map the radar themes to practices fromAccelerate, so high‑performing teams can use that book as a reference framework.

As for other changes, my suggestions are:

– DevEx → early‑adopter (driven by Atomist and others as a shared concern)
– GitOps → early‑adopter (driven by Weave)
– Chaos Engineering → early‑adopter (driven by Gremlin)
– AIOps (formerly “ML for operations insight and alerting”) → stays as is
– Immutable infrastructure in the delivery pipeline → early‑adopter
– Log aggregation / analysis → early‑majority
– SRE / CRE → should be split; Google is still exploring CRE (innovator), while SRE is early‑majority
– Kubernetes → early‑majority
– Self‑service platforms → early‑majority
– Software‑defined networking → likely early‑majority
– Containers / general DevOps / Infrastructure‑as‑Code → late‑majority
– Centralized log aggregation → overlaps with “log aggregation”; we may want to consolidate these.

I’d also add a new topic in the innovator phase: software‑defined delivery, championed by Atomist and the SDD manifesto.

Daniel Bryant

Thanks for the great discussion. I’ll summarize this into a trends report. I’m particularly interested in the maturity gap between CI and CD, and in the differences between tools and practices at a more granular level; we should highlight this more clearly, as it directly affects our tagging and taxonomy.

Manuel’s suggestion to integrate key ideas fromAccelerateinto this trends radar is excellent. The book is rapidly becoming a key reference in the industry, much like other works by the same authors that Manuel recently recommended in InfoQ’s editorial reading list.

About the authors

Daniel Bryantis a technologist and organizational change agent. His current work includes helping enterprises adopt agile through better requirements and planning models, focusing on architecture issues related to agile development, and driving the adoption of continuous integration and continuous delivery. His technical interests include DevOps tooling, cloud/containers platforms, and microservices. He is a leader of the London Java Community (LJC), a contributor to several open‑source projects, a writer for InfoQ, DZone, Voxxed, and a regular speaker at international conferences such as QCon, JavaOne, and Devoxx.

Chris Swanis CTO for Global Delivery at DXC Technology. He was previously CTO of Global Infrastructure Services and General Manager for distributed computing at CSC. Before that he held CTO and director roles at Cohesive Networks, UBS, Capital SCF, and Credit Suisse, working across application servers, compute grids, security, mobile, cloud computing, networking, and containers.

Steffen Opelis a managing partner at Utoolity, a tooling vendor focused on cloud operations and software delivery workflows. He originally trained in C++ and specialized in rich‑client technologies, later transitioning to RESTful web‑service architectures. The industry’s shift toward cloud computing sparked his interest in full automation of the development process; he now focuses on DevOps adoption and enjoys API‑driven development in agile teams.

Helen Bealis a DevOps consultant, coach, trainer, speaker, and author. She helps organizations optimize the flow from ideas to value by improving behaviors, interactions, and technology. She likes camels and once even saw flamingos laying eggs. You can reach her athelen.beal@infoq.com, especially if you’d like her to write about something.

Manuel Paisis a DevOps and continuous delivery consultant focused on teams and workflow. He helps organizations implement automated continuous delivery and understand DevOps from both a technical and human perspective. He is co‑curator of DevOpsTopologies.com, editor for the DevOps section on InfoQ, co‑founder of DevOps Lisbon, and co‑author of the upcoming bookTeam Guide to Software Releasability. Twitter:@manupaisable.

Technical support

Technical support

扫码加入技术支持微信群

Scan the code to join the technical support WeChat group


Official account

Official account

Scan the code to follow and get the latest updates