DevOps and agile for all: technology professionals need to lead the way in the post-Covid era ahead

DevOps and agile methodologies certainly have shifted the priorities and practices of technology teams. Now, as organizations prepare for the post-Covid boom in activity ahead, these efforts will need to be at the forefront of cultural change and engaging customers. DevOps and agile need to break out of the IT department and become everyone’s best practices. The role of IT professionals and managers in the year ahead will be to apply their learnings and educate and demonstrate the power of these philosophies to the rest of the enterprise.

There’s no question that DevOps and agile are everywhere now, at least within the IT world. As discussed on our previous post, 74% of organizations now report having an active DevOps program, up from 47% five years ago — and the Covid crisis of 2020 only put an exclamation point on these efforts. Agile is more widespread, with the most recent data showing 95% of organizations practice agile development methods. However, only 33% report that more than half of their teams are actually practicing agile principles. 

Agile practices may have been what pulled many enterprises through the Covid crisis, as the great corporate dispersal took place. “Just look at how quickly many could adapt to the new situation when the pandemic struck,” says says Johan Karlsson, senior consultant at Perforce. “It was mind-blowing how even very large enterprises could switch to being fully distributed in a very short time. That is one example where already implemented agile improvements became extra valuable.”

As we move toward into the next era, a lot of what goes forward with both DevOps and agile is about having the proper mindset. DevOps, for one, “is a movement that refuses to be defined,” says Mike Loukides, vice president of emerging tech content at O’Reilly Media.  “Many organizations are leveraging DevOps as a specific set of tools and pipelines for deploying applications. However, it’s still spotty in terms of organizations leveraging DevOps for cultural changes around the interaction between operations groups, developers and management. When it comes to providing direct collaboration between developers and operations, this is observed more in the breach.” 

Is DevOps delivering on its promises, then? At some companies focusing on getting DevOps right, the results have been impressive. “We have adopted multiple practices and, most importantly, the cultural mindset change which is at the core of DevOps,” says John Donoghue, director of development at Smart Communications. “This closer alignment of development and operations teams has resulted in less rework of features to meet operational needs, more frequent releases, better operational stability and improved quality.”  

The maturity of DevOps initiatives is a major factor. “On one end you have entire industries such as the game development industry that has been revolutionized by live operations, and DevOps has become a natural part of that quickly,” says Karlsson. “On the other side you have industries that are coming from more siloed approach that are just touching the surface of the possibilities — but they have started the journey.”  

“You can’t just take an agile certificate, read a book, watch a video and implement practices suggested there,” says Karlsson.  . “Management teams looking to implement large-scale agile frameworks run into this risk. The most real pain is typically not on team level — it is on system level across teams.”  

It’s the soft skills that can make or break DevOps and agile engagements. “You can have all the DevOps tooling, pipelines, and dashboards setup, but if teams start to revert to old ways of ad hoc changes, deployments without testing, the process and the methodology falls apart,” cautions Joe Clarke, distinguished customer experience engineer at Cisco: “Likewise, with agile, you need a culture of transparency with alignment to strategy where all work is recorded so that you have end-to-end traceability and accountability to the strategy.”

At Donoghue’s company, the biggest challenge with DevOps has been extending the benefits to the customer. This means “balancing support for customers with differing appetite for changes that impact them,” he relates. “Continuous delivery is a prime example: some customers welcome knowing they always have the latest updates, whereas others are concerned about the frequency of change. Ensuring all customers have high confidence in the quality of changes is a key part of continuing to move forward.”  

The customer also needs to be front and center of agile efforts as well. “Demoing to the customer” should be at the core of any agile initiative, says Loukides. “Agile is really all about getting to the right end point by making many small, mid-course corrections to your direction. So, if there’s one thing that’s missing, in most instances, it’s that commitment to the customer.”

Of course, customers come into the picture at all levels, and aren’t necessarily consumers with mobile phones. “We have found a DevOps approach is required to continue improving and extending the benefits into operations,” says Donoghue.  “As a supplier to very large enterprises, we haven’t found it practical for development teams to directly collaborate with end-users. We prefer to work with a customer representative whose vital role is to find appropriate collaborators within each customer, gather and consolidate their feedback, then balance their varying needs and priorities.”

Donoghue advises starting DevOps with “automated build, test and deployment. A micro-services approach helps encapsulate the scope of changes, allowing targeted releases of reduced size. These become faster to roll out with clear understanding and more easily controlled risks. Containerization improves this further by minimizing the difference between development, test and production environments, further increasing control and reducing risk.”   

Communication cuts through the rough patches that may be encountered with DevOps or agile efforts that go astray. “This is especially true with the reduced face time and direct collaboration these days,” Clarke says. “Don’t let problems fester and derail the efforts.” Good collaboration and communication tools help, he adds — “it’s so much easier to get things done where these tools fit your workflow versus having to do unnatural things to sync up with people.” At the same time, he advises keeping this tooling simple. “The more boxes in your architecture, the more fragile and fragmented things become, which leads to frustration and softness of process.” Research the right tools “for source control, CI/CD builds, testing, and deployments, and issue tracking, so that you have single tools for specific tasks  — and not a new tool every day or competing tools to do the same thing,” Clarke advises.

It’s important to note that simply announcing that you are creating a DevOps practice doesn’t accomplish anything. “Organizations creating DevOps teams or hiring DevOps engineers has almost completely missed the point,” says Loukides. “DevOps is about collaboration between existing teams, rather than creating new teams and new specialties. For the most part, the kind of collaboration between dev groups and ops groups that DevOps envisions has not taken place.”

The advent of AI and machine learning may be helping to shift this equation, Loukides continues. “There has been some progress on this, as people are now talking about MLOps — and DevOps is well-positioned to solve these problems. But MLOps at this point is still very immature. There’s a lot of tooling needed that we’re only starting to have, such as version control for data, version control for models, testing for applications that aren’t deterministic and deployment pipelines for models that can take days to train.” Simultaneously, “there’s not enough recognition that an AI application is radically different from the typical web/database application that DevOps grew up with,” Loukides adds

Progress is also slow on the agile front. “If you speak to people in software development, you’ll find that many of them are doing agile processes that are just renamed versions of their old processes,” says Loukides. “Agile has a lot of value, but 20 years after the Agile Manifesto, we’re still not delivering on the promises.”  While the manifesto emphasizes deep contact with the customer, “some engineering groups still avoid contact with the customer,” he says. “This means the single most important thing in the Agile Manifesto is the thing that most development teams are least likely to do.”

An enterprise view is needed. “One team may be really getting it achieving local success with the right toolsets and mindset,” says Karlsson. “However, without the right support from management and other teams to achieve global optimizations and cross-pollination of the successes, they may just be moving underlying issues from one area to another.” As with agile, he adds, “if an organization would tell me that they have reached the final station, they have reached the agile nirvana, then they missed the whole piece around continuous improvement and are not to be believed.”