Keep on Rockin' in the Agile World

Like rock and roll, some say that Agile might be dead. Maybe this news is a bit exaggerated, but, the fact is, there's a growing roar in the industry about how Agile is doing, and no one will be indifferent to it.

Things got real when I participated in a talk with Kent Beck, one of the original Agile Manifesto authors, earlier this year. He stated that we got it all wrong; what we see being done is not at all what the group had in mind back in the day.

Moreover, Kent is not the only Manifesto author voicing this opinion. Dave Thomas has also done the same with Agile is Dead, Long Live Agility, and Andrew Hunt is moving on with a new ideology called GROWS. All this motion has made me think deeply. What is happening to Agile? And where are we going next?

We Loved the Smell of Agile in the Morning
As you may know, the Agile Manifesto was created in 2001 by a group of developers who wanted to improve the way software was made: “We are uncovering better ways of developing software by doing it and helping others do it.”
This software development philosophy was grounded in the formation of highly collaborative multidisciplinary teams that thrive on change, living off short end-user feedback loops. The ultimate goal? Maximizing value, thereby ensuring better customer satisfaction.

It was this spectacular early promise that made Agile the reference approach for product development. Agile spread like napalm-fueled fire, and everyone in the business followed to the point that 97 percent of organizations report they are practicing Agile methods these days.
During the past 17 years, we’ve witnessed Agile making its way into official institutions and organisms, pricey certifications, job titles, consultancy firm names, marketing materials, and I could go on (and on)! Everyone is Agile it seems. But are they really?

I Can't Get No Satisfaction
The fact of the matter is that we have twisted Agile’s arm so that it’s mostly about how to deliver software faster. Do you see the words value, user, or customer in this graphic?





(Source: 12th annual State of Agile Report)

Not only was the core message corrupted down the path of adoption, but also, too many of us are just doing it wrong and calling it Agile anyway. Unsurprisingly, this is generating a growing frustration in the market that Agile is failing to live up to its promise. It’s this Cargo Cult phenomenon that is generating the frustration.

Agile appears to be failing many companies out there because these companies only appear to be practicing Agile. Though their efforts in mimicking the rituals of Agile methods are impressive, no real understanding and organic adoption of its core principles and values ever took place.

“Fake It Until You Make It” Didn’t Come Through in the Clutch
It’s quite easy to understand if a company is practicing Cargo Cult Agile. If you can spot some of the patterns below, I can assure you that you have been hijacked by a cult.

#1: Practice Obsession Over Value Adoption
In their haste to adopt Agile, the vast majority of companies haven’t interpreted nor put its values and principles into context. The fast-paced, competitive arenas want to be given repeatable formulas on how to thrive; don’t you dare make them think! As a result, 56 percent of Agile adopters have gone full speed ahead with Scrum, because this flavor comes with a detailed guide and many certifications to get you started.

“Practicing Agile,” broadly, became about blindly abiding by a series of ceremonies and staffing specific roles. How often have you heard someone say, “Yeah, we do Agile. We work in sprints, write user stories, conduct stand-ups, and have a product owner and a scrum master.” And then, full stop. Nothing in these sentences speaks of value, end users, or shorter feedback loops.

So, the result is a scrum team that:
• Religiously holds daily stand-ups but holds back for the sake of job security or competitive pride—probably the customer or a manager who also attends. There is no organic adoption of “it’s ok to fail, as long as you fail fast.”
• Does retrospectives but either focuses on the problems a lot more than looking for agreeable solutions or won’t address the elephant in the room because its members do not see feedback as a gift to be used for improvement.
• Has the proper roles but they are so segregated that they’ve become silos. “The scrum master clears obstacles and dynamizes, the product owner owns the backlog and writes user stories, developers code what the user stories ask for.” The ultimate success of the product is no one’s actual job; loss of accountability is through the roof.

The day that developers don’t care about contributing to the product backlog and product owners are not promoters of engineering best practices is the day for you to realize that you’re certainly not practicing Agile!

#2: Organizational Resistance in the Enterprise
Successful Agile transformation requires a top-to-bottom adoption of its core values and practices. Comparing the hierarchy of a startup with that of an enterprise is an excellent proxy metric for inferring how challenging Agile can be at an enterprise scale.

Many organizations see Agile as this “IT thing,” so they send the IT guys to attend training, at times get help from an Agile coach, and wait for the ROI. This approach results in no shared understanding between IT and the business of what Agile entails. And that causes the inability to recognize that the Agile transformation requires a profound mindset shift from strategic planning all the way to deployment and production.

Practicing the initially intended form of Agile requires that:
• Failing is part of the process. This doesn’t play well with tight budget control and margin-centered ways of operating.
• You drop the carved in stone outcomes. Yes, you have an objective, but you should also be open to change in objectives or business goals, which might be a tremendous leap of faith that doesn’t find a place in larger initiatives with budgets approved based on clear business goals.
• Comparisons not be drawn from team velocity. Traditional PMOs can’t cope with this. For them, velocity is the number one reason to be doing Agile in the first place, so they believe it must be used to evaluate each team’s performance.
• Teams become autonomous and self-organized. Which, accordingly, makes management structures leaner. This tends to trigger a fear of survival at the management and executive layers in this new reality, and they perceive this apparent loss of control as a loss of power.

Given that executive sponsorship is among the most critical factors that contribute to a successful project, Agile is in a very tough position right out of the gate. “Organizational culture at odds with Agile values” and “general organization resistance to change” are being reported as the two top barriers to adopting Agile at scale.

#3: Legacy Code and Monolithic Architecture
Agile has gotten a pretty bad rep for not performing well in the context of larger and more complex products. Many companies that are running on legacy systems adopted Agile methods with the apparent expectation of speeding up development.

This so-called “promise of agility” reveals itself to be a myth: velocity drops over time, quality is low, and technical debt squashes all hopes and dreams when a team:
• Spends twice as much time on impact analysis than on developing the actual code to implement a user story. The team systematically gives in to pressure by cutting corners, leading to an increase of technical debt.
• Isn’t able to get the job done without involving several other teams. It keeps synching up, accommodating impacts, and continually getting stuck waiting for the stars to be aligned. This takes a toll on velocity.
• Deploys to production using heavy and clunky processes that span several hours—sometimes days—requiring intervention from others outside the team. This also takes a toll on velocity.
• Hasn’t been able to establish automated regression testing because the product is so complex that no one knows how it's supposed to behave anymore. Obscene amounts of defects then spill over to production.

Several organizations have attempted to force Agile methods on internal teams when their product architecture or technology stack is in no condition to support it. Agile adopters that live with older and more complex products need to put into perspective how to evolve — or even replace — their code base to enable agility at that scale.

Hit the Road, Jack
In 2015, the Standish Group issued its Chaos Report after having assessed 50,000 world projects of all sizes and complexity. While it confirms that Agile is unquestionably more successful than waterfall, it also shows that its adopters are not systematically successful in their projects.

While we’ve certainly been doing a great job at misusing Agile, many adopters are also acknowledging they’re still learning. This is finally shifting the mindset for measuring success in Agile projects to where it should have been from the start:
• Velocity continues to be the number one measure of an agile project’s success, but it has decreased from 67 percent in 2016 to 42 percent in 2017.
• Business value increased from 23 percent in 2016 to 43 percent in 2017.
• Customer and user satisfaction increased from 28 percent in 2016 to 46 percent in 2017.

As the adoption of core values seems to finally be taking place, it’s time to walk the walk and determine precisely how you will position your end users at the front and center of your Agile product development process. To do it, you should remember three things.

#1: Stop Seeing Your Agile Practice as Dogma
Context has a lot more to say about how you should be implementing Agile than anything you might be told in a training room. A crystalized Agile practice is not serving anyone well, so challenge assumptions and ask why (five times if you have to) to keep evolving it.

#2: Agile Thrives on Highly Engaged Business and Tech Teams
A team can only deliver a constant stream of value to end users if they work hand-in-hand with subject matter experts in daily operations. Looking higher up though, an organization that has several Agile teams on the ground must also establish the agility mindset into its layers of strategic planning and governance. When the field runs into paths that deliver more value than initially planned, portfolio planning must be able to cope and transversely propagate course adjustments quickly.

#3: Salvation Through Refactor and DevOps
From a technological perspective, Agile came well before its time. Existing IT companies were unprepared from an architecture and operations standpoint to adopt it successfully. The recent DevOps wave is the answer, so your company needs a proper DevOps initiative for an integral Agile transformation. However, even with DevOps, you are still fooling yourself if you’re not taking measures to modernize your code base.

Get Your Act Together
Real agility can only take place when an organizational culture releases itself from the illusion of control; it’s not all about delivering projects on their planned due date anymore. It’s a lot more about how to challenge assumptions cleverly to provide more value than anyone had ever imagined in the first place. This is agility at its very core: “inspect and adapt.”

And that’s why I’m pretty sure that Agile won’t just go away. Although it passed its peak of inflated expectations, as confirmed by Gartner’s 2017 Hype Cycle for Project and Portfolio Management, it’s still the most used development approach in the industry, and it’s actually good that we are past this peak.

With unreal expectations for it meeting their end, Agile will surely resurface as a better version of itself. It will probably be called something entirely different as the world spins on buzzwords, but the ideal, the original ideal of maximizing value through short user-feedback loops, will never go away, no matter what you call it this time around.  

Silvia Rocha is the engagement practice lead for OutSystems.