Software is not a Twinkie: Talking Enterprise Apps and Agile with Andy Hunt

At the end of the 1990s, Andy Hunt and his friend Dave Thomas were working as consultants, helping their clients write better software. At each new company, they found familiar problems.

“We’d get to an organization,” Andy told me, “and they’d be making the same mistakes that the last organization was making. We thought, ‘Well, okay, every time we go into a new client, we have to do this sort of song and dance and tell the cute little stories and the anecdotes and do little exercises to get folks up to speed with our way of thinking’. So, we figured we would write a little white paper.”

The Little White Paper that Could

That little white paper turned into the Pragmatic Programmer, a book that has helped to shape modern software development with its focus on craft, taking responsibility, learning through experimentation and finishing what you start. It even became a top-ten best-seller. “And it would have been number seven except some new author, some gal named J.K. Rowling, had two books in the top 10 at the time, which kind of pushed us down a bit.”
A year after the publication of The Pragmatic Programmer, Andy was one of 17 authors of the Agile Manifesto, perhaps the single document that has had most influence on how we organise software projects today.

The Birth of Agile

I asked Andy what led to the writing of the Agile Manifesto. “The seventeen of us, we all had similar experiences with occasionally disastrous projects. One of my favorites from that time was this project where they had an official architect. They had several teams working underneath him and it was a multiyear, very expensive project. Tens of millions of dollars easily.”

“The architect sat in this large conference room and spat out UML diagrams. They had this idea that all they’d have to do is make this big UML model and, bang, a product would shoot out the other side. And this fellow spent two and a half years papering his office with UML diagrams, and they never shipped a single line of code.
To me, that was one of the real low points of watching a project just sit there and burn through cash and never deliver any value whatsoever. Everyone had had experiences like that where the folks, for whatever reason, were stuck in some form of analysis paralysis or they never produced any code at all or what they did produce wasn’t useable, didn’t actually solve what the users needed because they were spitting out stuff with no feedback.”

Contracts Not Code

The Agile Manifesto’s twelve principles were a response to a software culture that Andy says was stuck in a mode of production that belong to another age.
The methods in place at the time were very ponderous. Lot of paperwork, a lot of documentation, because of this leftover 19th century notion that if the going gets tough, then write it down. Document everything, make it a legal document.”

“Section 1.2.3, sub-paragraph 5b says, ‘The dialogue box shall be pink,’ or whatever. As a developer, you just want to build something useful, and you’re sitting there working on a bloody legal document for two years, not churning out any code. And then the project gets cancelled. Everyone gets fired. You bring in new people, and you start all over again with the same frustration.”

There was a growing awareness in some parts of the software industry that the old ways weren’t working. Scrum and Extreme Programming were born of similar frustrations and they surfaced shortly before Agile. But it was Agile that took hold of people’s imaginations and both Scrum and XP are now thought of as Agile methodologies. Perhaps that’s because Agile was expressly about drawing the general principles from these new ways of thinking.

“The manifesto was born by hammering out some commonality from these 17 mostly very disparate folks with different ideas. And I think that part of why it still rings true today is because of that genericness, that idea that this kind of fits all the ideas that these people had been working with.”

Beyond Software

Agile’s general applicability allowed it to spread beyond software development. Andy and his co-author Dave Thomas founded the Pragmatic Bookshelf  publishing company in 2003 using Agile principles. “When we set it up, the idea was to make it look like a software project. So, all the authors write in an XML derivative or mark down. There’s a continuous build machine, continuous integration. Everything’s under version control.

“I was at a publisher conference early on talking to publishers in traditional workflows and they were just stunned because, with this simple approach, we could tell where every author was in their manuscript, how far along, when they worked on it last, how much progress they had made. If their laptop got hit by a meteor, it didn’t matter. We had a copy of everything in version control.
So we had all these astonishing benefits that they were agape at. And to us, it was like ‘Well, of course, that’s how you would do it,’ because we don’t know anything about publishing, but we do know how to have an Agile approach with a lot of feedback, and we knew how to run a software project.”

It has been a two-way street, with the broader Agile movement taking influence from other industries. In the case of Lean Software Development, that influence has come from the car industry and, in particular, the Toyota Production System.

Care About Your Craft

A common complaint about enterprise software is that it appears no one really cares about the finished product. Principle number one from The Pragmatic Programmer is “Care about your craft: why spend your life developing software unless you care about doing it well?” A common theme in Andy’s work has been the idea that well-crafted software is a function of small working groups.

Just Desserts
“I had a friend who worked at one of these giant food conglomerates. Regardless of what area you worked in, you could sign up as a dessert sampler. You’d go into the cafeteria, and on the first day they had a chef come in and handcraft this incredible dessert. Handmade, top ingredients, really just exquisite.
But as a condition of signing up for the test, you had to come back the next day and sample the mass-produced horror that was their attempt to recreate the chef’s recipe in an enterprise fashion. Now you have to use cheaper ingredients. You can’t use fancy, imported cane sugar. You have to use the cheapest corn syrup that fell off the back of the truck. You have to have shelf-stable ingredients. If you’re trying to mass-produce it, you’re gonna get a Twinkie.”

“So you’d go from eating this wonderful, handcrafted chef specialty the one day to this attempt to recreate it in the mass production environment the next day. And, you know, she told me that everyone came to dread that second day because it was so absolutely awful.
And it occurred to me the other day, that’s actually a really good metaphor for what happens in software development. If you are passionate about it and you view it as a craft and you put in the attention, get the feedback, you come up with something really excellent. If you’re trying to mass-produce it, you’re going to get junk.”

Scaling Agile … Scary
“There’s a lot of conversation around Agile saying, ‘Hey, this worked absolutely great for small teams. Now, we wanna scale it,’. The reason it worked great was because you had small teams that talked with each other and that were committed. Instead of having three or four really good expert developers and folks learning from them in a small team, now you wanna go to 10,000 novices that you just hired. Well, it’s not gonna work.
People will try to scale and, in scaling, lose sight of what it was that made the small thing good in the first place.”

Beyond Agile

For close to two decades, Agile has shaped how we think about software development. But it has largely been a story that stops at the door to the engineering room, “We’ve gotten a lot of buy-in at the developer level, which is fabulous. But now we need to go beyond that in the organization. Almost everyone I talk to they’re like, ‘This is great, but we need to get our executives on board.
Other partners in the organization don’t understand the principles of agility. They’re defaulting to that 19th century ‘I need a detailed plan for the next two years’ budget’ kind of thing.”

Andy, with his late colleague Jared Richardson, has been working on a successor to Agile that works for the whole organisation. Named GROWS, it focuses on experiments.

You Can’t Sprint Through an Obstacle Course
“Sprint is really an unfortunate choice of word. For a sprint, what do you do? You run as fast as you can till you’re out of breath and puking, and then you have to do it again and again and again. It’s a longer-term game than that. It’s a marathon. Again, it’s not really quite the right metaphor, because both a sprint and a marathon are in a straight line or known path, at least. There’s a route.”

“Development, these days, tends to be much more like an obstacle course. You don’t know what you’re gonna find. You don’t know that suddenly there’s this wall you’ve got to scale or this stream you’ve got to ford. All of a sudden, S3 is gone or left-pad is gone from the Node repo.”

Development GROWS in Small Bites
“So, the idea isn’t really that it’s a linear sprint. It’s an experiment. We’re trying this, and we’re gonna get data from what we’ve tried. And, hopefully, the data says that we’re on the right track, and then we do it again. Or the data will say, ‘we did this thing. We put it in front of the users, and they hated it.’ We call that a Heisen-requirement, right? It’s like the Heisenberg uncertainty principle. A Heisen-requirement is something everyone knows until you look at it. And then suddenly no one knows what it is anymore.”

“So, all this uncertainty all comes down to the idea of taking small bites, small batches, having continuous delivery, and thinking of it as an experiment. And an experiment doesn’t ever fail. You get data. The data might say that was a really stupid idea. Great. Now, we know that. And we were able to determine that relatively cheaply and inexpensively in the context of one experiment, one iteration during development, not in the context of 6, 12, 18 months’ delivery.”

Scale, Contracts and Time

For Andy, the problems of enterprise software are the problems that Agile solved for software engineers and that GROWS aims to solve for the whole organization.

Enterprise-shaped companies, and the bureaucracy they engender, are ill suited to software craft. Small teams working iteratively on manageable problems make better software. But scale is what large organizations know best and so they look to force that model onto how they make software.

Similarly, it’s impossible to specify a project in full up-front, whether it’s with UML diagrams or the contracts discussed by Paulo and Rodrigo of OutSystems. But even when companies claim to be agile, they’re hamstrung by requirements that were set months or years ago and technology that keeps them from moving as fast as they need to.

Matthew Revell is writing code for OutSystems and heads up Hoopy, the developer relations consultancy he founded. He also runs DevRelCon, the developer relations conference, the DevRel blog and DevRel Jobs.