Tuesday, October 13, 2009

SyXPAC, Agile and time to change

Last night at SyXPAC I gave a short talk (lightening talk) on the last 10 years of Agile. The point of the talk was to say that Agile has helped raise the issues to the forefront of peoples minds but it hasn't really solved anything,
In software development we have seen cycles of approximately 10 years. In 1968 was the first NATO Software Engineering Conference, one of the aims was to drive the adoption of the practically unknown term "software engineering". The title of software engineering was decided upon to be deliberately provocative and spark discussion. At the time it did this and from this methods were invented to attempt to create predictability in software development cycles.
In 1980 the first version of SSADM (Structured Systems Analysis and Design Method) was released and the 1980's saw a far more rigorous approach taken to software development. In 1982 Roger S. Pressman released the first edition of Software Engineering, A Practitioners Approach. This book become the bible for software development methods and teaching.
In 1991 James Martin first proposed RAD (Rapid Application Development) as an approach to delivering software using a more light weight and iterative process. Other great authors such as Steve McConnell and Jim McCarthy adopted these ideas in different degrees and help publicize this new approach. In 1999 eXtreme Programming was first publicized, along with Kent Beck's book on the topic. Following this the Agile Alliance was formed and a collection of other agile approaches have been proposed and introduced.
From 2001 forwards the term Agile was used as a buzzword to ensure that what ever changes were made could be signed off by the big boss. A number of Agile methodologies were pushed in to the market, SCRUM, Lean, MSF agile etc... some good, some bad. Mostly they all missed the point that XP was a set of dev tools not a methodology, philosophy or religion. In many ways the Agile alliance, although good in theory, didn't help with this.
Recently I have noticed a new issue with Agile practices and specifically some of the XP practices. There has evolved a breed of Agile coach that has a 'do what I say not what I do' approach. This is with a valid reason. Many of the practices that XP preaches are really like scales when learning music or mental exercises when training the brain. As a developer becomes more adept at them they become embedded into the psyche of the activity and the props become less necessary. To the point that some really experienced developer can move away from the practices and deliver even more amazing software without noticeably paying attention to the core practices.

As it approached the chasm, like all things, XP got watered down to become acceptable to the stakeholders that need to buy in. I don't think that this watering down of XP actually hurt that much initially, it enabled XP to get a foot in the door. It was then up to the developer to implement the solution. Here-in lies the issue. Fundamentally most developers do not have motivation to improve the state of the industry. They need to get their paycheck so they can eat, be clothed and have a roof for themselves and their families. The real passion for most developers is not business focussed. The real passion is technology for the sake of technology.
Yet through all of these changes, the software industry has not improved its reputation for delivery of poor quality, over budget, bug ridden software. Each shift has brought some new thinking and new tools to the tool box of the software development team, yet the basic principles remain the same.

Back to the Point; Agile helps but it hasnt really solved anything, it is time for a change and we embrace change, right ?


Anonymous said...

So what do we change to? Is there a path forward or are we waiting for the next big idea?

Garry Pilkington said...

What is the next thing on the horizon. I cannot see anything that will be the next 'big thing' to replace Agile, scrum etc. Any ideas?

Robert said...

What you're saying here meshes with my experience, Dr Neil. And to echo the other commenters: what now?

I don't think we should be looking for the "next big thing" (Brooks told us 30 years ago that There's No Silver Bullet and he's still right).

Is the answer to simply keep looking for interesting projects and smart people, work on those and bugger everyone else?

Dr. Neil said...

I don't have the answer of what is the next new thing, if I did I expect I would be telling you all about how wonderful it is.
I agree with Robert that we should always be looking for interesting projects and to work with smart people. I am not so sure about the last statement. The software industry is still very broken, to take an approach of _ignore_ everyone else is to _ignore_ the issue. What I have tried to do is educate, mentor and teach others using techniques I have found useful.

Robert said...

So when faced with those "most developers [who] do not have motivation to improve the state of the industry" with whom the introduction of Agile methods has not "stuck", we should try the next new thing on with them and see if that works?

That sounds like a Quixotic and ultimately demotivating quest.

Maybe it's a generational thing. When the students who are coming through university now and being taught about test-first, iterative development, pairing and close customer collaboration (they are being taught that, right?) are the majority of the industry, people who disagree with those ideas will weed themselves out of the pool and the problem will be solved.

vadi said...

I actually enjoyed reading through this posting.Many thanks.

Agile Training