Evolutionary Design, YAGNI & Agile Enabling Practices

Posted by - raja 2008/02/24 16:26:53

How to balance out the agile light weight tenets with the agile enabling practices

Most programmers practice it. Yet they don't realize its true impact. I am talking about evolutionary design.

I had always liked Martin Fowler's article on evolutionary design. I think he does nail the worry that was raised by many people about how Agile may be USED as an exoneration of either deplorable or non existent design practices. (The two hats picture above came from his blog) Especially the agile tenets of just enough documentation and YAGNI (You Ain't Gonna Need It). In the article, Fowler mentions enabling practices of Agile such as continuous testing, integration and refactoring and how these enabling practices, when used in conjunction with YAGNI ,would foster an extensible design.

Continuous integration and testing fall under the banner of what I would call "continuous * " best practices since anything needs to be continuous for ensuring a successful outcome. I had covered these a little bit with my post on "Project Automation" in this blog. In this post, we are going to talk about my experiences with a recent project and how it could have benefited from refactoring.

I find that in many projects that my team is engaged on, there is no dedicated time allocated to refactoring. This is probably the single most important reason for dooming the project and thence agile. Take, for instance, a recent engagement that my team was involved in. There was this customer, who shall remain nameless, that wanted to build this huge portal site. One of the chief components of this portal is an Auction site. The Auction site required some key components such as an auction engine, payment gateway integration, notification engine to report on the status of the auction, auditing for each step in the auction process, a customer credit module that tracks the customer's credit limit and adjusts it according to the auctions that he is participating in, adhoc queries etc. Of course, add to this the fact that we should develop an AJAX enabled web site with the basics of customer information such as login screens, CRUD screens for maintenance etc. And Oh by the way - we also need to make room for two such portals - one for B2B and one for B2C interactions. The requirements were listed down as functional features. The functional features were spaced over a period of time and allocated to individual agile iterations.

The first iteration used a nice abstracted application framework that jump started this entire process effectively. The team was trained and brought up to speed on the technology stack. Then the trouble started. The team barely had enough time to deliver some of the basic features in about four weeks when there started a torrent of requirement requests. The team was perpetually busy delivering functional features from release to release and never had the time or inclination to re-factor the code that was already written.

As a result, when an audit was conducted in just three months time, I found basic design flaws. Controllers did all the work without delegating anything to business layers. Key parts of the framework that the developers were supposed to be using, were ignored. The fact that so much bad code was written so fast took my breath away! This application was legacy by design! Isn't agile to blame for this debacle with its emphasis on YAGNI and the fact that enough time was not spent in designing the application?

Self Organized Teams

How come the teams were not trained on Test Driven Development for instance? Agile, presupposes self organized teams. These teams have to be equipped to be self organized due to another enabling Agile practice called "Pair Programming". If programmers sat together and disbursed knowledge about the framework and best practices, then we would not have had programmers who did not know better than to put business logic in a controller!

Evolution Perspective & Refactoring

The chief problem in the project was that the emphasis was placed on functional requirements rather than QoS kind of horizontal requirements and much less - architectural or "evolutionary" requirements. The evolution perspective (see here for a discussion) is an important consideration that determines the current system's ability to evolve. These evolutionary requirements would have been addressed has the team followed another agile recommendation - refactoring.

The project cannot be planned with iterations emphasizing only on functional requirements without giving room for refactoring. This lapse in iteration planning was, in my opinion, one of the most important reasons for the debacle.

YAGNI (You Aint Gonne Need It)

I also want to discuss on the agile aphorism about "Just enough requirements" a little bit more. Just enough requirements should mean that time is not dedicated for adding potential features that are not required today. This should not be taken to mean that we have to start from scratch on every new project. If I have implemented transactions consistently in a previous project, I want to use this knowledge in the current project as well. This includes using frameworks that have already implemented these features. Most horizontal requirements must be delegated to frameworks, that have already abstracted and implemented them in a generic fashion.

YAGNI implies non obsessiveness with forthcoming features that we have not been threshed out in depth. YAGNI will only work if the code is continuously refactored as we realize new requirements.

Continuous delivery, integration & testing

It goes without saying that Agile requires the team to continuously build the project. This implies full project automation and meaningful test cases written possibly following TDD principles. Do you have a mock framework in place? Do you make sure that the team has unit tests that give them confidence to release the code?

A few pre-requisites for agile in my opinion are:

  1. The developers are good and understand or can be brought to understand the underlying process (not just the design of the system). This means that they understand project automation, agility and also look for avenues to abstract functionality. SOLID principles must be known to them.

  2. The customer understands the complexity involved and gives time for "refactoring" which is the basis for evolutionary design.

  3. The project manager must know to perform iteration planning in an agile fashion. She determines what is good for the current iteration and what is not. Further, she earmarks the correct iteration for refactoring.

  4. Technical evangelists with the ability to do domain modeling exist in the project. These people are constantly looking for opportunities to refactor the existing code to comply to new requirements.

  5. I am a big fan of domain modeling and evangelizing an ubiquitous language for the project (See the book Domain Driven Design- Highly recommended) An agile project with an evolving domain model and language is almost sure to be a success.

In summary, Agile will only work if the adoption of the recommended minimal practices is supplemented by the adoption of the Agile enabling practices.