Scaling Agile – Is it possible without violating Agile Manifesto?
In short the answer is a big NO.
Agile@Scale ≠ Agile@Manifesto
Why do I say that Agile@Manifesto cannot be scaled?
Apart from what is explicitly stated in the manifesto, there is an unstated belief that “any” software can be developed by a small group of highly talented & motivated individual as long as they are sitting within shouting distance, have continuous access to people who can decide what the software should do and they can work the way they deem fit. This belief is elaborated through 4 value statements and 12 principles behind it. These 16 statements can broadly be classified into 5 categories of “how to” or “preferred method of” …
- …Interaction and Collaboration (4)
- …Measure of Progress (4)
- …Responding to Change (2)
- …Forming Self-Organizing Team (3)
- …General Gyan (I mean advice) (3)
However, when complexity increases beyond a certain point, it becomes infeasible for a small collocated team to handle it. The reason can be many. For example, you may not find enough people like Dennis Nedry (Jurassic Park) “…who can network 8 connection machines and debug 2 million lines of code…”
Or, you software size may become much more than 2 million lines of code.
Or, you are not able to, for whatever reason, to assemble the whole team in one location.
Or, you are unable to find “business people” who know everything about what is needed and can take an instant decision.
I am sure you can think of many more reason of why enterprises have to go beyond one co-located cross-functional team.
When you scale, you will have multiple teams and they may not be located in the same place. So, let us examine the statements in each category and see if they can be applied in the context of a large and distributed enterprise engaged in software development and what their limitations are.
Interaction and Collaboration
- Individuals and interactions over processes and tools
- Customer collaboration over contract negotiation
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Business people and developers must work together daily throughout the project.
When you have teams distributed across multiple locations is it possible to function effectively without a clearly defined process (which everybody understands) and without having a tooling & technology infrastructure (which facilitates interaction among individuals across teams)?
When you have the involvement of multiple organizations, is it possible to start work without a clearly defined contract?
When you are targeting applications which potentially have thousands of users, will it be possible for few “business people” to define what the software should be? What about those pieces of software which is only interacts with another piece of software – what role will “business people” play in specifying them?
Is it always feasible to get the right “business people” to interact with the developer on a daily basis?
1) Tools, processes and contracts are the backbone of any distributed large scale software development project. Without them interaction and collaboration will collapse.
2) To say that you need only business people and developers to complete a software project is very simplistic. You need specific expertise from different fields like usability, architecting, performance testing etc.
Measure of Progress
- Working software over comprehensive documentation
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
- Working software is the primary measure of progress.
If “working” and “valuable” software means “potentially shippable” software, then can customer always take continuous delivery of such software?
In a complex project which has many components and tiers, is it always technically feasible to partition the application in such a way as to deliver “working” or “potentially shippable” software?
Can you take up large scale software development without comprehensive documentation of all the interfaces – interfaces between different software components – interfaces between the human and the system?
3) Slicing a complex software project into working, valuable software may not always be feasible. Attempting to split it into testable chunks is more practical.
4) Measuring progress only through working software is too limiting. There are so many other things that we need to keep track – Performance tuning, UI design, Prototyping, Code refactoring …
Responding to Change
- Responding to change over following a plan
- Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
In the real world where the delivery of the software is tied to many downstream activities – is it always feasible to change your plan?
Can you always makes changes to the software late in the development based on changed requirement and still guarantee the reliability of the software?
5) Any change that is required to be done on already developed software has potential side effect. Any decision to incorporate such change needs to carefully weigh the potential cost and benefit.
Forming Self-Organizing Team
- Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- The best architectures, requirements, and designs emerge from self-organizing teams.
- Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
Is team of motivated individual always sufficient to get the job done – doesn’t skill, experience and knowledge play a role?
Is there any guarantee that a group of motivated individual will always self-organize into an efficient and cohesive team?
Is there any hard evidence to support the statement that “the best architectures, requirements, and designs emerge from self-organizing teams” – especially when we are talking about many teams?
Isn’t “maintaining a constant pace indefinitely” a utopia – haven’t we heard of things like loss of interest, fatigue, boredom, attrition etc.?
6) It is naive to assume that any team of motivated individual will always self-organize. The team may require mentoring, periodic review of effectiveness as so on.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity–the art of maximizing the amount of work not done–is essential.
- At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
These are good advice and can be applied to anything.
So, if Agile@Scale ≠ Agile@Manifesto, what is it?
Since we don’t have any well accepted definition of agile to fall back on – we need to go by what the popular opinion is.
Everybody agrees that agile means “iterative development”, “progressive refinement of the requirement” and “emergence of the design”.
Most people want the teams to be “self-organizing” but some people are totally silent about it.
Many people feel that there is more to software development than just “working code” and we need to look at how the software development gels into the overall enterprise context. There are also suggestions that we should borrow idea from traditional software engineering techniques where appropriate.
As long as you…
- …Develop software iteratively,
- …Have multiple teams of small size and allow them to self-organize, and
- …Do not try to document the complete requirement and design upfront
…you can assume that you are following Agile Methodology.