Reblogged from Algorithms in Real Life:

Reblogged from Algorithms in Real Life:

• 

Many interesting meta-heuristics have been bio-inspired. Some of them are:

• Ant colony Optimization
• Cuckoo Search
• Genetic Algorithm

Ant Colony Optimization

This technique has been adapted from the studies carried out on the behavior of ants. It was observed that foraging ants quickly traversed the shortest path from the anthill to the food source and back. This was initially proposed by Marco Dorigo in 1992 PhD thesis.

Read more… 436 more words

Reblogged from Algorithms in Real Life:

• “Parkinson’s Law” does not set in, that is work does not expand to fill the available time
• “Understanding gap” between users and developers gets ironed out quickly
• “Quick response to change” becomes possible

However, there are other factors which may make too short a sprint cycle counter productive.

Factors influencing sprint duration

There is a certain amount of overhead associated with each sprint. Every sprint requires some initial planning time plus validation, regression testing and integration time and effort at the end. This overhead is proportional to the total size of the software and the level of automation in place and does not depend on the number of stories delivered during the sprint. In short…

Do not attempt short sprint cycle without a mature engineering practice.

Otherwise, it would lead the team to burn midnight oil and lead to frustration.

If the product owner is always available for the team…

If all the stories are clearly spelled out…

If there is no confusion or disagreement on how the user stories are to be implemented…

…then short sprints are fine. Otherwise, stories cannot be delivered in a short sprint.

Do not attempt short sprint cycle without clear user stories.

But longer sprint may not be a solution because it will only introduce slack and waiting time. So, better idea would be to create a buffer of clearly defined user stories. Applying Kanban principles may help.

Third key point in deciding the sprint duration is the amount of time and effort required to implement a story. Several factors may influence that:

• Architectural complexity
• Existing technical debt
• Skill level of the team

So …

Do not attempt short sprint cycle if, for whatever reason, the team cannot implement stories fast enough.

In short

If you need to have short sprint cycles, do the following:

1. Improve your engineering practice to bring down the sprint overhead
2. Ensure availability of clearly defined user stories
3. Simplify your architecture and reduce technical debt
4. Improve the skill level of the team

Udayan Banerjee on Google+

Related Articles

• A Rule of Thumb: Ideal Sprint Length
• Determining Sprint Length
• Going the Agile way…what is the ideal length of a Sprint?
• Agile Scrum Sprint Length: What’s Right for You?
• Choosing a sprint length
• Ideal Iteration Length, Sprint Length, How Long should be a Sprint or Iteration

An agile puritan would have a simple answer – Don’t.

There is an underlying assumption behind this statement. The assumption is that the product owner can provide an instant clarification and decision to any of the following types of questions:

1. About the domain of the problem
2. About the current process and workflow
3. About how the software is expected to work
4. About the business logic that needs to be implemented
5. About the usability of the software

In real life is it possible for the product owner to provide immediate answer to all these types of question?

For questions of the type (1) and (2) it is possible. For other types of question it is possible only for small projects.

Enterprise Scale Real Life Problem

To be able to answer the types of question (3) and (4) the product owner need to have a complete mental picture of how the proposed software should work, have a complete picture of the proposed workflow and have a thorough understanding of the business process and logic.

Question type (5) cannot be answered without a formal usability test. In fact you may need a usability expert to provide that advice.

In short, many questions of the type (3), (4) and (5) requires deliberation, required consultation with other subject mater experts and with specialists.

Once you reconcile to this fact then you will realize that remote product owner may not be that much of productivity killer.

It can be done!

The good news is that people have tried and successfully done it with little bit (but not too much) of productivity loss.

1. If there is no overlapping working hour then adjust the working hour to have an overlap.
2. Have a proxy product owner who is in the same location as the development team.
3. Use communication and collaboration tool to resolve issues.

In the world of ubiquitous communication and collaboration, it is naive to think that having a remote product owner in not feasible.

Udayan Banerjee on Google+

Related Articles

Why it is a bad idea:

• Scrum vs. The Time Zone

How it can be made to work:

• Agile Offshore: Create Effective and Decisive Product Owner Proxy to Offshore Team
• Seven Strategies for Handling Distributed Agile
• US/India time zone difference and daily scrum

Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations.

Or more concisely…

Any piece of software reflects the organizational structure that produced it.

For example, if you have four groups working on a compiler, you’ll get a 4-pass compiler!

This became known as the Conway’s Law.

Implication of Conway’s Law on Evolving Architecture

Let us assume that there is some truth in Conway’s Law. Then what implication does have on large agile projects?

In agile project you do not do a “Big Up Front” architecture. You do it in smaller increments. You take the architectural decisions at the last responsible moment. So, the architecture of your software evolves parallel to the stories getting delivered.

Small agile project can have a flat structure, a single team and no hierarchy.

For larger projects you will necessarily have to create multiple teams. You will need to partition the problem and assign a slice of the problem to a team. There are different ways to partition the problem like – feature wise, component wise, technology wise, tier wise etc. But there is no escaping the fact that you need to partition your problem.

If we accept Conway’s Law then your architecture would reflect how you have partitioned the problem to assign the work to the team.

So, here is the question that you need to answer – in which order should you do the following:

1. Split the problem
2. Form the teams
3. Arrive at the high level architecture

How about…

…doing enough “Up Front Design” to be able to arrive at a high level architecture so that the problem can be split in such a in way that each slice can be assigned to a team?

Research study that support Conway’s Law

Harvard Business School: Exploring the Duality between Product and Organizational Architectures: A Test of the “Mirroring” Hypothesis

…products tend to “mirror” the architectures of the organizations in which they are developed … we find strong evidence to support the mirroring hypothesis…

Microsoft Research: The Influence of Organizational Structure On Software Quality: An Empirical Case Study

Well … this research is strictly not a validation of Conway’s Law but it remonstrates link between organization structure and software quality!

Udayan Banerjee on Google+

More about Conway and his law:

• Mel Conway’s Home Page
• Melvin Conway – Wikipedia page
• Conway’s Law, Fred Brooks and Agility
• Conway’s Law

BUT…

…is it really that simple?

When you have a large software project it is too simplistic to assume that all you need to do is to decompose the problem to individual manageable chunk which can be handled by one scrum team and then continuously keep integrating the code produced by the teams. Keep doing this and your software would be ready.

4 Assumptions you make while splitting a problem

In agile we call each chunk a user story. There is an underlying belief that it is possible to take one story at a time and code it into the application. The design of the application would emerge and the architecture of the application would evolve. But for that to happen the following assumptions must be valid:

Granularity of the decomposition: Each story is not too large – one person code it within a couple of weeks.

Loose coupling: Each story in not too interconnected with the rest of the already developed application – the code can be tested without too much complication.

Ability to iterate: Every story can be plugged in easily – it does not necessitate too much change in the existing code base – it does not create too many regression error.

Minimum emergent behavior: The impact of adding a story is predictable – when a new story is added then there is no unpredictable change in the behavior of the application.

There is no guarantee that this can be achieved for every problem. There is no scientific method which can be applied.

However, experience teaches us that a good architecture can enable this process.

The best way to arrive at a good architecture is to find an expert who has solved a similar problem and has learned the hard way what not to do!

Such a person can make the problem as simple as possible but not simpler.

What if you cannot find such a person?

Then you will need to go through architectural cycles to arrive at the right architecture before user stories can be taken up.

You also need to keep in mind that any architectural decision once taken cannot be changed easily. It would involve too much rework. So, you should defer some of the architectural decision to the last responsible time.

Udayan Banerjee on Google+

Related Articles:

Simplicity: A New Model

Simplicity Is Highly Overrated

Do you think in terms of:

• Programming construct
• Database design
• Abstract algorithm
• User interaction

Looking back

Way we think about writing a program has undergone several phases of evolution. The evolution has a close link with increasing complexity and technological progress.

Flow chart era

There was a time when it was considered a very bad practice to start writing code without drawing a flow chart.

Those were the days when the concept of interactive debugging did not exist. You would submit you deck of cards to the mainframe data center and wait for the result to come. One compilation means another long wait.

Structured programming era

Then came a revolt against GO TO.

People stopped using flowchart and started using structure chart.

DB design era

Since the time RDBMS became popular most programmers used to thing in terms of database design. They believed that if you get the database design correct then the rest of the application would fall in place. They thought in terms of Entity-Relationship diagram and Database Normalization.

The ability to write an optimized SQL statement was a prized quality for a programmer!

Event Driven programming era

Once GUI became popular, structured thinking could not be used to solve the new class of program required.

That is when event driven programming and thinking in terms of even became popular. A new classes of tools and programming languages evolved. They were clubbed under the title of 4GL or 4th generation languages.

The event thinking and DB thinking coexisted. That was the Client-Server era. The debate was about what code should go into the DB stored procedure and what code should go into the event.

Object Oriented programming era

Then came Web. It needed HTML and multi-tiered application development. Event driven programming could not address this challenge.

As a result O-O thinking became popular. New classes of programming languages emerged. Design patterns became popular.

O-O thinking was not very compatible with DB thinking, so the O-R or object relationship mapping tools became popular.

DB thinking started loosing its preeminent position. DB thinkers lamented that the new generation of programmers could not even write a simple SQL.

Service Oriented programming era

When the systems became interlinked it became too complex to design using only the O-O paradigm. That is when SOA or service oriented architecture emerged. It was more to supplement the O-O thinking than to replace it.

However, the business consultants and the product vendors wanted to hijack the SOA term to mean something much more than a way of building a multi-machine application. Fortunately, that was not a success and the S-O programming paradigm and the O-O paradigm learnt to amicably coexist.

The Way Forward

Again, we are at a crossroad. Desktop and laptop are no longer the only means to access the web. Keyboard and mouse are no longer the only input devices. Number of channel through which we interact with each other and with organizations is exploding.

In short, the technology scenario and complexity of interaction is changing and changing fast.

As we have seen in the past, any such change have invariably impacted how we think about programming.

Looks like the future is going to be SMAC.

S = Social Media = User interaction design and Algorithm

M = Mobile Computing  = User interaction design

A = Analytics and Big Data = Algorithm

C = Cloud Computing = Parallel Programming = (?) Functional Programming

So, are we at the inflection point where we need to change how we go about writing program?

Are you ready to think beyond O-O design patterns? Beyond services?

Would you be comfortable thinking about user interaction design? About abstract algorithms?

Can you afford not to?

Udayan Banerjee on Google+

That is agile.

What happens if you cannot find sufficient number of master programmers?

Get as many master programmers as you can. For the rest of the team get programmers who aspire to become a master programmer and has a good chance of becoming one. Get them to form a self organizing team and do the development in short increments.

At the end of the development cycle the aspiring master programmers would have taken steps towards becoming master programmers. Few of the may indeed have become master programmers.

That is also agile.

But…

What if you cannot find enough programmers who aspire to and capable of become master programmers?

What choices do you have?

Somehow attract the right people. Pay more … create the right atmosphere … make the environment challenging … build a learning culture … solve interesting problem …

Wait till you assemble the right team.

What if your business imperative does not provide you with the luxury of time?

What if you organization culture, structure, dynamics is not conducive for creating such teams?

Should you reinvent your organization?

Do you think it would be easy? Do you think it would be possible to undertake such a transition? Do you think every organization can do it successfully?

What if there aren’t enough people in the world who aspire to and capable of become master programmers?

What about the software development problem that is not good enough to get master programmers interested?

Should you take sub-optimal approach?

How about building your team with best people that you can assemble?

This team may require the guidance of a master, may require to be managed, may need a process to follow and they may need to be directed.

Majority of the team members may not aspire to become a master programmer. They may visualize their career to follow a different path.

Many organizations may have no choice but to choose such a path.

The question is…

…can they adopt agile?

or

…is agile reserved only for the elite team?

If they do adopt agile, would an agile expert turn around, put his nose up and say that is not agile?

Udayan Banerjee on Google+

Related Article

• Agile Software Is A Cop-Out; Here’s What’s Next
• Agile is a Cop-Out?
• Is it justified that companies such as Google, Amazon, Microsoft, Directi or Walmart offer salaries to a fresher which are equal to those offered to someone with 7-8 years of experience in companies such as TCS, Cognizant, Infosys, Accenture or Wipro? Does a fresher at Google really deliver more value as compared to a veteran in TCS? Is it not a waste of money?

If you have any doubt about it just look at the adoption rate … look at the study results published … or just try it yourself.

Offshoring also works!

Again, if you have any doubt, just look at the growth of offshoring … look at the success of the success of the India based IT companies … try to find a reasonably large company which has no offshore presence.

People or Process … which is more important?

If you are following agile methods then … obviously … people are more important. In fact it is clearly stated in the Agile manifesto. Of course, the manifesto does not say that process is not important but it emphasizes that people are more important than process and tool.

If you are into offshoring then the clear winner is process. Almost all large offshoring outfit have focused on process and have adopted SEI-CMMi model. On the same vein, CMMi does not say the people are not important, but the focus is clearly on processes and practices.

But, is this not contradictory?

Dreyfus Model of Skill Acquisition

One possible explanation is that both people and process are important. Successful agile implementation does not ignore process. Similarly, successful offshoring does not ignore people.

However, a better explanation can be found in the Dreyfus Model of Skill Acquisition.

If you search the net you will find tons of material explaining what the model is and I have included few of the at the end of this post. However, the simplest and the shortest explanation comes from Seth Godin:

The Dreyfus model of skill acquisition posits that there are five stages people go through:

1. Novice = wants to be given a manual, told what to do, with no decisions possible
2. Advanced beginner = needs a bit of freedom, but is unable to quickly describe a hierarchy of which parts are more important than others
3. Competent = wants the ability to make plans, create routines and choose among activities
4. Proficient = the more freedom you offer, the more you expect, the more you’ll get 
5. Expert = writes the manual, doesn’t follow it

Offshore Software Development, CMMi and Process Focus

When the offshoring started, the workforce comprised of Novices and Advanced Beginner with a handful of experts guiding them. The growth of offshoring was too fast to move the Advanced Beginners to become Competent in the required number.

Therefore, it was logical for the organizations to focus on gaining process expertise and the most logical choice was the SEI – Capability Maturity Model (CMMi). The focus was on “Conformance to Requirement”. It was necessary to have more people to do the same work but that could be absorbed through the huge gap of salary.

Every project needed to find at least on expert. That expert could guide the team and interact with the onsite people. Projects which could not find such experts failed badly.

The growth of the offshoring industry have validated this approach.

Agile Methods and People Focus

Agile methods became popular where enough number of Competent and Proficient software developers where available. Once a developer becomes Competent he is ready and willing to take ownership of his work.

“…you might see folks at this level typically described as “having initiative” and being “resourceful.” They tend to be in a leadership role in the team…even at this level, practitioners can’t apply agile methods the way we would like—there isn’t yet enough ability for reflection and self correction…” - Andrew Hunt

Agile methods were ideally suited for Proficient software developers. All you needed to do was to…

“…Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done…” – 5th principle behind agile manifesto!

It is no surprise that agile methods worked and worked very well.

The story of offshoring is not over yet!

Offshoring growth came down from the ridiculous  30-50% per year to more down to earth 10-15%. That allowed more time to have sufficient numbers of Competent and Proficient developers. There are around 2 million ITES professional in India and even if only a small percentage can be classified as Competent and Proficient developers, you still land up with a huge number!

No wonder the India software industry has started looking beyond “Conformance to Requirement”

What about the expert?

The expert observes the world but trusts his inner vision. He concerns himself with the depth and not the surface, with the fruit and not the flower.

When expert leads, the team is hardly aware that he exists. He is content to serve as an example and not to impose his will. He doesn’t talk, he acts. When this is done, the team says, “Amazing: we did it, all by ourselves!”

The expert shapes events as they come. He steps out of the way and lets the design speak for itself. His workings remain a mystery, he just shows the results.

This might sound familiar to you because this is an condensed and modified version of The Tao of the Software Architect.

Udayan Banerjee on Google+

Related Articles

• Pragmatic Thinking and Learning – Refactor Your “Wetware” (Chapter 2) - by Andrew Hunt (one of the 17 original signatories of agile manifesto)
• Dreyfus Model Of Skills Acquisition
• Getting smart about the hierarchy of smart - by Seth Godin