The goal of this post is to introduce Extreme Programming to the reader. After reading the reader should understand the general idea of Extreme Programming, its properties and its basic application.

A typical reader could be anyone who is interested in software development. The reader could be a programmer, team leader, scrum master, or manager. No technical knowledge is required.

Introduction

Extreme Programming is a software methodology that is aiming to improve software quality and flexibility when customer’s needs change. The methodology is based on short iterations, making activities that matter and a fast feedback loop. As Kent Beck writes in [XP]:

XP is a style of software development focusing on excellent application of programming techniques, clear communication, and teamwork.

Extreme Programming is based on:

• Practices that work and have a good impact on the product and bring values to a customer
• Short development cycles that provide faster feedback loops and having a result early
• Incremental planning that allows changing a plan as soon as possible
• Flexible scheduling which allows the focus to be on business needs
• Automated testing allows deploying the product at any time, and catching defects as early as possible
• Close cooperation between the business and the programmers
• Adapting to changes by incremental design and development, only critical part of the system are designed at a time

These properties may sound very promising and abstract. But Extreme Programming has concrete guidelines on how to implement these techniques. The techniques are described in Values, Principles and Practices.

Software development has typical risks like schedule slips, defect rate, staff turnover and Extreme Programming tries to handle the risks.

Schedule slips

Short iterations for the win. A team is focused on the most prioritized features that a customer requested, so the scope of slip is limited. If a team deploys every new piece of code to production (Continuous Delivery), the code could be integrated every day and the team could focus on unplanned business-critical features or issues.

Project canceled

Because of the prioritized feature list, the risk of implementing and delivering unimportant features is eliminated. The list could be reevaluated during the cycle if anything critical occurs.

System goes sour

Extreme Programming relies on automated tests. These tests are automatically run when a new piece of code should be integrated into the codebase. This keeps the quality of the code high and deployable at any time. The problems can’t be accumulated.

Defect rate

The defect rate is eliminated by the automated tests. And also by user stories written as tests by programmers and customers (testers).

Business misunderstood

The business people (at least that define the product) are the first-class members of the team. They continuously define the user stories and they closely cooperate with the team, so new requirements could be applied to the product very quickly.

Business changes

Thanks to short cycles the requirements don’t need to be changed within a cycle and a requirement could be easily replaced by another new one because the team hasn’t even started working on it yet.

Staff turnover

A staff turnover happens no matter what we do. Extreme Programming tries to eliminate it by allowing programmers to do estimates. So when a new member joins a team, a whole new team does an estimate. All members accept their responsibility for estimating and delivering the work. This eliminates the frustration of programmers doing impossible jobs.

A newcomer can quickly fetch the knowledge of the project by pair programming (this is one of the primary practices).

Values, Principles and Practices

Extreme Programming stands on Values, Principles and Practices. Kent Beck defines Values as “Values as the roots of the things we like and don’t like in a situation.” [XP]. Values are the things that we do, they have got purposes and we see them as valuable. It means that values have a good impact on software development, so we value them. For example, when a developer knows that a code has a bug or lack of tests it’s a failure of values, not technique.

“Practices are evidence of values.” says Kent Beck in [XP]. Practices are techniques that we do to prove values. Because we value communication we write documentation. The technique of writing documentation is a practice. But if we wrote very long documentation that no one reads, it would not be a good practice and it would show that we don’t value communication. Instead of writing a long document, we could create a simple compact diagram with some notes and probably more people would read it. Choosing the most efficient way of communication we prove that we value communication.

We said that values bring purpose to practices and practices bring accountability to values. The last keyword to define is principles. Principles bridge values and practices by guiding behavior. Principles are more abstract than values so they are general things we do. Doing practices mean applying principles in current situations with current conditions.

Values

Extreme Programming stands on following five values: communication, simplicity, feedback, courage, respect. Let’s describe them.

Communication

Communication is probably the most important value in any human activity and software development is no different. When problems arise, most often someone in a team already knows the solution. But the knowledge about the solution doesn’t go to the one who has the problem. Everybody in the team needs to know when and how to communicate these problems.

Sometimes one person notices unusual behavior of a system, but doesn’t know that unusual behavior could lead to bigger problems. These problems start to accumulate and appear again as much bigger. This kind of muted communication is a sign that the team has a communication problem in general.

Communication is a baseline for a team and without it the team couldn’t be effective. Though communication is important it’s not the only value that the team needs to be agile.

Simplicity

Make things as simple as possible. Everything should be simple. If a feature could be done in two or more ways, choose the simplest one. Indeed there could be some constraints, security or reliability. Choose the simplest way that satisfies the constraints. Simplicity is not related only to software products. Sharing knowledge, asking for help, communication, all of these should be simple.

Feedback

Designing a system upfront may sound as the most effective way to do it. Unfortunately, it’s not in general. At the beginning, there is a rough idea where to go and how to get there. Extreme Programming is about making the smallest step possible and revalidating the direction. Designing the system upfront sounds cheaper than incremental design, but it has few problems:

• A team doesn’t know how to solve a problem right. There could be a lack of technical or domain knowledge.
• Even if a team knows how to solve a problem, the solution could behave differently in the new context than it did in the context where the experience came from.
• A solution for today could be invalid for tomorrow or next week, due to quick changes in the market. Or the right solution could take so much time that it would be delivered too late because a market has already changed and the solution is invalid.

Courage

“Courage is effective action in the face of fear.” [XP]. Courage without accountability is dangerous. But together with other values, it is very powerful. The courage is to say no when anyone from a team feels that something is wrong. Or when someone tries to do a job quickly and skip or reduce some values.

Respect

All team members should care about each other and the whole team as a solid unit should care about the project. Extreme Programming is not a silver bullet, if a member doesn’t care, it will not work.

Others

All previous five values are defined by Extreme Programming, but a team could define other values if they like.

Principles

“Principles are domain-specific guidelines for life.” [XP]. Principles connect values together with practices to address the values. If communication is a value and humanity is a principle, it leads to talking with others directly overwriting a document. Indeed this depends on the context. In case of sharing the knowledge of an API, written documentation would be better compared to talking directly, because it could reach a wider audience at any time. Extreme Programming brings the following principles.

Humanity

The fact that software is being developed by humans is not surprising. People aren’t machines and they have their own feelings and needs. People need some basic standards to do their job such as: basic safety (physical and mental), accomplishment, belonging, growth and intimacy.

Separating private and professional life matters because a member’s personal problems should not impact a team’s morality. In the case of any personal problems, it’s better when a member finds someone who they trust and who can help.

Economics

Everything that a team does needs to have a business value. Someone has to pay for all of it. And the team needs to know it. Incremental design helps here a lot because all of the unimportant design decisions are moved to the future, so resources (money) are only invested in features that bring values.

“The time value of money says that a dollar today is worth more than a dollar tomorrow. Software development is more valuable when it earns money sooner and spends money later” [XP].

Mutual Benefit

All activities you do should be beneficial for all. For current team members and also for future members. Writing automated tests today that help to design and implement features today, will also help newcomers in the future to understand a system.

Self-Similarity

Sometimes copying a solution from a similar problem can help even at a different scale (e.g. estimating new tasks compared to tasks from previous sprints). However be careful, because copying a solution of a similar problem may not work in a different context.

Improvement

“Best is the enemy of good enough” [XP]. There is nothing perfect in software development. There is no perfect design, process, implementation, etc. Everything depends on a context and situation. The idea is to do the best today and be prepared for tomorrow. Trying to achieve the best solution today may you slow down tomorrow, or today’s solution could be outdated tomorrow.  Trying to be a perfectionist could bring a loss for a customer and useless work for a team.

Your design will never be perfect, but should still aim to be close enough.

Diversity

When putting a team together keep diversity on your mind.  diversity means different skill sets including technical and soft skills, industry fields, and experience. Conflict is a common thing in a team with diverse opinions, but a really good team ends up with consent. When a team comes up with two different solutions and can’t agree which one to choose, sometimes the best way is to choose to implement/try both solutions and pick the best of them.

Keep in mind that a team is supposed to work together. If you compose a diverse team for the sake of diversity and the team doesn’t work as a team, you composed an unfunctional team.

Reflection

A good team doesn’t just publish its triumphs, but also its failures. Analyzing and learning from mistakes differentiate a good team from a mediocre team. A retrospective should be part of the cycle routine. Generally, a retrospective is placed at the end of a sprint. Sometimes is led by a mediator, sometimes led just by the team itself. A retrospective could contain good and bad things that happened. A reason to discuss the bad things is not to point a finger and blame someone. But analyze reasons why and how the situation happens, what conditions allowed it to happen, and discuss what to do in the future to prevent the situation. A result from this discussion could be a changed development process, more communication, increased pair programming, add more end-to-end tests, building software to a specific platform, etc.

It’s usual that after critical defects in the production environment, a retrospective is done during a sprint. This retrospective is typically called Post-mortem and it’s not unusual that a resume is applied during the sprint.

Flow

The term flow in software development means delivering a continuous flow of values. It’s opposed to big bang delivery. The continuous flow could be delivered every week, day, or even right at the moment when a new piece of code is successfully integrated into the codebase. More often a team integrates the code, a possibility for failure is less. You can simply imagine hunting for a bug in code that is a day old or three months old. In which situation would it be simplest to find the bug?

Some teams integrate a new piece of code as soon as possible to the codebase. But they defer deployment for days or weeks, to be sure that there are no bugs. This process does not flow. It’s like turning tap water to the washbasin, waiting for a time, and pouring that water on flowers. A pile of loam may or may not absorb all the water at a time and some water could be spilled out. There are bugs that are hard to find without real (production) data, some of them are hidden in edge cases, and testers hardly find them at all. Deferring deployment just leads to amplifying the risk of bugs.

Opportunity

Problems are a part of software development. No matter how hard a team tries to avoid them, they will appear sooner or later. To deal with them is to see problems are opportunities to improve and learn. The team may think that solving problems more than is needed, to get by and continue with development, will slow the team down. But learning from these opportunities will improve the team in the long term.

Has a member of the team high defect rate or other troubles with delivering? A solution is not give a warning and press but practice more pair programming.

Redundancy

A redundancy is a classic property of software development. Every programmer is passionate about removing redundant code. Redundancy lives in code and also in processes. Two different codes may look the same, they can be seen as redundant, but some nuances differentiate them and these two unrelated pieces of code should stay untouched. DRY maybe next time.

The same rules could be applied to processes. Testing software after programmers may be considered as redundancy. But we want this type of redundancy. This type of redundancy is like another net for catching defects.

Failure

Don’t know how to implement a task? Do you have more ideas for a solution, but you don’t know which one would be the best/cheapest/efficient? Don’t waste a time on arguing and give it a try. After that choose the best one and deliver it to production. If implementing all of the ideas would take too long, choose a subset of them, or limit them as prototypes.

Quality

“Quality is not a control variable.” said in [XP] and that’s the truth. You can limit the scope of features, cut a user story to a bare minimum, but not cut quality. At the very beginning of a project, you may feel that lowering a quality brings faster delivery and feedback. But this is a just temporal illusion. Lowering quality, like omitting tests, brings just an unstable product with higher defects rate and unpredictable delivery. Imagine delivering a car with autonomous pilot features with lower testing, just to fulfill some time promises. A customer will not be happy with a deferred delivery but will be less happy with an unstable car.

quality also affects humans. In general, people don’t like to deliver a product of poor quality. A team will be much happier if they can do their best and be proud of the product. Morale will be higher and team members happier than when doing a poor job. Delivering a product with high quality affects the economy (costs), the customer and the team. 

Baby Steps

The temptation of doing big changes in big steps is in software development from the beginning. From the very first view it may look as a logical thing, we have a lot of stuff to do, but just a short time for it. If we cut a time for integration, we can save some time for implementing features. Unfortunately, this is silly, as we said in previous paragraphs a longer time to production (integration of code), the defect risk is bigger.

Let’s watch a baby learn to sit, stand up, and walk. Everything a baby does, it does it in small steps. The reason is obvious, faster feedback. When a team is implementing Extreme Programming it should do it in small steps. Jumping in a pool to learn to swim it may work, but there are better options. Not sure how to do pair programming? Just sit together for a few hours and see what happens. Don’t know how to implement continuous delivery? Just try to cut your delivery period to a half and see what happens. Don’t try to implement Extreme Programming as a big chunk. Because it may totally slow down the whole development and ends an absolute failure.

Accepted Responsibility

“Responsibility cannot be assigned; it can only be accepted.” said [XP]. Practices show that people are more responsible when they accept responsibility voluntarily. By allowing people to estimate, design, implement and test tasks shows they have courage to be responsible for delivery.

Practices

“Practices are situation dependent. If the situation changes, you choose different practices to meet those conditions. Your values do not have to change in order to adapt to a new situation. Some new principles may be called when you change domain.” [XP]. Practices are activities that an XP team does on a daily basis. Doing pair programming just to satisfy a checklist would demotivate a team. If a team doesn’t see value in pair programming it needs to go back and discuss it again. Pair programming is about communication, getting feedback sooner, spreading knowledge, catching errors, learning new things, etc.

Sit Together

A team needs both a common space where all common communication is done and also a private space. A common space could be an open space office, but with enough privacy that another team could not make so much noise that the team members will not be able to hear each other. The common space is typically used for planning, everyday stand-ups, retrospective meetings, etc. A private space is used e.g. for pair programming, every pair should have enough privacy that pairs should not disturb each other.

As Flexiana is a full-remote company, the situation gets a little bit tougher for us. Because we are not able to meet physically together, more communication is really important for us. In general, all teams have a kind of daily stand-ups where we use video calls. During a day we do pair programming, again with video or voice calls with sharing screens. If a member needs to share or get some piece of information, we use Zulip for it (on-line chat platform). We use Zuplip as asynchronous communication for time-limited purposes. If we have something that should remain for a long time, we put this information into a wiki or similar type of documentation (it depends on the team).

Whole Team

The old idea of a cross-functional team is still being used and it seems that it will never be outdated. A benefit of the cross-functional team is that a pair or a bunch of people can implement a whole feature as a subteam. So you can eliminate phrases like “We wait for the frontend/backend team to implement their part.” or “A fault of the frontend/backend team.” because they implement the feature as a unit.

Another crucial thing is the size of a team. In [XP] Kent Beck talks about number 12. Our experience in Flexiana around 6-7 members per team is enough. When we have more people on a project we split them into more teams, so we still oscillate around the number 6.

Informative Workspace

Your workspace should reflect what you do. “An interested observer should be able to walk into the team space and get a general idea of how the project is going in fifteen seconds.” [XP]. Use whiteboard, flipchart or magnetic desks for user stories, diagrams or other important information.

With remote work, the situation gets a bit complicated. Some colleagues use whiteboards as well, have printed diagrams on walls, or have shortcut access to the documentation from a computer desktop.

Energized Work

A well-rested body works better than tired one. And the same thing applies to the brain. Everyone needs to work overtime sometimes. Adding extra hours allows you to finish the task and the customer will be happy. Working overtime needs to be used very carefully. It works for a short time period, but it will shoot you in the foot in the long run. Sitting for hours? Take a walk for 10 minutes and free your mind. Your body and brain need to take a rest. Know your limits.

Pair Programming

Pair programming is when two people sitting at a table programming together with one keyboard. “Pair programming is a dialog between two people simultaneously programming (and analyzing and designing and testing) and trying to program better.” [XP]. A benefit of pairing is that when a person is stuck the other one can help and prevent frustration. Also, it is good for spreading knowledge. New member in a team? Let’s pair together and discover how a system works earlier. Does a person have a high defect rate? Let’s pair and help to lower it and improve programming skills. Pair programming also improves skill with code editors and development workflow.

With remote work, pair programming is a bit different. You cannot sit together at one table sharing the same keyboard. At Flexiana we do paring a lot, generally with video calls and sharing screens. One person can control the code editor and the second one just sees the editor. After some time (e.g. an hour) first-person commits current code to the VCS and they can flip roles.

Weekly Cycle

Once a week have a meeting together and plan a work for this week and review progress. A customer may also participate in this meeting discussing the chosen user stories. At the beginning, this meeting will take some time, as time passes a team learns how to communicate effectively and this meeting will be shorter.

Quarterly Cycle

A quarterly cycle meeting is about high-level goals, compared to the weekly cycle. During this planning, a team should identify bottlenecks, and focus on the bigger picture regarding the project, etc.

Slack

It’s good to include some minor tasks into a plan. In case a team slips (e.g. an underestimated task) these minor tasks could be dropped, so the team can focus on delivering the important tasks. On the other hand, when a team is done with important things, it could take extra tasks and deliver more than promised.

Ten-Minute Build

Automatically build a whole system, run all tests, and code linters should take up to 10 minutes. If a build takes longer than 10 minutes, it will not be run so often, or people could start to ignore the build. A build that takes less than 10 minutes, doesn’t give you enough time to take a rest and make a coffee or tea. If your build is not automatically started this is a good place to start improving your process. Automated builds are much more valuable than manual builds. People may forget or be lazy (everyone knows “this is just a small change”) to run the build.

Continuous Integration

Integrate changes as soon as possible. The longer the time between finishing the task and integrating the code to the code base, the more it would cost take to fix bugs or to solve code conflicts with others in the team.

At Flexiana we go much farther, if possible. We split tasks into small commits and integrate them in small chunks. This allows us to keep the code base fresh and be sure that our changes will produce no defects. These changes are being deployed directly to the staging environment (or testing, it depends on the naming conventions), and after that to the production environment. Another benefit is that this process reduces code conflicts because the new code isn’t rotten on the developer’s machine or in an open pull request (or merge request, it depends on the naming conventions). So it happens that sometimes the production environment contains unused code or even features, but it doesn’t hurt anything. This also allows us to stop work on a task when something critical appears, and switch back to it later. The code is already integrated into the code base, so no code conflicts or rotting code.

Test-First Programming

Writing a test before adding or altering functionality is a classic technique. Writing a test before the production code has many benefits:

• Forces the programmer to think about the implementation before writing it.
• Leads to cleaner and better design (High coupling, Low cohesion, Inversion of control, …), because a code is testable by design.
• As a side effect, it produces tests that would be included in the project’s tests.
• Increases a thrush between programmers in a team.
• Works perfectly with pair programming.

The tests created with test-first technique will probably be really small tests and you might have thousands. These small tests would be fast and independent, so you can run them easily and also asynchronously. You can find more about this technique in the Test Driven Development [TDD] book by Kent Beck.

Incremental Design

“Invest in the design of the system every day.” [XP]. The idea behind this is that you don’t know what you will need in the next cycles, so you only design the necessary part of a system for user stories you are going to implement in the following sprint. Or you may think that you will need to implement this feature after 10 sprints, but the market could change (and it will!) and you could burn time and money on something that nobody will ever use. This idea is opposite of what I’ve learned at university. The idea of up-front design is from the 1960s. People may think that continuous improvement of the design costs more compared to up-front design. Unfortunately, the up-front design can easily cost more, because of wrong design decisions made at the very beginning when we aren’t sure how the system should behave.

incremental design is also related to the domain knowledge of the team. Even when the team contains members that worked on projects e.g. in financial technology, their up-front decisions could be wrong, because every project is different. So the suggestion is to design only the necessary part of the system and incrementally design others when adding new features. 

Real Customer Involvement

Having the customer on board is always a good idea. Sure both customers and programmers must accept their roles and privacy for work. Some customers may be busy with their business, so they are represented by product owners. In that situation, a product owner is the customer for a team.

Sometimes programmers are afraid that if a customer sees how messy the code is, or their working process or environment, they would lose trust. The process and environment is reflected in the software product anyway, in other words messy processes hardly produce clean and shiny software products. But if you think about it the other way, if a customer sees how good your process and code is, it would increase their trust! So make your processes and code clean, don’t hide your mess. The mess will not disappear itself.

Conclusion

Implementing Extreme Programming is not an easy task. It consumes time. Don’t expect that your team will be extreme in a few days. Start with a few values, do practices, evaluate them, see how they work, update them. Do baby steps, add other values when you see that your team delivers good value. Apply incremental design not only to the code but also to the team.

Bibliography:

[TDD]: Test Driven Development: By Example, Addison-Wesley Signature Series, Kent Beck, 2002

[XP]: Extreme Programming Explained: Embrace Change, Addison-Wesley Professional, Cynthia Andres, Kent Beck, 2004