When you are new to Scrum, it’s easy to focus on the concept of finishing everything you have committed to within an iteration. When the process instructs that you write user stories of manageable size and deliver small increments of functionality, it’s unsatisfying when the work is not done by the Sprint Review or the next Planning session, right? That is how we define the Undone User Story in Agile.
Ideally, a team would finish every item on its sprint backlog every sprint. But, for a variety of reasons, that isn’t always the case. It is not uncommon for a scrum team to get to the end of the sprint and find that they have a story that has been worked on, but is not yet done. Perhaps the story appears to be about 80% done.
This is one of the most common questions discussed among the Agile Community – What should be done when a team doesn’t finish a User Story in a sprint? Why does this happen? How should you deal with the undone user story? What should become of such stories? What should be done with the product backlog item itself? Should be it split or should it be carried into the next sprint? Should the team receive any velocity “credit” for completing a portion of the story? And how do you prevent it from happening again? This article addresses all of these questions.
Why do Sprints end with Undone User Story?
As a Scrum Master, you’re bound to come across the situation where you reach the end of a sprint and there are still one or more user stories that were not completed. The team have failed to complete the user stories that they forecast to complete during the sprint planning session. The team velocity figure will be affected and thus affect future planning. There can be multiple reasons for the undone user story,
- A team member off sick for half the sprint.
- A single User Story is found to be far bigger than initially thought.
- Over-optimistic developers. Even within small sprints, some developers will over-commit themselves. Best addressed with a one-to-one session.
- Mini-Waterfall. The team start all of the User Stories within a Sprint and either finishes them late in the Sprint or, not at all. Best addressed by encouraging the team to do everything possible to complete single stories.
- Poor estimation. The team thought the story was a configuration change, but it was actually hardcoded in 17 places, requiring a lot of updates and extensive User Acceptance Testing before deployment. Best addressed by getting the team together in a Product Backlog Refinement session and, with the benefit of what they’ve learned in prior sprints, re-examine the estimates provided for existing User Stories.
- A Work Item became blocked: Sometimes blockages can be avoided and other times they cannot. Always include blocked items as part of your iteration retrospective so that you have an opportunity to improve your process and communication for the next iteration.
- Every team deals with this to some extent. Something comes along that needs more attention than the time we set aside for interrupts. It could be unexpected host failures, a Production-facing bug, or HR trainings that took longer than we thought.
- The engineers working on the story are not familiar with the code base. Maybe it’s a relatively simple story, but you have never worked on the product before and need to spend three days understanding how everything fits together.
- The team finished everything originally committed to in the iteration, so pulled in some additional stories that won’t get finished. This is especially true when the highest priorities in the backlog are large stories that need to come next.
All of these are legitimate reasons for work not being completed in a sprint and by no means exhaustive. In the case of unforeseen circumstances, there is nothing that you can do. Best advice is to know how you will deal with the eventuality and do so consistently. In the case of emerging trends, the ‘inspect and adapt’ cycle of Scrum is your friend. Take every opportunity to examine the circumstances and address the trend.
First things First | Update Product Backlog & Re-prioritize
When a product backlog item has undone work at the end of an agile sprint, it should first technically be put back onto the product backlog. Work never moves automatically from one sprint to the next. Each sprint begins with the product owner making a conscious, deliberate decision about what the team should work on. If there’s undone user story from the prior sprint, it’s very likely the product owner will want the team to finish that work in the new sprint. But, the product owner should still make an actual decision to do that. The Product Owner re-orders the backlog. Typically unfinished items end up at the top, but not always.
Undone means Undone | The Undone Perspective
“A Product Backlog item is either ‘Done’ or ‘Undone’. There’s no in-between.”
This model keeps things simple and keeps teams from gaming the numbers. The idea is to put back ‘complete’ story (do not resize the items to represent only the remaining undone work) back on the product backlog for re-prioritization. Do not include any effort spent on the undone user story in the velocity calculation of the current sprint. Velocity is a reflection of rate of completion, not of team’s effort expended. Sure, the team won’t get any velocity credits for the story in the current sprint because it’s ‘not done’, but it won’t screw up velocity either because velocity will be a true reflection of what the team actually delivered during the sprint. On average, over a number of sprints the velocity figures will average out in any case.
The lack of detail in the scrum guide should be enough indication on the approach to take. If the scrum guide doesn’t talk about splitting stories then it’s probably reasonable to assume you shouldn’t do it 🙂 Reporting only the done work helps teams assess and report their progress correctly, create realistic plans, work together more closely and develop discipline.
- Get a clear picture of progress: In agile, a story’s status can be “not started”, “in progress” or “done”. We only demo and track done stories. This clarifies progress and helps us detect problems earlier.
- Make realistic plans: Most agile plans are based on velocity, the average team throughput for the past Sprints. But velocity only makes sense if teams are able to deliver constantly a similar amount of work.
- Encourage Teamwork: When it only presents done items at the Sprint Review, a team grows stronger. Why? Because it needs to collaborate more in order to finish things on time.
- Build Discipline: The team becomes more disciplined when it presents only complete items at the Sprint Review. Agile reduces waste by eliminating status meetings, weekly reports or useless documents. But for a team to self-organize, it still needs some rules to guide its behavior.
The Technical Perspective of Undone User Story
According to the undone work perspective – if a story is undone during the sprint, everything should be reverted back to its original state so you can have a potentially shippable increment. This would include source code, scripts, documentation basically whatever was changed when working with the story. From an engineering perspective teams can consider approaches to development that will allow team to easily exclude undone code from the sprint’s delivered increment. Two common approaches are to use feature toggles in the application and using source control with a branch per story pattern. However, creating branches causes merge-hell and if the work is planned to be completed in the next sprint it’s a lot of waste to revert the work already checked in.
But something got Done | The Done perspective
“If we can take some work to done so it can be inspected, do it!”
If the Product Backlog Item is not done, then the team can’t claim it is, demonstrate it, etc. But work was done and we want to know how much work remains in the release. Split the User story. There are many cases where the team realizes that a PBI can be decomposed into even smaller chunks of done work. For example, let’s say that feature A can actually be broken down into chunks B, C and D. The team realizes it can deliver B and C to done, but not D. After talking with the Product Owner, they should return D to the Product Backlog (PB) and finish B and C. D then gets estimated and ordered along with everything else on the PB. If teams cannot break a story further into work that can be delivered as done and remaining undone user story, then it should go back on the Product Backlog wholesale. The team may choose to re-estimate due to the new knowledge they now have.
Why NOT to extract Done from the Undone User Story
Team Discipline: In agile software development the status of a feature is binary. Done or not done – Nothing in between. It’s okay to slice User Stories, for example by workflow steps, business rules or by data variation to make them fitting into a sprint; but the defined scope and quality needs to be finished completely. Do not let the team go away with resolving User Story A because it’s nearly done and at the same time adding a new Story “Finishing last bits of User Story A” to the backlog.
Agile Philosophy: Splitting the Undone User story into done & undone work hurts the agile philosophy, in some way it would be a delivery without business value for the customer. By breaking an undone user story into two, you’ve produced two stories that, in isolation, provide no business value. Taking an incomplete story and splitting it just to make it look like you have a stable velocity reduces transparency and hides the truth of what you can actually achieve from your product owner, your stakeholders and most importantly yourselves.
Team Self-evaluation: Splitting an undone User Story gives teams an “out” for not finishing properly and they get credit for effort not delivery. It stops them pushing for a goal. Not having a story finished and getting no credit for it also gives the team something to talk about during the retrospective.
Keep it Simple: Additionally sometimes there can be a confusion regarding how to split the story that was almost finished? Can you split it in such a way that the work that was “almost” finished meets the “done” definition? Can you do it in such a way that your product owner is absolutely sure that the completed part of the story is done and that their acceptance criteria are satisfied? Can you separate the business value delivered between the completed part and the incomplete part? I doubt it.
What about the Agile Velocity?
“One User story delivered has more value than a lot of User stories 90% complete”.
A team should only take credit for the work that is truly done. So, in case undone user story is rolling forward to be completed in the next agile sprint, do not take any velocity credit. The team instead earns all credit in the sprint in which the work is truly ‘done’. Working with average velocities anyway, this averages out and avoids the risk of overstating velocity.
But when the team splits the story and puts the remaining subset onto the product backlog to be done in the future, go ahead and take some amount of velocity credit. The team will need to do its best to estimate a fair number of points for the subset of work that was completed. And, even though it did not finish the entire original story, the team may give itself all the original points if it feels the story was larger than originally planned. Beware – be reluctant to do that very often.
Retrospective | Looking for a Root Cause
Undone User Story isn’t the end of the world, and is pretty common in the execution of agile projects. But whenever work is unfinished at the end of an agile sprint, the team should take time in the retrospective to consider whether it was caused by something preventable. If it becomes a pattern then it could be a smell to be addressed. It’s always worth considering whether there is a root cause and whether something can be done to prevent it from affecting future sprints. Carry-over items can become a weight around a team’s neck that will eventually cause delays, or even project failure.
- If you discovered that one story is blocked, may be you want to do the advance work to avoid such blocking, in future.
- If you took on too much work, may be your stories are too large or poorly estimated.
- If you discovered more scope than when you estimated, perhaps the team should spend more time in backlog refinement meetings to get the stories ready with clear acceptance criteria.
From the Scrum Guide | Precaution is better than the cure
From the Scrum Guide, During the Sprint:
- No changes are made that would endanger the Sprint Goal;
- Quality goals do not decrease; and,
- Scope may be clarified and re-negotiated between the Product Owner and Development Team as more is learned.
“The Sprint backlog is a forecast, not a commitment”.
In the 2011 revision of the Scrum Guide Jeff Sutherland and Ken Schwaber made an important change. They changed the word “commitment” to “forecast” in regard to the Sprint backlog. The term commitment has two bad consequences:
- The stakeholders expect to have every single item delivered at the end of the Sprint, at any price. And, what is even worse, they begin making plans, assumptions and decisions based on this not yet confirmed fact.
- The development team focuses on delivering every single item of functionality which was promised at the beginning of the Sprint at any price… even at the expense of software quality or the real business value being delivered.
“If the Development Team determines it has too much or too little work, it may renegotiate the selected Product Backlog items with the Product Owner.”
Scrum does allow you to renegotiate the scope of the sprint backlog. It doesn’t make sense to start work if you already know you’re not going to finish it. But it puts a few rules on it. The scrum framework allows for these changes, but inherently dislikes them. The team should figure out a better way so that they’re not constantly having to renegotiate the scope. The most important thing is you prioritizing to deliver maximum business value to the customer.
What’s your Story of the Undone User Story?
Finishing a Sprint with undone User Story is a common event. How you deal with it depends on your interpretation of Scrum but best advice is to pick an approach and be consistent. The team may not like the fact that their work has ‘lost points’, but then that’s what non-delivery is – the team should be mature enough to account for their work and be held responsible. This must not be seen as blaming, but of being responsible.
One of the Scrum Master’s responsibilities for the team is to ensure Scrum is enacted well, and that the team is improving. Frequent undone user story could be an indication that the team needs help with their estimation technique, that a bottleneck in the flow of work is slowing them down, or that there is a persistent lack of understanding about the requested work. Addressing these concerns may reduce carryover, but more importantly, it will aid the team in frequently delivering valuable software.
Share your stories of when you’ve encountered undone user story in your project in the comments section below. We want to know how you’ve handled deferred stories in your project. Were you able to resolve it before it caused delay?