Can User Stories Make Software Projects Late?
ฝัง
- เผยแพร่เมื่อ 14 มิ.ย. 2024
- Over my career I’ve seen more projects fail due to misuse of user stories than any other practice commonly used on agile teams.
You probably already know that user stories are just a simple format for writing requirements on scrum or agile teams.
Today I want to help you avoid some bad advice I see out there about what user stories are and how they can help (or hurt) you.
Why User Stories Were Created
Before agile methods for development, teams wrote big documents describing the design of a software product before building it.
Since the project was designed up front, it was also funded up front.
The more detailed the documents, the better the estimate of costs.
But companies were finding that by the time software was done, customers wanted something different.
Early leaders in the agile community came up with a great idea to only build and release a little bit of a software product at a time.
This would let the team change their mind about what to build once the project started.
Since it takes a long time to document something you might not build, the industry began using user stories to describe requirements.
The thought was to describe just enough about what value a feature offered to the customer that it could be prioritized.
The Pressure For Traditional Budgeting
But some leaders and business owners still wanted to know exactly what they were getting before approving budget.
They forced people to estimate and scope every idea anyway because they were too scared to fund a team without knowing exactly what they were getting.
Many agile coaches tell teams that a user story is just a placeholder for a conversation.
Once the team starts working on it, they’ll talk with the customer or Product Manager and get the full details.
The problem is, once this conversation was held, many additional details emerged.
And this caused the estimate to go way up.
With Non-Agile Leaders, You Need More Than User Stories
If your stakeholders (product managers, customers etc.) are focused on cost, you are much better off creating detailed acceptance criteria that describes a user story in detail before you give out any estimates.
Acceptance criteria reads like a test script and describes exactly how someone can use the software, or write an automated test, to verify it’s done and works right.
If you don’t have acceptance criteria, the rough estimate you give out for a user story will change.
And every time it does, you’ll lose trust with your stakeholders because you’ll look like you estimated wrong.
So match the level of detail in your design and requirements on your project with your stakeholder's tolerance for uncertainty.
If they are focused on costs, certainty, and predictable deadlines - writing only user stories and committing to estimates without acceptance criteria is foolish.
It’s like committing to a waterfall project with 10% of the documentation necessary to really estimate it.
Subscribe for more healthy software development videos!
bit.ly/2I0sXay
#programming #agile #scrum - วิทยาศาสตร์และเทคโนโลยี
Is your team giving out estimates with enough detail in your requirements? Leave me your thoughts and feedback below!
I think successful manager must be able to give with confidence plausible explanations why the estimates are not met and at other times be lucky to deliver so that he does not have to use his explanation skill too often.
That’s definitely a big part of it in our current system!
Think of the User Story as a Placeholder of a Feature Conversation.
What a brilliant definition 😍
I had started a relatively big project, where the budget and period of development were based only on "user stories".
After reaching the deadline (2 years period), client started to hate me, and we ended up in a bad relationship.
What actually happened there - client was not happy with the speed of development, regardless of me explaining the reasons of delay, which were the extra tiny-small feature changes they kept asking for.
I wish I had watched your video before that project!
Thanks for sharing.
You’re very welcome. These things I share I only learned from doing them wrong first, so be kind to yourself. We’re all learning!
@@HealthyDev Appreciate your reply) By the way, love your musical instruments!
People wanted me to write automated tests from the sprint planning meeting. I tried to explain that the stories are not discussed in nearly enough detail to do that. I have to wait until the developer actually starts to implement it to know how to test it. The higher you go in the management stack, the less interested in the details people will be.
Excellent job articulating that to them. It’s funny how people don’t want to be “bothered with too much detail” - but engineering requires detail!!
This channel is a gem, really clear and useful explanations. Thanks a bunch.
I find the Agile manifesto in and of itself to be sound. Its the "how to"'s that are currently being sold that I often disagree with.
Definitely. Which is why it’s so frustrating when I hear of people “abandoning agile” when they never were in the first place!
That's because the manifesto is open for interpretation which is a flaw in its self
@@brianlaudrupchannel The manifesto is plain and simple. It states what is desirable without dictating how every software development team or project ought to be managed, and rightly so.
The issue with use stories is that they are taken as gospel rather than a placeholder for a discussion. End result is that the deliverable is not what was needed.
Another great, insightful video. My experience is similar to yours, but also different in some key ways. Let me explain in the context of my second-to-last work experience.
My director was militant about writing vertically-sliced user stories that followed the INVEST criteria. He did want the scrum teams to consistently meet their sprint commitments, and he did believe in estimates, but at least he understood the limitations of estimates. Estimates were really only used in aggregate against the total size of the project ("Okay, we have 596 story points left to finish, and our average scrum team velocity is 20 points per sprint, times 4 scrum teams, that's 80 points per sprint, so we ought to be able to get done the remaining work in 7 or 8 sprints"). Estimates were refined on a just-in-time basis by doing a task breakdown. Teams would typically do their task breakdowns when they did their planning for the current sprint.
So the motivation to organize the work into user stories wasn't so much to estimate them perfectly, but it was to be able to deliver completed, working, valuable functionality in a chronological manner. That's why they were supposed to be vertical slices. It did make sense about 80% of the time.
The problem was what did you do when you got to cross cutting work like "we need to refactor our config engine notification because it sucks", then how do you do something like that in a vertical slice. Sometimes it's not always easy to vertically slice something. Eventually I got better at that part. The basic formula that I used was "As a user, I can be assured that [insert-some-horizontal-work-here], so that [insert-some-vague-high-level-user-benefit-here]". So for example, even though it appears to be a horizontal piece of work, I could refactor the config engine with a user story like "As a user, I can be assured that the config engine properly notifies the system of changes, so that the system doesn't crash when I add a new server." As long as I could phrase the work in a way that showed how it would ultimately benefit the end user, then that was okay. Sometimes it took some grammatical gymnastics, but as long as I was making a sincere effort (my director loved to use that term a lot ... "sincere effort") then it was okay. I learned the hard way not to say "we're doing our best". Oh, no, no, no, no. Don't say that. Say "we're making a sincere effort."
This business you're talking about though of *detailed* acceptance criteria, that was really tricky. My PM was also really militant and particular about the way he wanted things, and he wanted acceptance criteria phrased from a human point-of-view. He didn't want these super technical, operationally focused acceptance criteria. He didn't want "When I click on the More ... button, an area of the screen is shown to edit advanced options". That's too implementation focused. He wanted much higher-level acceptance criteria like "The user can effectively use the admin screen to add a new user". He was hard to work with. We were always walking a tight rope between expressing enough details that would help the developers and testers know what we were trying to build, and not putting in implementation details that he didn't care about. It was frustrating at times. He definitely didn't want those super detailed kinds of acceptance criteria that you're talking about. I tried that once and he had a shit fit.
“We have 596 story points to finish” means it’s a fake agile (waterfall with agile trimmings) project. There’s absolutely nothing agile about forecasting and building that much work before getting feedback and adapting the backlog. I talked about that in the “Agile Project Management - Is It Stopping You From Being Agile?” video. I’m sorry you had to work in a project like that. I have several times too. 😢
@@HealthyDev Yeah. You have a larger body of agile work experience than I do. This was the only company I've ever worked at that did textbook scrum, the way Mike Cohn says it should be done, sometimes a bit too rigidly. Estimating is one of those things that people do without asking if it makes sense or not. I don't see the point of it really. Here's the thing. If you size your stories properly, then they end up all being about medium-sized. You don't want your stories too big, otherwise they don't fit into a sprint, and you don't want them too small otherwise they aren't complete vertical slices of user-valuable functionality. So they all end up being these medium-sized things. And then you estimate them with story points and you get 5, 8, 8, 5, 5, 5, 8, 8, 5, 5, 8, 5, 8, 8, 5, 8. If your whole backlog is filled with 5 and 8 point stories, then why bother? Can't you just count up the total number of stories and make some rough estimates with that.
@@kdietz65 thanks for sharing that. Yeah to be clear, I definitely have experienced that MANY companies still force you to estimate regardless of their value. However here's a video from Allen Holub that's pretty cool. While I don't always agree with Allen (and I haven't looked into follow up information on this presentation since I added it to my "Favorite Videos" playlist on the channel), it's got some good things to think about with respect to counting up stories like you mentioned: th-cam.com/video/QVBlnCTu9Ms/w-d-xo.html Not sure if what's in there you've already run into, but maybe it can give you some ideas.
Thanks for these videos - they resonate with me. Ya, too often "User Story" is just a term for an intentionally simplistic spec. How about being asked by your certified scrum master to only provide estimates in story points that represent "complexity and risk" - with no consideration of hourly effort and the physical limits of time within a finite sprint? I think most of us programmers dislike estimating hourly effort, but it is a skill we have to figure out and do well - and demand that we plan our sprints using them, and that they be based on at least some minimum standard of detail.
Thanks, I totally get what you’re saying.
It’s really valuable to be able to estimate but it’s when teams try to forecast work done after multiple sprints that gets them in trouble IMHO.
If they aren’t planning on reprioritizing the backlog when there’s feedback honestly more teams would be better served just doing waterfall. User stories are cool, but I just keep running into projects where they’re treated as enough to make commitments to.
Just my experience of course.
I'm in a project where a User Story pretty much is a Requirement in disguise. It is handed from the customer down to our team(s) after cost-estimates. We are not "agile", this is no secret. And we don't have any discussions about the stories during development. Our team is not focused in developing the service we are primarily developing right now. There are other stuffs that the customer wants. Then a lot of the members are quite junior.
I don't like working from requirements, I want a conversation!
I'm sorry you're having to deal with this. If it makes you feel any better, my experience has been that it's more common for "agile" teams to get this wrong than right. Hopefully it changes over time at your current company/client, or you have a better experience wherever your career leads you to next. Hang in there!
What I don't understand is this: in your experience, are estimates on a fully-documented Waterfall projects any better?
Because in my experience, there is no such thing as a fully-documented project that won't change over time or won't have any "Oh just a little thing..." moments. Agile is only about recognizing that this happens and not make any promises the way we did with Waterfall, but instead offer flexibility and the ability to start with a much smaller investment/commitment.
I had a bit of an ah-ha moment when you mentioned that companies used to decide every little detail upfront AND committing a budget to it before even starting. That's the thing! And if we don't change this part of the process, then most of the advantages are gone.
In my experience the more we design / estimate in one go, the more risk.
So no, waterfall is worse (even if there’s more thought up front to the requirements).
This is another benefit to small batch processes like scrum and kanban: they help you find out early when something is overlooked so you can adjust for it going forward.
I mentioned this in “The Secret Of Scrum” video.
In my experience, when this happens on a waterfall project everyone tries to horse trade scope so they can fit the uncovered complexity into the original deadlines by pushing the devs to work overtime.
But either way, it’s still incredibly useful to create acceptance criteria whether you do it all up front or “just in time”.
It may not eliminate all misunderstandings, but it cuts them WAY down to a more likely understanding of the change or feature.
YMMV
@@HealthyDev I believe that UX design and non functional prototypes play a major role in testing requirement are actually what the business wants them to be. A proper UI/UX work with user testing can get rid of the need of documenting the way waterfall works and without introducing scope creep by creating poorly defined user stories.
Stephane. You are wrong if you think all that ever existed is either Waterfall or Agile, much worse if you think everything about Waterfall was bad (evil) and everything about Agile is good (holy).
See, between Those 2 are a bunch of other methodologies. Before Agile existed they were methodologies that proposed incremental and iterative development (like RUP).
I wish i had seen this video 2 years ago, honestly. 2 years and everything he's saying on here is totally legit!
Another gem.
What happened to the ol good requirements and use cases?
Good stuff.
Hey man, how’s going?
I have a question.
What would we do after have finished and delivered a sprint? If we start working in the new sprint, we would have double work as I’m sure that the user would ask to change many things, so we would have to work with the changes and in the sprint 2 as well.
Should we abandon the sprint 2 for a while until finish the sprint 1?
Henrique Palomo you can either address changes due to feedback in the current sprint by pushing other work to the next one, or just plan to address it in the next one.
The important thing though is to get the change made as soon as is reasonable to minimize effort wasted building additional functionality on the feature being changed. I hope that helps.
Huh, quite interesting. The way I first learned about userstories was with I believe exactly the aspects you were criticising. Meaning, you did *not* just write the userstory. You also had a section "Examples" that specified in a pattern "Given X When Y Then Z" the behaviour of a feature, with a section "Rules" arising from them that described the behaviour in a more summarizing way to give quicker ideas.In this scenario, Examples really just act as acceptance criteria and Rules as their meta description.
What you’re describing sounds like a good process. My main point was user stories on their own don’t offer enough detail to have a contract with people who demand certainty. It sounds like you have preconditions with examples, which is pretty close to the use cases we had back in the waterfall days. Your examples sound like what we used to call alternate cases. Regardless - not to get hung up on the terminology that sounds like a good approach (without knowing the attitude of your stakeholders).
its not clear to me how can i define the acceptance criteria without documenting the whole story. any links or book i can check? ty :)
Hey there yes acceptance criteria is a format for documenting the whole story. I want to do a video myself on this but there are also several really good ones already on TH-cam if you search. Unfortunately I don’t have any particular books I can suggest though I’m sure they exist.
I've had experience working at a company with a multi-step workflow: one individual offers task estimates, another translates them into User Stories and Acceptance Criteria, and a third team member executes the work. While this process seems somewhat disjointed to me, I welcome any input or insights.
The more I watch your videos the more I acknowledge the lack of expertise on agile my PM and my Scrum master really have.
You could always try sending them to the playlists I have on agile. I think the key would be offering it as help for them to get recognized as a better scrum master vs “you’re doing it wrong!” (which admittedly is how I feel at first sometimes).
Still helpful info!
Can you look at what happened to the NHS app and why it cost so much but failed? Was that to do with a bad use of agile?
Hey there. I don't really have the time to do analysis of specific projects. I appreciate the suggestion though.
It sounds to me that you're describing situations where implementation teams are offering estimates on stories that haven't been thoroughly discussed with stakeholders. How on earth can implementers have any idea how much effort it will take to deliver the features described by a story if acceptance criteria haven't been adequately discussed and agreed-upon with the stakeholders? That estimate can only be a lie. I've never been a part of an agile team that proffered story estimates prior to establishing agreed-upon acceptance criteria. That's just insane.
There's a fundamental problem with this video, same as every User Story discussion: It goes into great detail about the problem, but not the solution. There remains a lot of ambiguity around what "requirements" means for a user story. To a fully-Agile/Scrum team, there should be no requirements. But at some point, accountants demand budget estimates. At what point do you shift all the way into "When the button labeled 'X' is clicked, Y happens"?
Hey there. It depends on the level of certainty stakeholders require. As I said in the video, if they want up front estimates and commitments, you need to capture that level of detail in acceptance criteria for every user story *on the project* (not just in the current sprint). In that case, you may as well not use scrum because the purpose of a backlog is agility. You can't have agility if your scope is fixed.
See this other video for more info that may help about this one: th-cam.com/video/OosYzkP-pLk/w-d-xo.html
I have to pull teeth to get anything even approximating acceptance criteria.
Our team has gotten to the point of making MVPs, quickly getting them into the customer's hands, and then getting them to fund improvements. The contracts for improvements come with deadlines. So the constant juggle between getting in high value improvements and meeting deadlines for low value improvements is our biggest pain point right now. Our product owner seems to think giving a wider cushion when negotiating deadlines will help, but we are not yet sure. Do you have any suggestions or should we live with the constant juggling as it comes with being agile?
When you talk about contracts, are y’all building software for other businesses (professional services) or directly for end users?
I'm a BA and I'm not allowed to talk to stakeholders. It's a nightmare. I have no fucking clue what I'm doing.
How can you analyze the business without talking to them??? Incredible. I’m so sorry, that sounds frustrating as hell.
you finally got there around 4:20. the answer here is to not accept, as an R&D leader, an executive team perspective of long range, fixed-scope estimates. They are proven to be inaccurate even when you do get to the detailed level for estimates--for a variety of reasons.
You're calling for a full waterfall process and thus calling anything agile there is deceptive.
Did you watch the whole thing? I'm saying to match the level of detail of your stories to the company's desire for certainty. A company that is OK with uncertainty doesn't need the detail, they have the ability to be agile. The rest unfortunately don't - by definition. They are creating rigidity which resists change.
@@HealthyDevI did watch it all and the point I saw was that if your organization needs schedules and has well defined requirements (one of the places that every new product development organization I've ever been part of insisted on) then waterfall (or scrummerfall for a bit of protective coloration) is what you need to do.
I can't really disagree as most of the weird hybrid attempts I've experienced where some form of agile tries to be mixed with hard requirements and fixed schedules (I work in medical device development) seem to have incorporated the worst parts of agile and waterfall into something that is slower and more cumbersome than either.
@@kylewilson4097that makes sense. This is just my opinion and experience. Yours is just as valid.