The problem here is that developers are rewarded for achieving user stories, and so those who focus on delivery of the narrow user story without caring about the larger picture of code quality and future maintainability, or even the broader picture, will look more successful than those who do care. The problem here is that you end up with what you reward, so if you reward completion of user stories as fast as possible, even if you have tests and QA, you end up with a nightmare of code that has been developed piecemeal without adherence to a broader architecture. You can't expect the development team to care about doing a 'good job' when they are rewarded in a narrow way for stories.
I guess rewarding based on such kind of metric is an issue on it's own. As.a developer (which i am) i would search for a new employer if i would be rated with such a metric. In my opinion the measure for rewards should be done by a technical experienced boss who really understands what's going on inside the projects / teams. And which has a the ability to understand (also technical) the strength and weaknesses of each member in his team (and how they complement each other)
Take a look at the DORA Metrics (i think dave has a video on them) - these metrics measure throughput and delivery, and are harder to game then just # of completed stories or velocity
I have never heard about a project where rewards are shared for completing user stories. This method will reward those who can write user stories that are fast to complete and encourage skipping anything that isn't needed to complete it. Writing any test automation code becomes a burden. I can only imagine how awful the development will come. Developers will naturally focus on getting better rewardsrl rather than producing high-quality code.
I tend to agree, even though I respect and understand Dave's point in this video. What gets measured gets managed. If the success of a team is measured by how many US they close, there will be a huge motivatiin to make a mess, especially from younger, immature teams and team leads who are very dire to please management. Of course, they will do that at the price of destroying the system, as Dave clearly explained. But that means really nothing to some people unfortunately, because some of them will be already promoted to some leadership or management position before that happens. Or, other times, the team that makes the mess is not the one which is most affected by the mess. This is many times true in the projects I work in.
This video has changed my mind about how to prioritize tech stories vs user stories. Previously I had always advocated a tech story (or tech debt) story budget per sprint. This never worked too well for the reasons you state. I like the idea that everything is a user story and what we thought were "tech stories" have business value trying to express itself.
had the same experience for one story i created and since i created it and not our PO it was deemend lower priority and pushed like way too many sprints down the backlog until the issue it tried to prevent surfaced in a loadtest. maybe my selling skills of making it look like a user story are not good enough yet, but my takeaway for the next time: don't bother trying to "sell" a technical necesitity to management folks in isolation via a technical story. like Dave said, they simply can't judge/prioritize something that is outside of their domain of knowledge, that is our job! just put it in the original story as part of a necessary implementation detail.
That manager is rewarded on short term deliverables, not long-term maintenance, and it's reflected in the software akin to Conway's law. He'll be promoted, cuz he's obviously 'effective', and eventually some other sucker will take the fall when the system is found to be 'unmaintainable'.
Often we focus on functional requirements and neglect operational and developmental requirements. Many of these "technical stories" are operational or developmental requirements. I think it helps to ask the questions: How often do you want this to fail? How much do you want development costs to increase per feature? Do the accounts always have to balance? Will it affect the business if a hacker publishes all our customer data? Etc.
I love your work. And I have been quoting your work internally a lot and sending other viewers your way. And we all appreciate your contribution to software engineering
I think perhaps one of the ways people who are struggling with this approach can move towards it more gradually is to define their user stories as their Epics and then their technical implementation stories that achieve that goal as tasks and subtasks. Essentially, use the hierarchy of your task solution to show non-technical colleagues your user focussed work efforts, while allowing your technical teams to still understand and track the implementation details. One other point on the last couple of minutes of the video where you touch on technical debt; I've recently been reclassifying all "tech debt" as either "risk work" or "scale work" in order to be more able to communicate the business need to, for example, update an essential library or framework. By moving away from terming anything as "technical debt", it's much easier to emphasise the necessity to take on those tasks as part of our ongoing work rather than endlessly kicking the can down the road.
I've worked in places where the manager at 6:36 have ground all work to a halt in a company. Nothing can get done, and absolutely nothing can be improved. It was the most demoralising soul crushing place I've ever been. I don't think I've recovered my enthusiasm for the craft that was drained from me by working there.
I tend to agree that we should be focusing on user stories, as they are one of the best ways we have to describe the valuable outcomes our customers desire. They work well for established and experienced teams. My team isn't as experienced, so I like to sprinkle a few "technical breadcrumbs" - questions, pointers, artifacts - attached to the user story to help give less experienced team members a few sign posts, so they can get a bit more context and deliver those valuable outcomes quicker. Not everyone has a photographic memory, or crystal clear mental model of the systems they're working on. I find they give a bit more clarity to more junior team members. Hell I find them useful on tickets I wrote last week - and I've been doing this for 25 years!
Nice video. There is a series of videos by Robert C Martin where he conveys pretty much the same ideas. And he repeats the mantra: "The only way to go fast is to go well".
The ideas presented are accurate, particularly the problem statement - avoid the extremes, learn how to prioritize value propositions. But if tech stories work for a team, let them do it - it's simply a consequence that leaves semantic gray area in contending priorities, philosophies and the constraints of task tracking tools like Jira. That gray area gives developers flexibility they wouldn't otherwise have, which is a good thing. Example: vanilla Jira specifically has Epics, Stories, Tasks, Sub-Tasks. Sub-Tasks don't count toward velocity. And Stories can easily take multiple sprints. One place I worked said, track everything in Stories, we're not doing Sub-Tasks. Great, then I'm going to do technical stories when my PO can't slice work down better and there's five layers to work on that can each be isolated and done by different people. I'm going to get my sprint points, show decent progress and the story will get done over two to four sprints. - you won't really get product owners to easily break large stories into smaller stories. In my experience, they generally get confused trying to track those extra pieces and resist such effort. - and it's a waste of time to think of user value for tech debt or other work that has internal company audience that's better stated as engineering goals or the process of reducing the accidental complexity to essential complexity.
Great stuff as always. I would really appreciate a video (or miniseries) on best practices for undoing technical debt. Some of us are eyeball deep and just keep getting handed shovels.
Sounds good in theory. It's like Mike Tyson says, "Everyone has a plan until they get punched in the face." My punch in the face was "Build a network analyzer that processes every single packet." I've finally discovered the right answer to this though ... Go to HR. It's the only reasonable response. Then quit and go work for a more reasonable director.
I absolutely agree that having always the courage to immediately quit when it's required to avoid certain and predictable failure is an absolute prerequisite for any technical leading role. Don't waste your time and energy on failure. Let them have it all for them self.
I think using the terms Essential and Accidental are a problem. I would think that Domain and Technical are more appropriate and have less confusing connotations for non technical people
I agree that terms 'domain' and 'technical' are easier terms than essential and accidental complexity. But the latter terms convey more meaning regarding the nature of software. You may want to look up an old paper by Fred Brooks "No Silver Bullet: Essence and Accidents of Software Engineering".
John - You beat me to it. I was going to post almost the same thing as you, even with the same terms. But I was thinking of 3 types: 1 - Domain Complexity, i.e., it's baked into the problem. 2 - Technical Complexity, i.e., it's baked into the solution. And I agree with Dave, that we want to keep these two separated. They can change independently, and we want changes in one to have little or no impact upon the other. 3 - Technical Debt [Complexity?], i.e., it's when the overall system becomes overly complex based upon our lack of attention to keep the design and implementation neat and tidy.
As soon as we launch something new here comes Dave with a new video to make me question everything again… just hope my developers are not following this channel that I keep top secret
Say a user story involves changes in the DB, the pipelines, the API, an eventhub or two, terraform, redis, the UI and maybe documentation. Now this is of course a lot of work so we should divide them into smaller tasks. This tasks need to be tracked in Jira since multiple members of the team are going to work in them in parallel. Additionally Jira is used to track working hours for each member. Now a manager comes after watching this video and says we can't have anything that's not a user story in Jira, we need to convert our tasks into user stories. What to do? Each of the tasks adds no business value by themselves.
Why do they need to be tracked separately in Jira? I wouldn’t do it this way. The separate pieces of work are meaningless in terms of tracking the work, until all are in-place the job isn’t done. So much better to tie all of these things together, with say a story-id. Every commit, whoever makes it related to this “user-visible story” is tagged with the story-id. Now we can automatically generate an audit-trail of changes. What benefit have we lost? What does tracking tasks in Jira add?
There are a few reasons for breaking stories up into subtasks: - If a team is working on a fixed sprint cycle a user story might be too large to complete in one sprint. I think for most teams a fixed sprint cycle is not desirable, but that doesn't change the fact that it's pretty standard in the industry right now. - a Jira board can be helpful for team members to decide what to work on next. It's much easier to just check the board and pick a component not being worked on than it is to have to message everyone else working on the story and make sure you aren't duplicating their work. - sometimes multiple user stories can depend on the same technical work. Ticket dependencies can help illustrate this. At the end of the day each ticket should be tied back to some business outcome, but that can be done by linking tickets. Then managers can know what technical work to prioritize because of what user stories they are unlocking by doing so. The alternative is having weeks or months with no visible changes to stake holders and developers having to give constant status updates, which often are meet with diminishing credulity.
We work like this: The subtasks of a user story are owned by the developers to efficiently organize their work. Developers divide the story into many small tasks, most of which can be done in less than half a day, this is the only recommendation. Subtasks are the free space in Jira for developers. They create them, write them, delete them and complete them when and how they want, changing them dynamically at any moment. No one outside the development team has any control over them, but everyone can see them. Stakeholders (as well as developers) enjoy seeing tasks being completed quickly, seing them moving from left to right on the board. Commits are tagged with the user's story ID, and hours worked are captured at the story level, nothing at the task level.
What does doing a task by a few people have to do with Jira? In my experience, Jira gets in the way more than it helps. For example, my team uses a board with sticky notes to track the progress of a user story and our main tool is group work and conversation.
Another problem is that in most companies management wants to have straightforward ways to predict time and effort. This creates the idea that the work should have a time frame, and all the extra time means that you are doing it wrong. This creates another level of demand in managing technical debt that makes our work even more difficult.
Yes, and sadly that is an irrational desire. Pragmatically, we all have to live in that world, but it doesn't apply in other areas of endeavour, and shouldn't apply here. You don't get Venture capitalists saying at the early stages, "tell us exactly how profitable you will be and when", they expect this to be unpredictable, they say things more like "when do you expect to be profitable, and what are your plans to get there". This is a very different kind of question. Some things are literally *unpredictable* so we should treat them differently, and not expect them to be *predictable*. SW dev is a process of learning and exploration, always, and so is inherently unpredictable. The orgs that treat it that way, generally do a lot better at it.
@@ContinuousDelivery Wrong. Predictability of results is something that is understandably quite important to running a business. In very diffuse and not obvious ways it actually translates directly in to money for a business. More frequently than not well profitable companies value that much more than pure costs. Predictability of results includes risk mitigation as well. When a project is hinging for example on a single brilliant guy - it is a huge risk to the company. People get sick. People get in to private trouble. People quit for greener pastures.
Thank god I'm in a team where the PO trusts us and we agree on what we do. And I totally agree that 90% of tech stories have immediate user value. E.g. not having a downtime through smart database migration. Lower downtime with good monitoring or eventually testing.
Thank you. I used to think that technical stories are a necessary evil, so I was really interested in what do you have to say about solving problems that normally justify creating a technical story. Very educating!
I just posted a link to this video to my team's chat. We are a complex subsystem team working on a system specified by our architecture team. It is going to be interesting to see if we can convince the architects to define things as user stories.
I always had a different picture on essential vs. accidental complexity. For me they both occur in user and technical solutions. For me it is often connected to if you need to stick with the first thing that came into your mind because of e.g. time pressure. If a user says, that feature X would be nice, and there is room for discussions (oh, you could do that already that way…), you can make requirements easier and valuable. The same applies to technical solutions. Accidental complexity is when you make it more complex than needed. What is really needed isn’t obvious in most cases.
I have 2 approaches. (1) dedicate a sprint here and there to technical work, no stories needed. And (2) a user story as Dave suggests, but from the perspective of the product owner (or other stakeholders) e.g. as a product owner i want to upgrade log4j so that the platform team doesn't disable my application
Thank you Dave for this great video on a very important and challenging topic, I enjoy watching your content. This is a topic of interest for me for a few years now - how to achieve a good balance between the business and the technical things which need to be done in a project. I'm a technical guy with around 15years of technical-only experience. I know alot of people do not like Scaled Agile (SAFe), but the way they aim to solve this problem is by assigning the System's Architect with the same power as the Product Manager, such that these 2 collaborate in achieving a balanced mix of Feature and Enabler stories inside the backlog - like you mentioned too in the video as an option. For sure, the worst thing we can do is create Technical stories and let the Business prioritize them. That is a disaster, I can confirm I saw this happening in our project. I humbly think that your approach works well for mature agile teams. For immature teams, where developers are dire to ass-kiss management in order to get promoted to management and escape the rat race, this approach may not work. These immature developers simply cannot understand the difference and they will always make a mess. There is still the need for strong technical leaders, who are empowered to push back on the business drive to have "feature factories". Without these technical leaders, I think none of these solutions will work.
Hi Dave. One thing I find hard to understand is how to do user stories if you are in a small team for a microservice, where to complete the story it would require multiple other microservice teams to contribute. Would you do a single user story for each team to use? Or would you have each microservice have their own story?
I think there's some valuable lessons here, but I did not catch a viable alternative. Especially, for larger scale enterprises or multiple teams working in the same space. At the end of the video...he suggests: "keep a list of tidy ups", "make a list of big ticket items of technical work that you want to do". Which...is just another form of "technical stories". If you have to collaborate with multiple engineering teams, work in multiple codebases, etc., then this simplification does not work in my experience. And, I am not talking about creating technical stories just to expose them to the business and let them decide. In an environment with any more complexity than one single team, one product...you just need something more to organize thoughts, collaborate, get consensus, etc.
tidy up list - spend a week to analyze the crap code base you just got put on, put all the hundred items in the backlog as technical work so everyone can see what needs to be gotten to someday. 🤭this was done by our hero in a previous dev life
I did have a case lately where i updated junit library because there was an upstream bug, which would allow developers to violate the API without any notice. My way of reasoning was that developers are stakeholders too. The other case was where i wanted to update selenium library from 3.x to 4.x There my reasoning was that it should either improve the flakiness or it should bring new possibilities to make tests which would be impossible or hard to implement with older version. I tried the update it in a branch and it didnt improve test results substantially. So there was no case for the first reason. For the second reason, i need to create a test case first which would utilize new possibilities to have a sound reason for a change. So i think developers need to learn to better explain the reasons and implications to any group of stakeholders from their decisions, then bringing in purely technical changes wouldn't be a problem .
Thanks Dave, always quality content and interesting reasoning! I've always felt that having separate tech story items in the sprint feels wrong but could not find words to properly explain the reason of my feeling. Now I know what to tell my colleagues
I use the user story format for similarly sizeable chunks of technical work because I find it just as valuable to think of the developers who are going to make use of the technical things created as users and the bdd-style acceptance criteria and tests just as valuable for describing and validating expected bahavior as for end user features. But I do find that there is a natural tendency towards corruption of this usage and drift towards using them as you describe here.
What I don't like about defining everything as a user (end user) story is the risk of making up a reason from the user perspective for a technical need or desire. Being less hackable sounds like an excuse to use the newer C# version because most likely the devs came up with the idea of using newer C# but for other reasons. Then someone needs to bend their mind to find a reason how the user might benefit from it. In out team we too (try to) define everything as a user story but we have broadened the term user to stake holder. And we as developers are also stake holders and we benefit from the newer C# version. Now we write a user story from the devs perspective and that can be an honest one. As a developer on the project I want to use the new C# language features to be more productive.
I think that it is needed, but is often widely, misused. I have seen "Backlog refinement" taken to mean agreeing in a bunch of technical tasks then allocating them to people. That is not its job. The idea is pretty simple, which are the next most important set of features that users want, and prioritise on that basis. It can be a useful, regular sanity check, or it can be radical shift in direction based one new information or priorities, but it is still really only about "which one is next", or should be. If, on average, it is taking more than a few minutes each week or two, it is probably not working properly. Not a bad idea for a video, thanks. 🤔
@@ContinuousDelivery Thanks. It makes sense. Though I wonder at what point stories should get their details in? We mostly need well understood business requirements in stories and I see many of them come from POs having not much more than a title. What session/procedure should take care of this? I don't like my team having to ask all the questions when they already brought the story to the current sprint. Especially that every question asked at this point produces anxiety as developers supposed to be implementing the story which is in the current sprint instead of asking multiple questions about it. And singular planning meeting seems to not be enough to think stories through.
Thanks for this video, the industry is still unmature about this topic, specially tech startups. Recently I was rejected from one of them, literally they mailed me the following: "He wants to focus more on spending time on building things in a quality way than solving problems" It's very paradogical, and the feedback was from the lead architect. Industry is not interested on doing things well, it's disappointing and frustrating. But on the other hand, I feel relieved to don't work on this place.
Yes, it is strange to me that people think this way. The idea that "doing a good job" is a bad idea makes no sense to me. But the assumption that "doing a good job" takes too long and is inefficient is very deeply rooted, wrong but still deeply rooted. Sounds like a lucky escape.
And when the SCRUM master, even when they used to be a Developer no longer values the Tech Stories aspect of the Tech Expert's input, then only User Stories are prioritized as value. It would seem a great SCRUM Master needs to properly be able grok the value of both aspects, and have the Dev-Lead or an assigned representative for self-organizing teams, be part of "World-Control" who are responsible for prioritizing in the SCRUM Agile workflow, or risk Tech-Bankruptcy for failing to address tech-debts weight on the product. And that's if they're unwilling to adopt changes similar to those presented here in an attempt to address the issues also so presented. The organization can adopt the above suggestions to compensate, adapting to the design flaw Borg style and making the best of it as a collective of sorts dual(or multi)-core teaming without it being a full-blown committee, the SCRUM Master retains ultimate say, but has that tech input on priorities and can trust the judgment of the Dev-Lead or Architect giving their perspective on the ground of the priorities based on technical matters. I say OR here between Dev-Lead and Architect because I've been on teams where the two were interchangeable in competence and have been on teams where they're competence wasn't at issue. But I've been on others where the Architect's focus was too concentrated on details being asked of them to be bothered with the task of also being Vice SCRUM Master (after a fashion), and the Lead would likely be the better choice. And in other organizations (bigger one) where there were more than one Lead (teams if you will), so the SCRUM - LEAD processor core will be a bit larger. But that's not specifically a bad thing, just A thing. But it's an alternative\\stop-gap that I'd thought of before finding this presentation when this exact issue presented itself at one of the companies I worked and stymied it's development just as Dave had mentioned, on a legacy core component that was set into MVP (minimum viable product) work only mode, meaning no work to correct issues that was not deem essential by Upper Management, or specifically asked for by the client. And while this node was a core element, it was buggy, and fragile due to it having been in this MVP state for a very long time worth of changes, so any knock-on effects for changes and workarounds to bugs that weren't properly bug-fixed thanks to the MVP policy, simply got addressed only as someone complained about it specifically if ever, or not at all. The construct had no initial test coverage in its design, and thus was not SOLIDly built. It was tightly coupled and a monolith that would ultimately need to be strangled out into various subsystems or services deemed unrelated to the core function from the no less than four known functions and recent fifth it was uncovered that it supported. This was an acquired bit of kit and thus why it was treated with such "love" and affection, but and like "Stargate: Atlantis" had areas that were as yet, undiscovered even after years of ownership. I understand MVP, but view it as a tool intended for rapid release of a prototype or concept, not really for maintaining a live product. I am not the biggest fan of the concept, if nuts and bolts are left on the shop floor when the product rolls out, leaking oil and sparking with a trail of smoke; that might work for the Orks in WH40k if you paint it red, but... . Can MVP work I'm sure it can it all comes down to execute doesn't it. But, it's not meant to be more that a means of RAD, it's not maintainable for a live product. And trying to work around such blockages behind the scenes fixing bugs as part of requested work trying to game the system doing things after hours, etc. is seemingly a surefire way to get sacked as that is was some what the previous Developers did as well as myself(I was found out much later on) that was the result, a hero ain't nothin' but a sandwich! But seriously, if that's the reward for trying to go above and beyond, I hate to see what slacking off gets you! Moral of that story don't game the system, it likely won't work for long. Some efforts will take, "One man can make a difference Michael!" But in the end if you're sacked, does it really matter, unless you're saving lives or the like? But I ain't one to gossip... .
Same! In your case the "other systems" are users of your system. So capture the requirements only from their perspective. In these circumstances it is even trickier to "stay honest" and not allow you implementation detail to leak out into the requirements, but if anything it is even more important. Because your SW doesn't have a human to moderate any complexities in its use. The point here is really to try to ensure that you are focused on what the system needs to do, rather than how it does it. So you design and test from the perspective of the consumer of your system, not from your perspective as the implementer of it. Specifying things, and writing tests this way means that you experience the SW from the outside perspective, and so you will do a better job of designing good APIs to your software.
Well thought out. Could do with more videos on this subject. It is a bad day when those who know the tech need to go to ask tech guidance from those who don't. Doom must surely be looming if that has to happen. Better to leave things legacy and ugly than be led technically by those who have little or no technical knowledge. Great that this video seeks to steer away from this scenario by encouraging developers that they are better placed to come up with plans on how to make the technical changes than are those without their technical knowledge. Maybe a bit more on what to do when legacy tech debt is so severe that the onus shifts to the less technical leaders: Can there ever be a logical case for doing this?
The main problem I experience with technical stories is that nobody will fund the work. Features are customer funded, that's easy. Technical Stories aren't customer funded, and the business won't fund something they see as non-profit. Bureaucracy declares that we cannot just make a change without permission and funding, but this leads to maintenance being side-lined. Doing something quickly for the small win on customer money is prioritised over doing it correctly or carrying out maintenance or improvement work. Things get worse until they hit a breaking point that makes the Bureaucracy have to put out the fire.
Sometimes it's not straight forward to take responsibility for technical changes needed to improve the system. I've worked where we clearly needed a new storage driver for the administration of the system to not grind the scaling of the system to a halt. But there was no way to do that without making it a management approved first class project. It was simply too large a task to do it with the current team resources. ... So nothing happened. Management just kept hoping for a miracle to solve it.
I can defo relate to this. My company often ignores technical restrains, instead they spend on a 'great marketing idea' which then fails after a few months. But never to tackle the basics first, always to jump on the next great idea..
Which is kind of the idea... 1. doing some tech things anyway - as part of our professional duty of care. Don't offer the choice for these sorts of things. 2. Find a way to describe why this change matters to users, even if you think of it as a technical change, find the user need and you have a better shot of convincing people that these "technical things" matter.
how high/low level should these user stories be? “as a user i want to be able to do x“ might range from a days work to a multiple week effort. so if you want to break down a big story, understandably, some might end up with something that resembles a “technical“ story. i feel like this is the black art that nobody ever masters sadly, i have yet to witness it myself. we break up stories into subtasks, not for timekeeping but for refinements and gaining/documenting knowledge. if we come to the conclusion the user story is too big to be delivered in one sprint we break it up. is that bad practice?
Same experience. IMO the root cause of this issue is the forced sprint length, forced maximum story size, and incentives/punishments. Teams are penalized for not making stories small enough to be finished in a company-wide, no-feedback-requested, pre-decided sprint length and constantly berated for 'carry-over' stories. So OK, we're forced to break it down to a level that it no longer resembles a "user" story anymore. Hence tech stories. Maybe leadership could agree to allow "Features" to span multiple sprints (because that's what Teams say it will take....) and then teams could include multiple technical tasks (let's not say "stories") in the "Feature". Good luck with that though.
@@RedBeardedRabbit since we kind of are forced to use Jira, we sometimes circumvent this issue by creating an epic that resembles the higher level user story and link the "Stories"(our name for tickets in our sprints) but that approach has its own quirks and fallacies. still working on a reasonable approach
Sorry, but yes, I think that is bad practice. You have put the planning and execution back into a realm where non-technical people don't "get it" and so they will be tempted to make bad calls, or they will abdicate that responsibility to you. Neither is the best outcome. You will ignore the technicalities, the risk for you is that you get too focused on the technicalities and either over-engineering, or just engineer for stuff that doesn't matter now. Better to facilitate an on-going, collaborative conversation on "what comes next". The way that you do that is to still break the stories down into smaller pieces, but each small piece is user-focused. This allows you to have the conversation with non-technical people, and it also allows more chances to trim scope and to implement the bits that really matter, rather than the bells and whistles that no user really cares about. There is a great book with the best advice on techniques for splitting stories that I have seen: "Fifty Quick Ideas to Improve Your User Stories" - Gojko Adzic & David Evans amzn.to/3jXM481
Fantastic video. In my team we recently started following a somewhat similar approach, but we call user stories "epics" and inside each epic we have a list of "stories". Our stories are usually somewhat technical, but our epics are more user-focused.
Looking for the source of that quote "High performing teams Spend 44% more time on new features." I searched a few of the recent DORA reports and couldn't find it. Can someone please point me to the source? Thanks!
It is mentioned in the Accelerate book (Accelerate, The Science of Lean Software and DevOps, by Nicole Fosgren, Jez Humble & Gene Kim amzn.to/2YYf5Z8), but is based on the State of DevOps research from either the 2014 or 2017 reports I think.
Technical stories in my experience, are usually an eventual consequence of handing down tasks from UI/UX teams to back-end/middleware/API teams where most of the work are indirectly traceable to actual users, so we sometimes begin our user stories with "As a user of this API…", or "As a caller of this Stored Procedure…", which I find very absurd. User stories are supposedly broken down to technical tasks, but what if these tasks become work items of another Scrum team?
If that's how work is segregated, these are not Scrum teams. A Scrum team gathers the people with all the skills required to deliver software by working *together*
@@tomwolverson2500 In large organizations where you have APIs and microservices as deliverables this has always been a challenge because we want to keep our Scrum Teams as small and manageable as possible.
I agree that this type of story sounds dumb. At this level it should just have the problem statement and expected results. As a writer of this comment, I hope I've provided relevant feedback, so that you are entertained.
I don't understand how you can have a user story without a plan on achieving it? Some stories are easy... but others take planning... you cannot just start jamming out code or you end up just throwing mud at the wall... hoping something sticks. I understand what you are saying about user stories vs technical stories... I agree that technical stories should be reworded to user stories... but seems like you would want to put some sub-tasks under a story once you grab it? Or does this mean that the story is not small enough? If that is the case then how is a really small user stary not defining the technical solution? I think I might have to watch this video again... Just feels like you have exposed the problem without really giving a solid answer on how to fix the problem... Some concrete examples perhaps? This is hard stuff... maybe I have to find a video with the answer...
Can the same perhaps be said about mandated innovation days? I haven't ever seen anything worthwhile come out of these. I'm sure it happens sometimes, but half the time people have been too tired and stressed out from barely keeping their heads above the water on the regular backlog grind, and the other two quarters have been either pure technological masturbation, or trivial improvements that should have been done anyway. If they really want genuine innovation to happen, actually respect the practice of user stories which gives the tech team real freedom to decide the implementation, and for the tech team, to have the confidence and free cognitive capacity to learn and try new things, use TDD.
Tech stories may be a bad idea, but tell me how to do this ... "Build a network analyzer that processes every single packet of a fully saturated network." So first, it's impossible. It's ridiculous. There's no product on the planet that does that, and had we been able to do that, it would have represented at least a 3 order of magnitude performance improvement beyond the current product at the time. But assuming it were possible, how would you build that with vertically sliced user stories? How? How are you going to slice that up? It's a quantum thing. Either it processes all the packets, or it doesn't. It's a "you can't get there from here" situation. It defies vertical slicing.
Why a myth for the iron triangle? The trade-off includes Cost. So fast and high quality is fine if you can accept the cost. Have I missed something here?
Yes you have missed something, going fast with high quality reduces rework to the extent that it more than covers the cost. So it isn’t a cost, it’s an investment. The data say that tesms that go fast with high-quality (measured by throughput & stability) spend 44% more time on new features. That is a big hike in productivity! (Data is from the State of DevOps report, and the science behind the analysis is described in the book Accelerate).
Then you should get out more 😉 Just because lots of orgs do something, doesn't make that idea right or good. Lots of bad ideas are popular. We, the software industry, are really bad at eliminating bad ideas.
This is the og scrummaster: advocate for sustainable development, shield the team from wonk, and coach the devs into being accountable for their decisions. Taking a random graduate and telling them to book the retrospective and make a dashboard is pretty far from that
This term , technical stories doesn’t exist in any agile framework I’ve used. User stories, tasks, enablers, sub-tasks of a story, features, epics etc …. Technical stories 😂
Are you sure that the user stories and technical stories are so decoupled. The problem I have experience is that the UI gets no attention as to what the backend should be providing. After all we input something visually and then an output is to be displayed. Arguably the input output process is more important than the backend as the user of the software will be directly effect. The accidental part you claim is as important and part of our job. Indeed in most cases it is the entire reason our jobs exists in the first place. This is way we as an industry have gone to far lengths to define things like MVP and MCV and all it other flavors. A great system can fail because it cannot display data accurately in a way the user can understand. Often the problem to be solved is the UI so then now what? This is why we need the roles because a junior dev as no clue to architect delivery to a UI and we do not want each developer and senior developer reinventing the process in their own image. We need a way to define the system at an engineering level once and definitively. Now we work on stories and it really does not matter what the story is. We have defined units of work and that is the task to be solved at this point.
My point is that I don't think that they should be decoupled at all, but they are often treated as separate pieces of work. I think that we should try to always find the real user need behind any change and work to achieve that. This doesn't mean that the technicalities are unimportant, it means that they are more clearly important, even to non-technical people. We technologists are the experts in this part of the problem, so it is overly naive, though common, for dev teams to defer all planning priorities to non-technical people. It is important that the technical work is prioritised appropriately, and that takes collaboration and negotiation between people that represent different perspectives on the system. I think that this is best done by focusing on what matter to users. This is neither front end nor back end, user stories or technical features. All of these things matter to users. So we organise and plan our work to deliver what our users want, and we add things that our expertise tells us they want even if they don't ask for it directly. Like security, resilience, maintainability and so on. All of these things are clearly, and importantly, in the users interest, but they may not have thought about them in those terms. It is part of our job as technologists to advise them in ways that prevents them from making dumb, naive, overly simplistic prioritisations. It is my view that surfacing "technical stories" for example doesn't help with this. A much better way is to always find, and express, the user value inherent in the technical things that we must do, or just take on the responsibility to do high quality work (from a technical perspective) and don't surface it or ask permission - technical improvements and enhancements are rolled-in to normal, everyday feature development - we don't ask for permission to do a good job!
The problem here is that developers are rewarded for achieving user stories, and so those who focus on delivery of the narrow user story without caring about the larger picture of code quality and future maintainability, or even the broader picture, will look more successful than those who do care. The problem here is that you end up with what you reward, so if you reward completion of user stories as fast as possible, even if you have tests and QA, you end up with a nightmare of code that has been developed piecemeal without adherence to a broader architecture. You can't expect the development team to care about doing a 'good job' when they are rewarded in a narrow way for stories.
I guess rewarding based on such kind of metric is an issue on it's own.
As.a developer (which i am) i would search for a new employer if i would be rated with such a metric.
In my opinion the measure for rewards should be done by a technical experienced boss who really understands what's going on inside the projects / teams.
And which has a the ability to understand (also technical) the strength and weaknesses of each member in his team (and how they complement each other)
Take a look at the DORA Metrics (i think dave has a video on them) - these metrics measure throughput and delivery, and are harder to game then just # of completed stories or velocity
I have never heard about a project where rewards are shared for completing user stories. This method will reward those who can write user stories that are fast to complete and encourage skipping anything that isn't needed to complete it. Writing any test automation code becomes a burden. I can only imagine how awful the development will come. Developers will naturally focus on getting better rewardsrl rather than producing high-quality code.
I tend to agree, even though I respect and understand Dave's point in this video. What gets measured gets managed. If the success of a team is measured by how many US they close, there will be a huge motivatiin to make a mess, especially from younger, immature teams and team leads who are very dire to please management. Of course, they will do that at the price of destroying the system, as Dave clearly explained. But that means really nothing to some people unfortunately, because some of them will be already promoted to some leadership or management position before that happens. Or, other times, the team that makes the mess is not the one which is most affected by the mess. This is many times true in the projects I work in.
@@gewusst-vim9583 Not sure that it is going to change much. I have worked in big companies and there is not much of a difference there, imo.
This video has changed my mind about how to prioritize tech stories vs user stories. Previously I had always advocated a tech story (or tech debt) story budget per sprint. This never worked too well for the reasons you state. I like the idea that everything is a user story and what we thought were "tech stories" have business value trying to express itself.
had the same experience for one story i created and since i created it and not our PO it was deemend lower priority and pushed like way too many sprints down the backlog until the issue it tried to prevent surfaced in a loadtest. maybe my selling skills of making it look like a user story are not good enough yet, but my takeaway for the next time: don't bother trying to "sell" a technical necesitity to management folks in isolation via a technical story. like Dave said, they simply can't judge/prioritize something that is outside of their domain of knowledge, that is our job! just put it in the original story as part of a necessary implementation detail.
Thanks, I am pleased that you found it helpful. 😎
That manager is rewarded on short term deliverables, not long-term maintenance, and it's reflected in the software akin to Conway's law.
He'll be promoted, cuz he's obviously 'effective', and eventually some other sucker will take the fall when the system is found to be 'unmaintainable'.
Often we focus on functional requirements and neglect operational and developmental requirements. Many of these "technical stories" are operational or developmental requirements. I think it helps to ask the questions:
How often do you want this to fail?
How much do you want development costs to increase per feature?
Do the accounts always have to balance?
Will it affect the business if a hacker publishes all our customer data?
Etc.
I love your work. And I have been quoting your work internally a lot and sending other viewers your way. And we all appreciate your contribution to software engineering
Thank you! 😎
I think perhaps one of the ways people who are struggling with this approach can move towards it more gradually is to define their user stories as their Epics and then their technical implementation stories that achieve that goal as tasks and subtasks. Essentially, use the hierarchy of your task solution to show non-technical colleagues your user focussed work efforts, while allowing your technical teams to still understand and track the implementation details.
One other point on the last couple of minutes of the video where you touch on technical debt; I've recently been reclassifying all "tech debt" as either "risk work" or "scale work" in order to be more able to communicate the business need to, for example, update an essential library or framework. By moving away from terming anything as "technical debt", it's much easier to emphasise the necessity to take on those tasks as part of our ongoing work rather than endlessly kicking the can down the road.
I've worked in places where the manager at 6:36 have ground all work to a halt in a company. Nothing can get done, and absolutely nothing can be improved. It was the most demoralising soul crushing place I've ever been. I don't think I've recovered my enthusiasm for the craft that was drained from me by working there.
The problems in this business are so entrenched and systemic that we could probably finish each other's sentences and yet never have met.
I tend to agree that we should be focusing on user stories, as they are one of the best ways we have to describe the valuable outcomes our customers desire. They work well for established and experienced teams. My team isn't as experienced, so I like to sprinkle a few "technical breadcrumbs" - questions, pointers, artifacts - attached to the user story to help give less experienced team members a few sign posts, so they can get a bit more context and deliver those valuable outcomes quicker. Not everyone has a photographic memory, or crystal clear mental model of the systems they're working on. I find they give a bit more clarity to more junior team members. Hell I find them useful on tickets I wrote last week - and I've been doing this for 25 years!
Nice video.
There is a series of videos by Robert C Martin where he conveys pretty much the same ideas. And he repeats the mantra: "The only way to go fast is to go well".
The ideas presented are accurate, particularly the problem statement - avoid the extremes, learn how to prioritize value propositions. But if tech stories work for a team, let them do it - it's simply a consequence that leaves semantic gray area in contending priorities, philosophies and the constraints of task tracking tools like Jira. That gray area gives developers flexibility they wouldn't otherwise have, which is a good thing.
Example: vanilla Jira specifically has Epics, Stories, Tasks, Sub-Tasks. Sub-Tasks don't count toward velocity. And Stories can easily take multiple sprints. One place I worked said, track everything in Stories, we're not doing Sub-Tasks. Great, then I'm going to do technical stories when my PO can't slice work down better and there's five layers to work on that can each be isolated and done by different people. I'm going to get my sprint points, show decent progress and the story will get done over two to four sprints.
- you won't really get product owners to easily break large stories into smaller stories. In my experience, they generally get confused trying to track those extra pieces and resist such effort.
- and it's a waste of time to think of user value for tech debt or other work that has internal company audience that's better stated as engineering goals or the process of reducing the accidental complexity to essential complexity.
Great stuff as always. I would really appreciate a video (or miniseries) on best practices for undoing technical debt. Some of us are eyeball deep and just keep getting handed shovels.
My guesses would be prototyping, refactoring and redesign
Sounds good in theory. It's like Mike Tyson says, "Everyone has a plan until they get punched in the face." My punch in the face was "Build a network analyzer that processes every single packet." I've finally discovered the right answer to this though ... Go to HR. It's the only reasonable response. Then quit and go work for a more reasonable director.
I absolutely agree that having always the courage to immediately quit when it's required to avoid certain and predictable failure is an absolute prerequisite for any technical leading role. Don't waste your time and energy on failure. Let them have it all for them self.
I think using the terms Essential and Accidental are a problem. I would think that Domain and Technical are more appropriate and have less confusing connotations for non technical people
I agree that terms 'domain' and 'technical' are easier terms than essential and accidental complexity. But the latter terms convey more meaning regarding the nature of software.
You may want to look up an old paper by Fred Brooks "No Silver Bullet: Essence and Accidents of Software Engineering".
John - You beat me to it. I was going to post almost the same thing as you, even with the same terms. But I was thinking of 3 types:
1 - Domain Complexity, i.e., it's baked into the problem.
2 - Technical Complexity, i.e., it's baked into the solution.
And I agree with Dave, that we want to keep these two separated. They can change independently, and we want changes in one to have little or no impact upon the other.
3 - Technical Debt [Complexity?], i.e., it's when the overall system becomes overly complex based upon our lack of attention to keep the design and implementation neat and tidy.
As soon as we launch something new here comes Dave with a new video to make me question everything again… just hope my developers are not following this channel that I keep top secret
🤣🤣 I will try and speak more quietly next time, so they don't hear.
Say a user story involves changes in the DB, the pipelines, the API, an eventhub or two, terraform, redis, the UI and maybe documentation. Now this is of course a lot of work so we should divide them into smaller tasks. This tasks need to be tracked in Jira since multiple members of the team are going to work in them in parallel. Additionally Jira is used to track working hours for each member. Now a manager comes after watching this video and says we can't have anything that's not a user story in Jira, we need to convert our tasks into user stories. What to do? Each of the tasks adds no business value by themselves.
Why do they need to be tracked separately in Jira? I wouldn’t do it this way. The separate pieces of work are meaningless in terms of tracking the work, until all are in-place the job isn’t done. So much better to tie all of these things together, with say a story-id. Every commit, whoever makes it related to this “user-visible story” is tagged with the story-id. Now we can automatically generate an audit-trail of changes. What benefit have we lost? What does tracking tasks in Jira add?
Jira is the root of all evil
There are a few reasons for breaking stories up into subtasks:
- If a team is working on a fixed sprint cycle a user story might be too large to complete in one sprint. I think for most teams a fixed sprint cycle is not desirable, but that doesn't change the fact that it's pretty standard in the industry right now.
- a Jira board can be helpful for team members to decide what to work on next. It's much easier to just check the board and pick a component not being worked on than it is to have to message everyone else working on the story and make sure you aren't duplicating their work.
- sometimes multiple user stories can depend on the same technical work. Ticket dependencies can help illustrate this.
At the end of the day each ticket should be tied back to some business outcome, but that can be done by linking tickets. Then managers can know what technical work to prioritize because of what user stories they are unlocking by doing so.
The alternative is having weeks or months with no visible changes to stake holders and developers having to give constant status updates, which often are meet with diminishing credulity.
We work like this: The subtasks of a user story are owned by the developers to efficiently organize their work. Developers divide the story into many small tasks, most of which can be done in less than half a day, this is the only recommendation. Subtasks are the free space in Jira for developers. They create them, write them, delete them and complete them when and how they want, changing them dynamically at any moment.
No one outside the development team has any control over them, but everyone can see them. Stakeholders (as well as developers) enjoy seeing tasks being completed quickly, seing them moving from left to right on the board.
Commits are tagged with the user's story ID, and hours worked are captured at the story level, nothing at the task level.
What does doing a task by a few people have to do with Jira? In my experience, Jira gets in the way more than it helps. For example, my team uses a board with sticky notes to track the progress of a user story and our main tool is group work and conversation.
Another problem is that in most companies management wants to have straightforward ways to predict time and effort. This creates the idea that the work should have a time frame, and all the extra time means that you are doing it wrong.
This creates another level of demand in managing technical debt that makes our work even more difficult.
Yes, and sadly that is an irrational desire. Pragmatically, we all have to live in that world, but it doesn't apply in other areas of endeavour, and shouldn't apply here. You don't get Venture capitalists saying at the early stages, "tell us exactly how profitable you will be and when", they expect this to be unpredictable, they say things more like "when do you expect to be profitable, and what are your plans to get there". This is a very different kind of question. Some things are literally *unpredictable* so we should treat them differently, and not expect them to be *predictable*. SW dev is a process of learning and exploration, always, and so is inherently unpredictable. The orgs that treat it that way, generally do a lot better at it.
@@ContinuousDelivery Wrong. Predictability of results is something that is understandably quite important to running a business. In very diffuse and not obvious ways it actually translates directly in to money for a business. More frequently than not well profitable companies value that much more than pure costs. Predictability of results includes risk mitigation as well. When a project is hinging for example on a single brilliant guy - it is a huge risk to the company. People get sick. People get in to private trouble. People quit for greener pastures.
Thank god I'm in a team where the PO trusts us and we agree on what we do. And I totally agree that 90% of tech stories have immediate user value. E.g. not having a downtime through smart database migration. Lower downtime with good monitoring or eventually testing.
Thank you. I used to think that technical stories are a necessary evil, so I was really interested in what do you have to say about solving problems that normally justify creating a technical story. Very educating!
I just posted a link to this video to my team's chat. We are a complex subsystem team working on a system specified by our architecture team. It is going to be interesting to see if we can convince the architects to define things as user stories.
I always had a different picture on essential vs. accidental complexity. For me they both occur in user and technical solutions. For me it is often connected to if you need to stick with the first thing that came into your mind because of e.g. time pressure. If a user says, that feature X would be nice, and there is room for discussions (oh, you could do that already that way…), you can make requirements easier and valuable. The same applies to technical solutions. Accidental complexity is when you make it more complex than needed. What is really needed isn’t obvious in most cases.
I have 2 approaches. (1) dedicate a sprint here and there to technical work, no stories needed. And (2) a user story as Dave suggests, but from the perspective of the product owner (or other stakeholders) e.g. as a product owner i want to upgrade log4j so that the platform team doesn't disable my application
Thank you Dave for this great video on a very important and challenging topic, I enjoy watching your content. This is a topic of interest for me for a few years now - how to achieve a good balance between the business and the technical things which need to be done in a project. I'm a technical guy with around 15years of technical-only experience.
I know alot of people do not like Scaled Agile (SAFe), but the way they aim to solve this problem is by assigning the System's Architect with the same power as the Product Manager, such that these 2 collaborate in achieving a balanced mix of Feature and Enabler stories inside the backlog - like you mentioned too in the video as an option. For sure, the worst thing we can do is create Technical stories and let the Business prioritize them. That is a disaster, I can confirm I saw this happening in our project. I humbly think that your approach works well for mature agile teams. For immature teams, where developers are dire to ass-kiss management in order to get promoted to management and escape the rat race, this approach may not work. These immature developers simply cannot understand the difference and they will always make a mess. There is still the need for strong technical leaders, who are empowered to push back on the business drive to have "feature factories". Without these technical leaders, I think none of these solutions will work.
Hi Dave. One thing I find hard to understand is how to do user stories if you are in a small team for a microservice, where to complete the story it would require multiple other microservice teams to contribute. Would you do a single user story for each team to use? Or would you have each microservice have their own story?
I think there's some valuable lessons here, but I did not catch a viable alternative. Especially, for larger scale enterprises or multiple teams working in the same space. At the end of the video...he suggests: "keep a list of tidy ups", "make a list of big ticket items of technical work that you want to do". Which...is just another form of "technical stories". If you have to collaborate with multiple engineering teams, work in multiple codebases, etc., then this simplification does not work in my experience. And, I am not talking about creating technical stories just to expose them to the business and let them decide. In an environment with any more complexity than one single team, one product...you just need something more to organize thoughts, collaborate, get consensus, etc.
tidy up list - spend a week to analyze the crap code base you just got put on, put all the hundred items in the backlog as technical work so everyone can see what needs to be gotten to someday. 🤭this was done by our hero in a previous dev life
I did have a case lately where i updated junit library because there was an upstream bug, which would allow developers to violate the API without any notice. My way of reasoning was that developers are stakeholders too. The other case was where i wanted to update selenium library from 3.x to 4.x There my reasoning was that it should either improve the flakiness or it should bring new possibilities to make tests which would be impossible or hard to implement with older version. I tried the update it in a branch and it didnt improve test results substantially. So there was no case for the first reason. For the second reason, i need to create a test case first which would utilize new possibilities to have a sound reason for a change. So i think developers need to learn to better explain the reasons and implications to any group of stakeholders from their decisions, then bringing in purely technical changes wouldn't be a problem .
Thanks Dave, always quality content and interesting reasoning! I've always felt that having separate tech story items in the sprint feels wrong but could not find words to properly explain the reason of my feeling. Now I know what to tell my colleagues
I use the user story format for similarly sizeable chunks of technical work because I find it just as valuable to think of the developers who are going to make use of the technical things created as users and the bdd-style acceptance criteria and tests just as valuable for describing and validating expected bahavior as for end user features.
But I do find that there is a natural tendency towards corruption of this usage and drift towards using them as you describe here.
What I don't like about defining everything as a user (end user) story is the risk of making up a reason from the user perspective for a technical need or desire. Being less hackable sounds like an excuse to use the newer C# version because most likely the devs came up with the idea of using newer C# but for other reasons. Then someone needs to bend their mind to find a reason how the user might benefit from it.
In out team we too (try to) define everything as a user story but we have broadened the term user to stake holder. And we as developers are also stake holders and we benefit from the newer C# version. Now we write a user story from the devs perspective and that can be an honest one. As a developer on the project I want to use the new C# language features to be more productive.
Dave, I wonder if you think that backlog refinement is needed or not. Do you have a video about it? If not could you tell me your thoughts, please?
I think that it is needed, but is often widely, misused. I have seen "Backlog refinement" taken to mean agreeing in a bunch of technical tasks then allocating them to people. That is not its job. The idea is pretty simple, which are the next most important set of features that users want, and prioritise on that basis. It can be a useful, regular sanity check, or it can be radical shift in direction based one new information or priorities, but it is still really only about "which one is next", or should be. If, on average, it is taking more than a few minutes each week or two, it is probably not working properly.
Not a bad idea for a video, thanks. 🤔
@@ContinuousDelivery Thanks. It makes sense. Though I wonder at what point stories should get their details in? We mostly need well understood business requirements in stories and I see many of them come from POs having not much more than a title. What session/procedure should take care of this? I don't like my team having to ask all the questions when they already brought the story to the current sprint. Especially that every question asked at this point produces anxiety as developers supposed to be implementing the story which is in the current sprint instead of asking multiple questions about it. And singular planning meeting seems to not be enough to think stories through.
Really cool t-shirt 😁 Where can I get one?
Thanks for this video, the industry is still unmature about this topic, specially tech startups.
Recently I was rejected from one of them, literally they mailed me the following:
"He wants to focus more on spending time on building things in a quality way than solving problems"
It's very paradogical, and the feedback was from the lead architect.
Industry is not interested on doing things well, it's disappointing and frustrating.
But on the other hand, I feel relieved to don't work on this place.
Yes, it is strange to me that people think this way. The idea that "doing a good job" is a bad idea makes no sense to me. But the assumption that "doing a good job" takes too long and is inefficient is very deeply rooted, wrong but still deeply rooted.
Sounds like a lucky escape.
Great video, really insightful, gotta love the Jinx shirt by the way.
And when the SCRUM master, even when they used to be a Developer no longer values the Tech Stories aspect of the Tech Expert's input, then only User Stories are prioritized as value. It would seem a great SCRUM Master needs to properly be able grok the value of both aspects, and have the Dev-Lead or an assigned representative for self-organizing teams, be part of "World-Control" who are responsible for prioritizing in the SCRUM Agile workflow, or risk Tech-Bankruptcy for failing to address tech-debts weight on the product.
And that's if they're unwilling to adopt changes similar to those presented here in an attempt to address the issues also so presented. The organization can adopt the above suggestions to compensate, adapting to the design flaw Borg style and making the best of it as a collective of sorts dual(or multi)-core teaming without it being a full-blown committee, the SCRUM Master retains ultimate say, but has that tech input on priorities and can trust the judgment of the Dev-Lead or Architect giving their perspective on the ground of the priorities based on technical matters. I say OR here between Dev-Lead and Architect because I've been on teams where the two were interchangeable in competence and have been on teams where they're competence wasn't at issue. But I've been on others where the Architect's focus was too concentrated on details being asked of them to be bothered with the task of also being Vice SCRUM Master (after a fashion), and the Lead would likely be the better choice. And in other organizations (bigger one) where there were more than one Lead (teams if you will), so the SCRUM - LEAD processor core will be a bit larger. But that's not specifically a bad thing, just A thing. But it's an alternative\\stop-gap that I'd thought of before finding this presentation when this exact issue presented itself at one of the companies I worked and stymied it's development just as Dave had mentioned, on a legacy core component that was set into MVP (minimum viable product) work only mode, meaning no work to correct issues that was not deem essential by Upper Management, or specifically asked for by the client. And while this node was a core element, it was buggy, and fragile due to it having been in this MVP state for a very long time worth of changes, so any knock-on effects for changes and workarounds to bugs that weren't properly bug-fixed thanks to the MVP policy, simply got addressed only as someone complained about it specifically if ever, or not at all. The construct had no initial test coverage in its design, and thus was not SOLIDly built. It was tightly coupled and a monolith that would ultimately need to be strangled out into various subsystems or services deemed unrelated to the core function from the no less than four known functions and recent fifth it was uncovered that it supported. This was an acquired bit of kit and thus why it was treated with such "love" and affection, but and like "Stargate: Atlantis" had areas that were as yet, undiscovered even after years of ownership. I understand MVP, but view it as a tool intended for rapid release of a prototype or concept, not really for maintaining a live product. I am not the biggest fan of the concept, if nuts and bolts are left on the shop floor when the product rolls out, leaking oil and sparking with a trail of smoke; that might work for the Orks in WH40k if you paint it red, but... . Can MVP work I'm sure it can it all comes down to execute doesn't it. But, it's not meant to be more that a means of RAD, it's not maintainable for a live product.
And trying to work around such blockages behind the scenes fixing bugs as part of requested work trying to game the system doing things after hours, etc. is seemingly a surefire way to get sacked as that is was some what the previous Developers did as well as myself(I was found out much later on) that was the result, a hero ain't nothin' but a sandwich! But seriously, if that's the reward for trying to go above and beyond, I hate to see what slacking off gets you! Moral of that story don't game the system, it likely won't work for long. Some efforts will take, "One man can make a difference Michael!" But in the end if you're sacked, does it really matter, unless you're saving lives or the like?
But I ain't one to gossip... .
"Inside every technical story is a user story struggling to get out" love this quote
I struggle with multiple “technical stories” that would relate to the same user story.
"technical masturbation dreams" ... is that a term we should have heard during college lectures :D
Hit the nail on the head as ever!!! nice
What if there are no users? How would do you do BDD for a batch processing system that exclusively process files sent from other systems?
Same! In your case the "other systems" are users of your system. So capture the requirements only from their perspective. In these circumstances it is even trickier to "stay honest" and not allow you implementation detail to leak out into the requirements, but if anything it is even more important. Because your SW doesn't have a human to moderate any complexities in its use. The point here is really to try to ensure that you are focused on what the system needs to do, rather than how it does it. So you design and test from the perspective of the consumer of your system, not from your perspective as the implementer of it. Specifying things, and writing tests this way means that you experience the SW from the outside perspective, and so you will do a better job of designing good APIs to your software.
Well thought out. Could do with more videos on this subject. It is a bad day when those who know the tech need to go to ask tech guidance from those who don't. Doom must surely be looming if that has to happen. Better to leave things legacy and ugly than be led technically by those who have little or no technical knowledge. Great that this video seeks to steer away from this scenario by encouraging developers that they are better placed to come up with plans on how to make the technical changes than are those without their technical knowledge. Maybe a bit more on what to do when legacy tech debt is so severe that the onus shifts to the less technical leaders: Can there ever be a logical case for doing this?
The main problem I experience with technical stories is that nobody will fund the work. Features are customer funded, that's easy. Technical Stories aren't customer funded, and the business won't fund something they see as non-profit. Bureaucracy declares that we cannot just make a change without permission and funding, but this leads to maintenance being side-lined. Doing something quickly for the small win on customer money is prioritised over doing it correctly or carrying out maintenance or improvement work. Things get worse until they hit a breaking point that makes the Bureaucracy have to put out the fire.
Sometimes it's not straight forward to take responsibility for technical changes needed to improve the system.
I've worked where we clearly needed a new storage driver for the administration of the system to not grind the scaling of the system to a halt.
But there was no way to do that without making it a management approved first class project. It was simply too large a task to do it with the current team resources.
... So nothing happened. Management just kept hoping for a miracle to solve it.
I can defo relate to this. My company often ignores technical restrains, instead they spend on a 'great marketing idea' which then fails after a few months. But never to tackle the basics first, always to jump on the next great idea..
Which is kind of the idea...
1. doing some tech things anyway - as part of our professional duty of care. Don't offer the choice for these sorts of things.
2. Find a way to describe why this change matters to users, even if you think of it as a technical change, find the user need and you have a better shot of convincing people that these "technical things" matter.
how high/low level should these user stories be? “as a user i want to be able to do x“ might range from a days work to a multiple week effort. so if you want to break down a big story, understandably, some might end up with something that resembles a “technical“ story. i feel like this is the black art that nobody ever masters sadly, i have yet to witness it myself. we break up stories into subtasks, not for timekeeping but for refinements and gaining/documenting knowledge. if we come to the conclusion the user story is too big to be delivered in one sprint we break it up. is that bad practice?
Same experience. IMO the root cause of this issue is the forced sprint length, forced maximum story size, and incentives/punishments. Teams are penalized for not making stories small enough to be finished in a company-wide, no-feedback-requested, pre-decided sprint length and constantly berated for 'carry-over' stories.
So OK, we're forced to break it down to a level that it no longer resembles a "user" story anymore. Hence tech stories.
Maybe leadership could agree to allow "Features" to span multiple sprints (because that's what Teams say it will take....) and then teams could include multiple technical tasks (let's not say "stories") in the "Feature".
Good luck with that though.
@@RedBeardedRabbit since we kind of are forced to use Jira, we sometimes circumvent this issue by creating an epic that resembles the higher level user story and link the "Stories"(our name for tickets in our sprints) but that approach has its own quirks and fallacies. still working on a reasonable approach
Sorry, but yes, I think that is bad practice. You have put the planning and execution back into a realm where non-technical people don't "get it" and so they will be tempted to make bad calls, or they will abdicate that responsibility to you. Neither is the best outcome. You will ignore the technicalities, the risk for you is that you get too focused on the technicalities and either over-engineering, or just engineer for stuff that doesn't matter now. Better to facilitate an on-going, collaborative conversation on "what comes next".
The way that you do that is to still break the stories down into smaller pieces, but each small piece is user-focused.
This allows you to have the conversation with non-technical people, and it also allows more chances to trim scope and to implement the bits that really matter, rather than the bells and whistles that no user really cares about. There is a great book with the best advice on techniques for splitting stories that I have seen:
"Fifty Quick Ideas to Improve Your User Stories" - Gojko Adzic & David Evans amzn.to/3jXM481
@@ContinuousDelivery thanks for the answer, will have a look into the suggested book
Fantastic video. In my team we recently started following a somewhat similar approach, but we call user stories "epics" and inside each epic we have a list of "stories". Our stories are usually somewhat technical, but our epics are more user-focused.
Looking for the source of that quote "High performing teams Spend 44% more time on new features." I searched a few of the recent DORA reports and couldn't find it. Can someone please point me to the source? Thanks!
It is mentioned in the Accelerate book (Accelerate, The Science of Lean Software and DevOps, by Nicole Fosgren, Jez Humble & Gene Kim amzn.to/2YYf5Z8), but is based on the State of DevOps research from either the 2014 or 2017 reports I think.
@@ContinuousDelivery Thanks a ton!
awesome content, awesome t-shirt
Was looking for this comment, did not expect Dave to wear an Arcane/League shirt
Technical stories in my experience, are usually an eventual consequence of handing down tasks from UI/UX teams to back-end/middleware/API teams where most of the work are indirectly traceable to actual users, so we sometimes begin our user stories with "As a user of this API…", or "As a caller of this Stored Procedure…", which I find very absurd. User stories are supposedly broken down to technical tasks, but what if these tasks become work items of another Scrum team?
If that's how work is segregated, these are not Scrum teams. A Scrum team gathers the people with all the skills required to deliver software by working *together*
@@tomwolverson2500 In large organizations where you have APIs and microservices as deliverables this has always been a challenge because we want to keep our Scrum Teams as small and manageable as possible.
I agree that this type of story sounds dumb. At this level it should just have the problem statement and expected results.
As a writer of this comment,
I hope I've provided relevant feedback,
so that you are entertained.
I don't understand how you can have a user story without a plan on achieving it? Some stories are easy... but others take planning... you cannot just start jamming out code or you end up just throwing mud at the wall... hoping something sticks.
I understand what you are saying about user stories vs technical stories... I agree that technical stories should be reworded to user stories... but seems like you would want to put some sub-tasks under a story once you grab it? Or does this mean that the story is not small enough? If that is the case then how is a really small user stary not defining the technical solution?
I think I might have to watch this video again... Just feels like you have exposed the problem without really giving a solid answer on how to fix the problem... Some concrete examples perhaps? This is hard stuff... maybe I have to find a video with the answer...
Great idea! Everything is perspective.
Can the same perhaps be said about mandated innovation days?
I haven't ever seen anything worthwhile come out of these. I'm sure it happens sometimes, but half the time people have been too tired and stressed out from barely keeping their heads above the water on the regular backlog grind, and the other two quarters have been either pure technological masturbation, or trivial improvements that should have been done anyway.
If they really want genuine innovation to happen, actually respect the practice of user stories which gives the tech team real freedom to decide the implementation, and for the tech team, to have the confidence and free cognitive capacity to learn and try new things, use TDD.
Is technical stories just a new phrase for technical debt? Is a technical just a documented task or issue to reduce technical debt?
very informative video
Tech stories may be a bad idea, but tell me how to do this ... "Build a network analyzer that processes every single packet of a fully saturated network." So first, it's impossible. It's ridiculous. There's no product on the planet that does that, and had we been able to do that, it would have represented at least a 3 order of magnitude performance improvement beyond the current product at the time. But assuming it were possible, how would you build that with vertically sliced user stories? How? How are you going to slice that up? It's a quantum thing. Either it processes all the packets, or it doesn't. It's a "you can't get there from here" situation. It defies vertical slicing.
Why a myth for the iron triangle? The trade-off includes Cost. So fast and high quality is fine if you can accept the cost. Have I missed something here?
Yes you have missed something, going fast with high quality reduces rework to the extent that it more than covers the cost. So it isn’t a cost, it’s an investment. The data say that tesms that go fast with high-quality (measured by throughput & stability) spend 44% more time on new features. That is a big hike in productivity!
(Data is from the State of DevOps report, and the science behind the analysis is described in the book Accelerate).
@@ContinuousDelivery I agree with you in that but isn't the triangle about upfront cost? The here and now cost that the business recognises
Where did Dave find these wonderful shirts! XD
I've yet to see a single shop that wouldn't black ball someone for suggesting any of this.
Then you should get out more 😉
Just because lots of orgs do something, doesn't make that idea right or good. Lots of bad ideas are popular. We, the software industry, are really bad at eliminating bad ideas.
@@ContinuousDelivery I completely agree, on all counts. Have any suggestions?? 😉
The problem is not technical user stories; it is technical work sitting on the product backlog.
and ironically, ALL the work done is technical!
This is the og scrummaster: advocate for sustainable development, shield the team from wonk, and coach the devs into being accountable for their decisions. Taking a random graduate and telling them to book the retrospective and make a dashboard is pretty far from that
Love the shirt!
Non-technical stuff I call "business" requirements
"As a REST API I want the DELETE function to... delete my object". 😂
Is that a League of Legends Jinx shirt?
Yes, though I think of it as an Arcane T-shirt, have you the “Arcane” animated series? Brilliant.
Arcane shirt for arcane concepts
THAAATS MEE
This term , technical stories doesn’t exist in any agile framework I’ve used. User stories, tasks, enablers, sub-tasks of a story, features, epics etc …. Technical stories 😂
Goldmine
Are you sure that the user stories and technical stories are so decoupled. The problem I have experience is that the UI gets no attention as to what the backend should be providing. After all we input something visually and then an output is to be displayed. Arguably the input output process is more important than the backend as the user of the software will be directly effect. The accidental part you claim is as important and part of our job. Indeed in most cases it is the entire reason our jobs exists in the first place. This is way we as an industry have gone to far lengths to define things like MVP and MCV and all it other flavors. A great system can fail because it cannot display data accurately in a way the user can understand. Often the problem to be solved is the UI so then now what? This is why we need the roles because a junior dev as no clue to architect delivery to a UI and we do not want each developer and senior developer reinventing the process in their own image. We need a way to define the system at an engineering level once and definitively. Now we work on stories and it really does not matter what the story is. We have defined units of work and that is the task to be solved at this point.
My point is that I don't think that they should be decoupled at all, but they are often treated as separate pieces of work. I think that we should try to always find the real user need behind any change and work to achieve that. This doesn't mean that the technicalities are unimportant, it means that they are more clearly important, even to non-technical people. We technologists are the experts in this part of the problem, so it is overly naive, though common, for dev teams to defer all planning priorities to non-technical people. It is important that the technical work is prioritised appropriately, and that takes collaboration and negotiation between people that represent different perspectives on the system.
I think that this is best done by focusing on what matter to users. This is neither front end nor back end, user stories or technical features. All of these things matter to users. So we organise and plan our work to deliver what our users want, and we add things that our expertise tells us they want even if they don't ask for it directly. Like security, resilience, maintainability and so on. All of these things are clearly, and importantly, in the users interest, but they may not have thought about them in those terms. It is part of our job as technologists to advise them in ways that prevents them from making dumb, naive, overly simplistic prioritisations.
It is my view that surfacing "technical stories" for example doesn't help with this. A much better way is to always find, and express, the user value inherent in the technical things that we must do, or just take on the responsibility to do high quality work (from a technical perspective) and don't surface it or ask permission - technical improvements and enhancements are rolled-in to normal, everyday feature development - we don't ask for permission to do a good job!