🚀 TDD, Where Did It All Go Wrong (Ian Cooper)
ฝัง
- เผยแพร่เมื่อ 19 ธ.ค. 2017
- ❗️ATTENTION ❗️
Registration to DevTernity is running:
👉 devternity.com
Since Kent Beck wrote the book on TDD in 2002 a lot of words have been dedicated to the subject. But many of them propagated misunderstandings of Kent's original rules so that TDD practice bears little resemblance to Kent's original ideas. Key misunderstandings around what do I test, what is a unit test, and what is the 'public interface' have led to test suites that are brittle, hard to read, and do not support easy refactoring. In this talk, we re-discover Kent's original proposition, discover where key misunderstandings occurred and look at a better approach to TDD that supports software development instead of impeding it. Be prepared from some sacred cows to be slaughtered and fewer but better tests to be written. - วิทยาศาสตร์และเทคโนโลยี
Notes I made during this presentation (just a dump, might be useful to some):
- Test requirements, not low level
- Test public API. Given when then
- Test the exports from a module
- Focus on higher-level
- Test modules, not class
- Refactoring is needed to see what is implementation and what is exports from module
- Test behaviours
- Think about your code as an api
- Test the abstraction, not the implementation
- Test are isolated and with shared fixture (to run quickly)
- Red-green-refactor (go fast to working code)
- No new tests during refactoring
- Heavy coupling is the problem with all software
- Thin public api
- Refactoring = changing internals
- Patterns in the refactoring
- If you're not really sure, write tests for implementation (delete the tests)
- Not classes, behaviours
- Don't isolate classes in testing
- Private methods (these are implementation details)
so it's just common sense?
@@vborovikov Yes, but people don't seem to be having it. Hence the need for this talk.
Also, keep tests fast (
@@DanHaiduc Clarification: whole test suite should run in under a few minutes, not just one unit test. And having DB communication during testing is OK as long as it is isolated from other tests (by either cleaning it or using exclusive instances or what ever. Basically memory based db that is recreated for each test).
@@dyyd0 You are absolutely right. After watching Fast Test, Slow Test I have realized that the whole test suite should run in **less than two seconds**. It is possible if you don't invoke any big frameworks during the tests:
th-cam.com/video/RAxiiRPHS9k/w-d-xo.html
I like this talk. What I learned:
"Tests protect something. You want them to protect requirements, not implementation details. Thats why you delete tests that were only there to make it happen, but are not a result of a requirement"
a beautiful TLDR , that you soooooooooo much.
That's*
That's loony, Kevin. Tests are small, mechanical procedures. They challenge mechanical workings of a program.
Getting reuirements and design right is an intellecual (and political) process. It has to be done by people representing the interests being served by the work at hand.
It's not a matter of test, it's a matter of debate and judgement.
@@TheDavidlloydjones What you describe regarding requirements is not opposing to what I wrote.
But you are wrong regarding how tests should be written. The way you describe will result in tests that break when you refactor SW. Thus your tests will hinder changes. It is a mistake that I've done in the past but one that is worth preventing.
When management makes the test coverage of some tool a "requirement" the whole TDD thing goes tits up. I've seen it a dozen times and a dozen times it has the same result - a lot of tests that don't test jack shit, they are just there to do some coverage by either "verifying" something or "testing" implementation details.
This guy is speaking from hard earn't real-world experience. We need more engineers like him imparting their knowledge. Rather than 25 year olds who have real industry experience of 1.7 years and have already written 2 books.
Robert C. Martin has been programming for over 50 years, and he disagrees. He says go for 100% coverage. If you don't test implementation, you are going to miss bugs and design flaws.
@@626Pilot Robert C. Martin actually agrees with what is said in this talk. It can be read in his article about test contra-variance. Just google: cleancoder test contra-variance.
@@626Pilot uncle bob is full of shit
"He says go for 100% coverage..." Code Coverage is a hollow stat that many inexperienced developers put far to high emphasis on. You can still have various bugs with 100% code coverage, just because you covered all cases doesn't mean the cases are valid behavior.
@Peter Mortensen no test fails when a comment starts telling lies due to the underlying code changes.
This is why better avoid comments. I appreciate there might be complexity in certain systems which is hard to grasp withouta comment though.
15 years ago in my one of my first contract programming jobs (I was a late starter), Ian took me onto his team, assigned me a more experienced programmer as a mentor and told me to read Kent Beck's TDD book. It completely changed the way I approached programming. I'm immensely grateful of the time he took to teach me. Great guy.
The problem comes from tools that do "CODE COVERAGE". Because of that metric, devs just end up testing their implementation so the chart that goes to the bosses and regulators is "oh look 99% code coverage of tests" when in practice it's doing absolutely nothing to improve the code.
I get the knee jerk reaction but IMO it can tell you a lot if it's below 80%. I think there should be a minimum but chasing 100% coverage isn't something worth having for the reason you mentioned.
Yes, I think test coverage tools mat have led me astray. I don't use them but it implies that all methods should be tested. Also the test first methodology makes me think of testing methods because I think in terms of writing methods. However I am going to try writing tests based on the expected behaviour. Generally everything I write starts with a web page and ends in a database update. Mostly of this boils down to the public services which are often not complex so may end up testing methods anyway. Where I have private methods this is arguably the tricky stuff that needs testing but may-be not.
No, that's not where the problem comes from. There's a fairly big name in extreme programming who does hundreds of talks that everyone watches and listens to who repeatedly ad-nauseam says you must write a test before writing each single new line of code. He even says you cannot call yourself a professional if you don't do that. The problem is that we have people like that, leaders and speakers in our field, who are talking out of their ass.
As far as I understand in here, 100% code coverage is exactly what Ian Cooper is trying to achieve. The main point of dissension is that his 100% is behaviour testing (but if all your app does is exactly what the behaviours expect it to do, that's 100% of code covered), and actually any line of code happens after having written tests. It basically tells you "don't test implementations, but test every. single. requirement", which is to say, 100% code coverage
i once had the idea of generating a file with a million lines in which each line just adds 1 to a number. at the end i write an unit test: assert(myFunction(), 1000000). And instantly our code coverage goes up 30%!
35:15 "the unit in isolation is the test". This is HUGE! Need a quote from Beck's book.
I must confess that I've been mostly reluctant to the idea of TDD until just now. This is how TDD should have been "sold" or introduced from the beginning.
Many people make TDD sound like if you have to know how the implementation of your methods is going to look like even before writing your first line of code.
Outstanding talk.
21:00 where did it go wrong in TDD
22:30 recommended book
23:53 24:01 *do not test implementation details, test behaviors*
24:15 in *classic modern TDD cycle* , I will write a test before add that method, and that test will govern will that method succeeds or fails 24:30 *the trigger to writing a test in TDD practice is essentially adding a method to a class, THAT IS THE WRONG THING!*
24:50 *THE TRIGGER in TDD for creating a new test is that you have requirement, you want to implement*
26:03 testing the public API
26:13 what is the contract your software has with the world 26:28 (API) will not change rapidly 26:36 *how you implement that requirement (contract, API) is unstable*
26:44 what your software offers to consumers is the *stable contract, that is what you should test*
26:54 not HTTP API
28:46 SUT (system under test) is not a class
classes have public contract too.
You know this is a good video when you look at the like to dislike ratio with the maddening creaking going on through out the presentation
Didn't aged to well :(
The creaking likely comes from something rubbing at the mic or its cable whenever he moves. Testing the recording setup before going on air still is extremely underrated...
@@Oktokolo maybe the conference should have followed TDD principles
@@basileagle314 I wonder, what TDD would look like outside development...
@@Oktokolo before you start using the microphone you talk normally in a large room to make sure they can't hear you at the back
I never understood the benefits of TDD until I watched this talk. This is gold. Ian Cooper seems to be about the only person making sense on this subject
I'd add to the short list: Kent Dodds.
Yea he definitely seems like he has a real understanding from his experiences as a software engineer and did what the book he described did for him - which was condense his years of experience in succint medium.
This comment is duplicated... odd.
IIRC, David Chelimsky's RSpec book made me understand benefits of TDD and trick of writing tests first. Then Uncle Bob's Clean Code series deepened my understanding. Now, 15 years later, I still struggle with using it, because when you work in teams, your colleagues are not disciplined enough to follow such practices closely and once they made a mess, it's really hard to stay on course... :( But it worth the effort!
@@0netom How is using TDD helpful if you already can just evaluate every function in Clojure?
this talk is twice as effective after I have used TDD myself, helped me identify mistakes or incorrect ways of thinking a lot
I’ve had a hard time getting inte TDD and now I realize why. Great talk giving the philosophy and practice of TDD. I’ll read Kent Becks book right away and start testing behaviour. :)
Amazing lecture. Appreciate the hard-experience led talk and that is what gives it immense value, not the TDD/BDD but the higher understanding WHAT should be tested, i.e. the user behaviour. Fantastic, thank you.
Its always goes straight in the box of understanding when someone talk about actual philosophy of subject.
He is talking with experience. Real Gold!!!
THANX :-)
This is absolute gold and an eye opener.. thanks, thanks a million, thanks without end.
lol @ the stick man reference
'Think about your code as an api' was my biggest take away from using TDD process. That was my biggest pivot in the way I wrote code.
The best tech talk I've ever watched. Had a huge impact on my career.
I was stuck for months on this question about TDD: "How are you supposed to write the tests for the methods before you know how the structure will look like?". This answers it completely.
hahahah same man! I was like - Am I supposed to be some sort of genius to figure out in my head all the functions I need to write and before I write them, write tests for all of them, and often those functions change because I refactor the code because the initial implementation was not optimal enough..
And what’s the answer?
@@oscarmvl You don't write tests at small scale: you write test for program's end goal, not tests for implementation details. That way you can refactor internal details without having to also rewrite all of the tests and still be able to find when program starts to fail.
@@BlazingMagpie makes a lot of sense, thanks for the quick reply!
@@BlazingMagpie actually that's a bad idea. You can never test all possible scenarios like that
I clearly hold a minority opinion, but it still seems to me that TDD (and all ideologies like it) comes from a magical place where business/user requirements are clear and determined in advance and never change, and developers somehow not only know in advance how they will develop a thing and what challenges they will face, they also somehow know the best way to do it, such that their design / API will not change. After coding for over 20 years and working many years in automated test, I have yet to meet such a developer. Coding involves exploration. You can't test for what you haven't yet discovered, and only a fool pretends there is nothing to discover.
My disagreement with TDD is not because I think it is slower. It is because I just don't think developers (or anyone for that matter) are good at predicting the future. I suspect what ends up happening is that tests get updated after the software is done, which is fine but it's not what we're being sold.
I have not written (m)any pieces of software where I knew the correct design or even how it would eventually work, before I started. I have a rough idea, but you can't write tests for a rough idea and expect nothing to change. Invariably the process of writing software involves some measure of exploration of the problem space and some iteration over various interfaces and designs, before eventually the API or UI or design begins to take shape and stabilise. Often the first attempt is not the best one, and yet you need to make that first attempt in order to better understand the problem space, and by then you will have gained the knowledge required to implement a more robust solution. Sometimes this process repeats several times. This solution may well (and indeed often does) change the contract with its would-be consumers (but that's ok because they haven't been finalised yet either). It seems to me that if you write the tests first, your lack of knowledge/foresight of the final product will be reflected in the tests, and they will then constrain the implementation unnecessarily, as you inevitably implement the code to conform to the tests rather than to solve the problem you originally intended to.
Is it really so bad to write software, or even just a component, and THEN write the tests now that you better understand how all the pieces fit together? You write the code, and iterate until the design feels right. The interface is also perfected here. Then having understood the thing to be tested, you (or better, someone else who understands the thing to be tested, or better still, both) write the tests that are designed to (a) validate the contract between software and consumer and (b) try to break that contract. Apologies for long rant but I assume someone will correct me and I will learn something new.
> comes from a magical place where business/user requirements are clear and determined in advance and never change
It takes several iterations to get the requirements right, and on each iteration, they will change. Looking at the latter part of your comment, I think you agree on this.
> Coding involves exploration
That's the key, where TDD helps is on exploring the unit (as a use case, not a class) instead of the entire system.
> You write the code, and iterate until the design feels right. [...] having understood the thing to be tested, you (or better, someone else who understands the thing to be tested, or better still, both) write the tests that are designed to (a) validate the contract between software and consumer and (b) try to break that contract.
Yeah, that's it. The difference with test-first TDD is and the iteration is done. Instead of favoring the iterations on design, it favors the iterations on getting the requirements right.
Around 40:20 Ian goes deeper into the Red Green Refactor cycle. First, the requirements (with tests) are a bare definition, something that doesn't work (red), some code is written and to make the test work, at this point the code is DIRTY and SINFUL, there IS NOT good design, but this initial design and this initial definition can provide feedback quickly, it can be validated and with more details come a better understanding (even validate them with someone else as you said), a few iterations might be needed to get to an acceptable behavior (green) the important thing is to not forget the exploration, to no forget the discoveries, the new details of the requirements, and the way TDD does it, is writing them in the test, once there, the design can be improved (refactor), so the design can be changed without changing the requirements (already validated by tests at this point).
With this approach, the requirements details (and their validation, with test) come before the design. The logic behind it, is: What's more important? The system behavior fulfilling the requirements or the design behind it?
In TDD, a change in a test should come from a change in the requirements (the contract), a change in the design (refactor) should not affect the test, in practice, this is hard to get, but knowing it, changes at what level the tests are written and lowers the coupling.
@@raulvazquez3511 good design fullfills requirements, there's no contradiction. And it is not optional, while unit tests restrict how you can (or willing to) change your system. I still fail to see why should I bang my head against every wall in the room (tests red), make random turns (bad implementation), and only then walk our of room suddenly realising where door is (gainining understanding). I prefer to look around, to see where the door is. Then write a test to cover that - the test would be a lot simpler then. Still no answer why I should write tests BEFORE the code. Also I think TDD is a misnomer. It is actually a Unit Test Driven Development, by the way.
@@raulvazquez3511 Also I have question about word "quickly". Why? I think here is the core of disagreement, I think that people who like TDD - are the ones who _feel_ productive doing it. You create problems, you solve problems, all that brownian motion. While if you sit there thinking you are not typing anything - that arguably may not feel like a productive activity. But it is the result that matters.
@@PetrGladkikh the result is what matters, of course. Some people feel safer doing small steps and constantly verifying they are on the right path.
Can I buy you a beer? I can't agree enough. If you do TDD for most projects you will spend your entire time and budget before you have even delivered a prototype to a very unhappy customer.
30:43 *don't write tests to cover implementation details (because those change)*
30:53 *write tests only against stable contract of the API*
31:02 *Refactoring* is the key step, which enables u to *achieve the goal of separating between things like implementation details u don't need to test and the things you do need to test*
The best talk about TDD ever, thanks!
Watched the original version of this talk a few years ago. This updated version is even better.
@Peter Mortensen there you go 😉
It's too bad he was running out of time at 57:00. I would have loved to hear more elaboration on the testing that should be done with the adapters in the Ports and Adapters model. Sure, I can test my application through my ports, but how do I know that my adapters and ports communicate correctly? How do I know, for example, that my HTTP adapter returns a 404 when my core applications throws a UserNotFoundException? Surely, I don't duplicate all my ports tests, but calling them through the adapter interface. It would have been nice to hear a little more about that part.
Since the video suggests that we should write unit tests for modules, not classes. Hence, before starting TDD, the first thing we need to do is to divide the system into modules well.
You don’t though! All you need to start test driven development is a single behaviour you’d like to verify functions as expected
Awesome talk.
Ian brings it to the point.
this is the best TDD talk I have ever seen !!!!
24:50 The trigger in TDD for creating a new test is that you have a requirement you want to implement.
I want to say, that this is one of the most helpful speeches on TDD or software development in general I have ever heard. Just one tiny thing...the background noice, I think it's the floor, drives me crazy.
Very informative. Thanks for posting!!
Creaking intensifies.
sniffs :P
Nearly drove me mad...
NVIDIA RTX Voice ftw.
why did I read this comment? the whole video is ruined now
I had to stop
Great talk. It mostly reflects the philosophy we used at the project I was working on in 2017.
Incredibly useful, especially when thinking through how to implement TDD correctly after starting lean, but your product starts reaching enterprise level
One of my favorite TDD talks ever! Thanks!
Thanks for sharing this talk
23:59 - "dont test implementation, test behaviour"
34:46 - "the test becomes the first consumer of your code"
Can you elaborate?
@@ashrasmun1 I can elaborate! This video is full of terrible advice, and the best reason to watch it is to find out what not to do. If you "only test behavior" you can have a huge pile of shit in your code, which just happens to pass your "behavioral only" tests. That pile of shit will slow you down as soon as you have some new use case that your original "behavioral test" didn't account for, because your code will tend to lean into your "behavioral only" tests. Whatever they let your code get away with is "okay" because the test says so, right?
This, of course, _is the whole point._ We _incorrectly_ think it's onerous to test 100% of our code, every line, so we try to get away with less than that. (If any at all.)
Undisciplined coders will flatly refuse (or strenuously object, under management pressure) to write tests, no matter what anyone says. Consummate professionals who care about what happens as a result of their work will, among other things, test 100% - every line. What this video presents is ~70-80% of the way there. (I mean this literally; you will hit about 70-80% code coverage with what he suggests.)
The last 20-30% of code that isn't tested is where much of the shit is going to hide. Some of the shit may be in your tested methods; but if you bring those un-tested methods under the control of tests, you will soon find it necessary to refactor and get rid of the shit.
I used to do this "integration only" testing myself. It was far better than nothing, but it still left much to be desired. I kept having to go back and fix shit in the untested code, over, and over, and over again. I kept finding bad designs hiding in that untested code, and lax behavior that enabled bad designs to persist in tested methods.
Finally, I got sick of it, and decided to try 100% coverage, just for a month, to see if I liked it. Everything got WAY better. It's so much easier to refactor when everything is tested, and so much harder to get away with a shitty design as well. If I have to go back and fix or extend this 100%-tested code, it's _easy._ Easier, by a wide margin, than that kind of work has ever been before, and I have been doing this for _decades._
@@626Pilot I do not agree. If you do TDD, then you can delete every line of code that is not covered by the test, because it can't be needed. --> 100%
@@BlackLinerer If you do TDD, you hit 100% coverage, so what isn't covered by the test is the lines you throw away. Any line that is important enough to put into production is important enough to cover.
@626Pilot : I'm no expert here, but I wonder. If you do 100% coverage, then when you change the way you implement the behavior, just small change, it doesn't impact the behavior, but it does break tests, I think it's unnecessary.
Since you said it was just 1 month that you tried, I think it's still kind of too soon to tell. However, here's another clip that talk about the bad parts of TDD, the speaker's team was obsessed with code coverage, and it nearly killed their project:
th-cam.com/video/xPL84vvLwXA/w-d-xo.html
And another article having a little part discussing about code coverage:
medium.com/javascript-scene/mocking-is-a-code-smell-944a70c90a6a
```
In my experience, increasing coverage beyond ~90% seems to have little continued correlation with lower bug density.
Why would that be? Doesn’t 100% tested code mean that we know with 100% certainty that the code does what it was designed to do?
It turns out, it’s not that simple.
What most people don’t realize is that there are two kinds of coverage:
- Code coverage: how much of the code is exercised, and
- Case coverage: how many of the use-cases are covered by the test suites
[...]
100% code coverage does not guarantee 100% case coverage.
Developers targeting 100% code coverage are chasing the wrong metric.
```
As I said, I'm no expert, still in progress of practicing TDD. However, I'll go with the more experienced experts since their explanations are more reasonable.
This presenta....CREAK...tion is absolutely ...SQUEAK... fantastic.
This video gave me a few really important insights into the development process. Thanks!
This is the most important thing I've seen this year. Wish I'd seen it earlier.
It's been a while since your comment. How is your code doing today? Still into TDD or alredy cured? 😅
That talk fixes all my problems I had with TDD... Have to read the originals... TY
"Have programmers speak to customers"
With most programmers I've met so far, this would be a great way of either getting rid of your customers or getting your programmers really annoyed.
Yeah, you need an SME of sorts who speaks customer and speaks engineer at the same time.
Sure, let's pile on with the stereotype, good job.
@@thibauthanson7670 - Stereotpes exist for a reason.
Programmers are usually not "peoples people", nor salesmen, nor managers.
There are some scenarios where it can be beneficial to have the programmer join the meeting with the customer, but it is rarely a good idea.
You see, customer relations is a 100% different cattle of fish compaired to programming. You need to know what can and what can't be said in front of the customer, you have to be extremely polite all the time, you have to agree to their scope while changing that scope, etc.
Having the programmer speak to the customer is a bad idea for the same reasons as having a physician as the Minister of Health. It sounds good, but the position won't utilize the very skills that makes somebody a doctor. Some time it works, but not because the human in question is a good doctor, but because he is good at politics.
Customers who reach engineering will be sacrificed to a Canaanite god, these are the rules of the game
Where did we go wrong? When we started making "units" small. Excellent presentation, with a lot to learn from it.
This talk is describing the original TDD concept or so called "classicist" school. There's also the new school of TDD often called "mockist". Both are opposing each other. And this talk is somewhat cliticizing the mockist approach. To get a full picture I'd recommend everyone to learn both concepts and decide for themselves which of the two could better fit their field. What works for microservice architecture might not always work for a mobile app.
What I don't understand from this talk and the classicist approach in general is the refactoring step, which seems completely optional to me. Why would you refactor after something is already working? Because of a smell? How would you know it's a smell? Because you've read smart books? (the majority does not read books).
Also a higher (than a class or function) level tests are more brittle. They do enable safe refactoring without changing the tests, but on the other hand require changing multiple tests should the business requirements change.
Likewise the mockist approach drives not only the implementation but also its design. One could say it's not TDD but DDD: Design Driven Development, as you have to come up with design before writing any unit test. Although you could write a higher level test if you like to, but I prefer writing it last to not have it hanging there red all the way through. And when the business requirement changes you often need to only change a test or two and write more for new units. You can also safely reuse your code, because even the smallest unit is tested.
The best way is to be flexible. Like everything in life, there are tradeoffs. There's a tradeoff between the time it takes to ship your code and make your boss happy and to do TDD. You should know when it's good to apply TDD and when it's good to selectively tests only behaviors and high-level codes. Most of the time you want to strike the right balance between time to market and the time you spend on the TDD routine. But when being duct tape programmer is needed, high level over implementation detail, behavior over implementation, functionality over structure, public API over internal classes, write features or refactor over idly watching reds and greens.
I was apprehensive about the length of the video but once I read “ports and adapters” two minutes into it I knew I had made the right call
Awesome talk! Bit of a mic drop at 1:02 DI Containers being evil, should use factories instead?!?! Love to hear more on this!
Amazing talk!
Imagine if this guy actually had presented some numbers. Bugs per developer, bugs per change request, cost of TDD before and after his approach... We need empiricism.
Yeah, I think anyone likes to replace feelings with evidence. But that's really hard to achieve. Did you have a look in "Software Creativity" by Robert L. Glass? That one's a nice, balanced book on this topic.
You mean numbers that are going to be manipulated, faked and staged for the sake of getting them as soon as they are tracked?
Good talk and i had the same experience with too much tdd. But i still make unit tests for classes that are low level or has no framework dependencies for example all value objects. I think you need balance even though it is less clear for junior developers when to write a unit test and when to write an integration test
It went wrong at the same point as everything else, when the professionals got ahold of it and wrote books on the subject, without ever actually doing it.
I believe it was UCSD Pascal that introduced the concept of unit (which I also believe was an inside the joke - Pascal is also a unit of pressure). A unit in Pascal was similar to the combination of the interface and implementation files separated as .h(pp) and .c(pp) files in C\C++. When referencing a unit you only had access to the interface section. It was essentially the "exports", the "interface", or the "header" section. You had no access to internals. The analogy with exports from Modules in Node.JS works very well, its pretty much a 1-1
Short version: If your tests will be broken by refactoring, you've written bad tests (generally speaking -- there's probably couple of unavoidable exceptions).
Every refactoring is an exception lol it's natural that tests break if you move functions around or create new classes or functions. Otherwise it wouldn't be refactoring 😂
Probably one of the best videos on TDD.
Great lecture! Explanation about the "duck tape programmer" was funny!
Fantastic talk!
Nice speech, thank you
An interesting viewpoint on TDD and i totally agree about how it slows down whole development process (by wrong testing implementation)
Great. Really clear. Thank you.
I'm about halfway through TDD by example and listening to this talk again for probably the fifth time. TDD is extremely straightforward and incredibly deep at the same time. I'm still working on fully incorporating it in practice but it's SO much fun.
Why are you telling us that you like to torture yourself? ;-)
Oh boy, the part where he describes "programmer anarchy" build fast, fail fast, ship small really resonates with what I had been doing!
16:16 What I have noticed is that BDD gives the illusion of better quality, while costing effort and not delivering more quality in a manner that matters. Yes, it sounds nice to wave a pretty test report in front of a Product Manager or Product Owner. Yes, it sounds nice to believe that PMs could be writing some of the tests. While this might generate enthusiasm in a few early meetings, the PM and PO will soon decide they have better things to do than worry about this stuff. Now you have additional code that is your BDD abstraction layer to maintain, for no actual value delivered.
This talk feels very sparse with information. The tl;dw is that you need to test requirements (public API; module exports; the behaviour that consumers expect from your software), not classes or methods. Ian gives few reasons why this can help you thrive in TDD, but doesn't really offer examples, so for most of the talk he ends up repeating the same statement, and doesn't explain it further for those who might not intuitively understand it. He gives off the vibe of a uni professor at a lecture, pacing back and forth, preaching the same idea over and over, bragging that they know this one great truth, and the students don't. Maybe Ian doesn't make it enigmatic, but I only got 5-10 minutes of information out of this.
If your tests are brittle, that means your program is brittle. Trust me, this is a common pitfall. For example, I have a TimeWatcher object with a bunch of tests. The TimeWatcher uses Utc to compare timestamps and does some stuff. My tests reflect this. Later I decide, oh no, I need to use my Local Time. You make the change, and your tests break. This is proof that your program is brittle. What you should have done was construct the TimeWatcher with an object that abstracts away dealing with variations of time. That way, you simply add a new LocalTime object, test it with new tests, and inject it into TimeWatcher. None of your tests break. Your code needs to be open for extension, but closed for modification. If it's not, your tests will let you know.
but then you would have to anticipate, that you might later want to use Local Time instead of UTC?
This is really helpful advice
An awesome talk. Big thank you.
A lot of wisdom in one hour
Great Talk. Use it to convince people to do it right.
24:55 I want to watch this but I'm not sure I'm going to make it with the mic rubbing.
It was about BDD and about how to not overkill ourselves with enforced TDD approach to have 100% coverage for every line of code in just a business code.
Of course there is still a space for TDD if you develop/maintain a very technical code with a stable interface, like a framework (f.e. Spring), or some algorithms, when you really need to test every method and almost every possible combination of interactions and data.
I particularly appreciated the bit about using unit tests to aid in building implementation details and then deleting them. I've got quite a few of those to hunt down and it makes sense, as these are often superseded by higher level tests that focus on the public interface.
If your public interface test is not performed by wetware, then you are doing it wrong.
My man's fought so hard to keep that snot in. Respect
What a find tests a lot useful for is when i need to check how a small piece of code works without running the entire app and navigating to that page, i know there is c# interactive but many times it's quicker this way. You write the code in the test and debug it, easy done.
Fantastic talk
Wow! I formulated my own scratch hacking method (for script writing) and tought it to my pupils, and Phase 1 and 2 sounds very similar to the Q'n'D phase of and the refactor phase of TDD. Thank you! That was interesting indeed..
So, if I need to get some specific object(or a list) from the db based on something (ex name, address etc) is it wrong to mock my repository in order to test a certain behavior of what this class does?
I have an ETL process where I need to test that my processor(module where the business is focused, the T in ETL) passes a specific object to the writer(the L) based on a specific reader input(the E).
What I do is test the processor(not its other 10 -20 private methods, those are implementation details) by inserting a certain object and I assert that the object after the process is the one I need.
like
input = new Foo(param1,param2)
This is where I mock the repository based on my business case.
every BarRepository returns "someFooStringBecauseOfFooParam1)
result = myProcessor.process(input)
assertTrue(result, Bar(someParamsBasedOnTestCase).
of course I could use an in-memory db for that...I just find it easier to create objects based on scenarios I want to test..
This is one of the most inspiring talks I've ever seen. It connects lot of pieces that for many of us are not effectively connected, which leads to very bad development practices. I've shared it to my colleages several times so far and I will certainly keep sharing/reviewing it in the future. Thank you very much!
Amen. For like 10 years i have this fight again and again that this IDE-driven approach of testing (tests focussed on methods) and especially TDD is horrible and responsible for the lack of adoption. It's tedious because while my public interface is pretty much set from minute one, i rewrite my class and method structure behind it constantly until it stabilizes. But that definition of what a unit is, was completely ruined by the industry, by IDE test-class generators and also by most TDD tutorials, which usually are so simple, that it looks like the method is the unit. If some of those tutorials would have added just another class to the unit and spoke about this issue, we would not have this mess today.
Also amen to the Gherkin-tool topic. I have not seen a single project in which the business actually cared (meaning read/write them) about those tests because most of them are not capable or interested to structure requirements precise enough even in natural language and most POs in large companies are just "managers managing external providers" anyways. So at the end, i still like to align automated acceptance tests with the customer, but i don't force my team to deal with stuff like Gherkin unless they really want to (as if that ever happens).
BTW: not complaining about Gherkin, Cucumber etc. Its a great idea, it works very well, but it's extremely hard to get the value out of it because almost every tool that tries to pull the business deeper into the development (BPEL, BPMN, BRM etc.) fails because at the end a developer has to do the job anyways. I love BPMN frameworks, but there are like 10 business analysts on the planet able to change an executable BPMN process without breaking it.
Well there were a lot of Bobs in our department ten years ago. Today all department is loked in maintenance...
He's absolutely right about not testing one class or function in isolation. You test a functionality. This was always super obvious to me. But he spelled out the reasons in a little bit more detail whereas I kind of just implicitly felt it.
I was doing unit test for the last 2 weeks in my current job for the first time . I was mocking to dependecies of the class and also spying on the implemntation . I knew that I'm doing something wrong . Nice Lecture Now I understand how to go .
what do you thing guy about mocking repositories with in Memory database for Unit test ?
35:11 I've spent nearly a decade thinking I was the only one who spotted that
35:11 I've spent nearly a decade thinking I was the only one who spotted that
great talk and well educative.thanks alot for this wonderful video
Easy and clear.
Maybe the reason I hate TDD so much is that I've always been on projects that are rapidly changing with poorly defined requirements. I very seldom encounter a project where I have good enough requirements to write tests BEFORE writing the code. I've also worked on projects where the leads are obsessed with code coverage and code smells, even though the code was still a bug ridden mess. I've spent more time writing tests, refactoring tests, debugging tests, and researching how to write tests than I ever did writing any of the code. The time and cost savings just wasn't there, and still isn't there. In my experience my untested code has been no more buggy than my tested code. You then have the massive bloat that comes along with making every single dependency in a class injectable (interface hell) so that it can be "easily testable", even though most of the dependencies you are injecting will NEVER need to have an alternative implementation during the lifetime of the code. You then have the problem of poorly written tests by inexperienced or low skilled developers, which will throw up false positives or falsely show that something is passing giving you a false sense of security.
This saved my life
Will the creaking ever stop! This is such a good talk and it is ruined by it.
this is gold. THIS is how testing makes sense.
Have you applied anything from this presentation that has changed your code? It's been a while since your comment.
Be conscious of right abstraction level. And the key is to identify, properly, your 'unit' under test.
The reason why developers tend to focus on testing methods, is that many organisations have established some kind of governance that uses code coverage or even method coverage as a central KPI. If you force developers to write at least one unit test for every single method that must achieve a code coverage of at least 75% of every single method, then you cannot complain when someone says "I have to write a unit test, because I have to add a new method".
I wish our industry could get its story right. My understanding of the point of TDD was that it was supposed to show you when your design needed re-working. Tough/weird to test? Refactor. The test drives development.
It is far more subtle than that, though when you start asking *when* to test. Before each feature? Each method/function? Each line? Do you test after a feature, or method, or line? There are trade-offs everywhere. I am far more productive if I can experiment with implementation without ANY tests and then write tests to cover the *behaviour* afterwards. There is no right way... but testing absolutely should be considered an integral part of development. Guesswork otherwise.
I don't see how test-last is beneficial there. With a failing test written first, you're free to experiment with implementation, always knowing that it still works. That's unless you've written a test at too low of a level, so that it's sensitive to changes to your implementation.
It's too subtle and specific to the exact work you're doing to prescribe a "test-first" or "test-after" across the board. For example, when experimenting with frontend implementations it often isn't viable to have a high level test which will not be sensitive to the implementation - what do you hook into? words on the screen or element classes or types.. what if they change (usually do during that early phase)? I'm not going to write tests, regardless of how high level they are, which hook into an experimental HTML structure which is under constant change. It would slow me down far too much as I'd have to keep updating the tests. Again, this is highly dependent on what you are developing, as test-first absolutely does make sense in a lot of cases.
Absolutely right. I think this subtlety is missing in a lot of TDD discussion. I really do not want to feel friction when I want to refactor just because a pile of tests are going to fail if I change one little thing. I use screengrab based regression testing sometimes (ie. the changes I'm making should not affect the output). Other times I'm testing business logic, so my "unit" is a feature (a BDD/TDD mix) and I'm looking for specific values in the DOM, not place in the DOM hierarchy or other specifics.
This video is not about TDD. It is about the half-measures developers resort to when they find TDD difficult to work with. It rewards code that's easy to test, and punishes code that's hard to test. The secret is that code that's hard to test is _bad,_ and needs to be refactored. This secret is not known to these developers. They confuse the steep learning curve for evidence that there is something wrong with TDD. They assume it will _always_ be hard. So they come up with any excuse to skip this test, skip that test, etc. And then they write a blog or give a speech about how clever they are to have done this.
Thinking that real TDD is too hard or not worth it is a _symptom._ It _goes away_ the more you stick with it. Being rewarded for writing easily testable code, and punished for writing code that's hard to test, forces you to give up bad habits. The habits are bad because of the fundamental truth at the center of TDD:
*Loose coupling and testability are the same thing.*
Very nice talk
Basically automate the test that developers do manually anyways after implementation to verify correctness of the feature. That automated test is TDD.
No. TDD is when you write test before you implement the feature.
Tests (unit tests) are a huge topic. There are some benefits and a purpose on them. Two main aspects of TDD:
1. TDD helps us to write code.
2. TDD helps us to maintain code.
The first one is pretty straightforward: fail-pass-refactor. This approach leads us to the coding speed increase. You can measure it. It will blow your mind.
You should focus not on the coverage but on behavior and requirements coverage. If you do that, that leads you to the second aspect: maintainability.
1. Behavior is covered
2. Requirements are met.
3. You do not fear changing the existing code.
4. If requirements are changed, you change the tests accordingly. They will be red (some of them). Make them pass by changing the code.
5. If requirements are changed and you do not change the tests (by not practicing TDD, some other person implements new requirements, etc.), some of them will go red. Fix them. Easy.
It is normal when some tests go red. Treat them AS A PART OF A PROJECT! You should maintain them along with the main codebase! They're a part of your program!
P.S. Do not ever comment out existing reasonable tests unless you're a douchebag.
Test Behavior not implementation, this mindset could be applied to Unit Test or Integration Test. What I was hearing, Testing the API, is more toward integration test, treating it as a black box. This is essential and important, and makes the test easy to write and focus. However, depending on scenario, a complex scenario would result is many complex test scenario and very difficult to debug and know what went wrong. Imagine manually testing the API and you get some error. Do you know immediately know what went wrong most of the time? When practicing TDD and really changing it minially and incrementally, you might know what went wrong, but if a developer writes many codes, implementing many logical units then runs the test and suddenly many test scenarios failed. Or, the code already completed, the requirement didnt change, but the devs went to refactor to improve the codes. This could be very risky because single point of failure can result in many test scenarios failed.
Furthermore, an API could be doing many checks, from other services or databases. How could the developer test without knowing what to mock or what test data to prepare? For example, the developer could be working on Order requirement which would need to ensure that the product exists, is active and probably several other criteria or condition that's more related to the Product availability, which maybe implemented in another requirement and handled in Product Service. Since the current requirement is focusing on making an Order, if one don't mock the Product Service, a lot of things can go wrong and fail the test which cause the developer debugging for no reasons. This often happen for Manual Test before Test Automation, we develop our feature, deploys them, do some manual testing and simething went wrong, we have to spend time to trace and debug and found out it has nothing to do with out code, there was condition in another service which resulted in the unexpected behavior.
Furthermore, we used to have comolex scenarios for a requirement where there are more than 10 complex precondition checks before the process can be allowed to perform, that is just the precondition checks and lets say fir 10 preconditions, there would mean at least 20 scenario for success and failed scenario each... But wait, in we're testing at the highest level, it would also mean, we have to consider any possible dependcy on the preconditions or chains/combinations, which means the scenario would've easily been more than 20 and complex to even handle and think. Even with manual testing, we would have to prepare all the possible datas before hand and test each scenario, at the same time not to make ourself overwhelm and confused, especially when something fails, we went to debug, and make some changes, and something else fails now. This is real experience as we have devs who prefer to write high level test, the test itself ishas so many combination of scrnario that reading it is scary and confusing, the dev ended up spendong more time to debug and fix than completing the requirement which was to add addtional precondition to the existing precondition. It was a big mess.
What we do then instead is to write unit test on each precondition, thay means a t method level. By writing at unit or method level, the focus is so much smaller and simple, yes it requires mock but we only needed to mock for what we wanted to test, we dont need to know the entire state of the object or record. The concept is just similar like how we should break stories smaller, make things small and test it and we should test behaviors.
Lastly, just saying testing API is incorrect. If you're writing a shared or reusable method which others will be calling it. Should you ensure its tested and behave accordingly? The requirement does have to be coming from the customer.
Similarly when working as a team delivering single user story, everyone maybe delivering parts of the feature and wouldnt it be better to ensure each parts is tested and behave according so that we would prevent something went wrong when we integrate the codes? Saying only testing API is very much about single developer mindset and a simple use case scenario. Sure, maybe to some developers they are so good that its not a problem to only test at high level even with complex scenario, but feom my experience and working with many developers, pairing and coaching, putting a bigger scope often create complexity, confusion, messy implementation, which is very difficult to read and maintain, and only testing at high levels, it doesnt require much of clean code, someone could write everything in 1 function and not worry about single rrsponsibility. The idea of making a unit testable is to make it small, single responsibility which makes the test easy to write, small scope and focused.
For my practice and experience, our goal is that we should test the requirement, which usually is on the API which we would write integration test, but we encourage unit testing at lower levels. In the beginning, we find most devs would not write unit or low level test and focus all on integration test which is the API since its easy to write and they have many private methods, but when a scenario fails, they would end up need to know which oart of the code fails, which could actually be more than the requirement have stated, e.g. request body is incorrect, the format is incorrect and many more conditions which are more technical than the requirement stated in the user story and because there are no unit test, high level test may not cover those since its based on the requirements. Furthermore, if we were to add those, the high level test complexity will grow and lastly high level test dont exactly run fast because usually we dont mock dependencies, that would also mean every test scenario we have requires more time to run and process.
I think and strongly believe, behavior test is not just about high level or Api test or integration. behavior is just that, behavior, a feature is a feature, if you provide a library or reusable method or deliver a working unit to another developer, you're delivering a feature that has behaviors, and you can unit test them to ensure they behave correctly. And for other dev using the unit, can choose to mock the behavior when they're unit testing their method when they are using your unit.
E.g. you deliver a unit that provide formatting of a value, or a unit that provide 'isProductValid' that accepts a product object, you can write test at unit level so that you dont have to test at higher level, imagine if there are many API thay needs to validate product, and shoud each of the API repeat the same redundant test??? If a requirement change, then what happens?
We often break and look at if the requirement could or make sense to be tested at lower level, especially if you have multiple layers and worked by multiple developers... Without mocking, each developer will try to test the same scenario over and over again; in fact, this just happen recently, when 2 developers are working together on the same user story, and one developer is working on a lower level layer, written some unit and tested them, and another developer which writes at rhe high level and calling the method written by the other developer, repeat the same test at high level. From the higher level, the test scenario on that method itself has over 10 scenarios. And that layer isnt even the API level yet, now do we have to repeat it again at API level, which would result in more scenario?
3X more "test code" than "dev code" (and other problems). This is where Design by Contract (see Eiffel) shines. DbC is when your test code comes out of the test environment/harness/mock and becomes a part of the production code itself. Where the compiler is smart enough to exercise, report, and utilize your "contract assertions" (e.g. "test assertions") in everything except the production code. Even then-you can include some, part, or all of your "contract" code in various "versions" of production code (if you choose). This means you only need enough "test" (TDD) code to spin-up objects and exercise them. Therefore-test code is smaller, mocks fewer, supporting data structures are smaller and the whole "3X more test code" scenario is greatly reduced in size, scope, complexity, time, and money impact on your project. You may further dice this up by creating many small libraries, where each library lives insulated with its own production product and test environment-such that-small changes to a library do not necessarily "leak" out all over your project code base.
excellent ppt sir
Made it all the way to 41:42 and just realized why the slides are so confusing. All the slide headers are outside the frame and we can't see them.
Nice Talk!
Great talk, this is going to change how I approach TDD. My only complaint is there's a irritating/distracting sound as he walks around on stage. If that could be edited out, that would be perfect...
It's the microphone rubbing against his glasses, not a whole lot can be done about it really
and the runny nose
He described the problems with too granular tests that verifies small implementation details - they do exist and cause pain. Because ideas taken to the extreme are tend to cause problems. So the other extreme - having a single test per externally "observable behaviour" of the system - is painful too. And the speaker didn't seem to mention that (or am I missed it?).
What about combinatorial explosion of execution paths? Having N simple if-s you have 2 ^ N execution paths - are you going to create matching amount of test cases for your behaviour? 256? 512? How far are you going to go on this way?
The other aspect - test are not just to see if code works, they are also play role of the documentation with true examples. That is an important part too - it is not enough to describe a purpose of the class/method in the comment (or a name if it fits) - examples are valuable, especially for the edge cases.
It is always a balance. You can't just run away from the thing that hurt you as far as you can - you'll got other problems there.
"What about combinatorial explosion of execution paths?"
You get this problem when you don't stick to SRP.
@@MrMartingale1 probably. But you also get it naturally when there a lot of special cases (typically happens in the UI). You may start splitting them in more classes but that will definitely kills readability due to inadequately high indirection/abstraction level - which also have a high cost that you have to compare with the costs of alternatives (e.g. complex/strict tests maintenance discussed in the talk). Why are you so categorical in your statement? What are your arguments? Please be more informative.
@@IlyaDenisov didn't mean to sound categorical. I tend to be laconic in YT comments. I don't know much about UI development so perhaps you're right.
My first introduction to TDD was through a talk by Uncle Bob. At least, that was the first time I properly understood what "Write the tests that fail" truly meant. I was horrified by it. But this makes the whole process much more appealing. Bob's talk effectively taught "Red-Green-Red-Green-Red-Green", and the whole concept of TDD seemed to suck any amount of passion out of programming for me. It turned a vocation into a chore. It personified the meme about programmers turning coffee into code. This outside-in (London School) approach is far better than Classicist (Chicago School). Although, both can be very useful. What Ian had to skip with shifting gears was that, when you do know the answers to the problems, and you're just writing tests to conform to the standards, use London School, and build from the outside in. But, if you hit a wall, and find a problem that really doesn't make sense, then you can go Chicago School, and write classicist tests to develop the implementation step-by-step, shifting the focus from the API to the raw code. In C#, I'd start these in a new Assembly that can be unloaded later, to save the tests for posterity, while not interfering with the "production" test suite. It's always good to have a Sandbox assembly for scratchpad code.
43:48 huh great point. I always get stuck on the analysis paralysis portion these days. I think i need to get Kents book