Don't Build Perfect Software
ฝัง
- เผยแพร่เมื่อ 27 ก.ย. 2024
- Many of us strive for perfection in software development, is this really an appropriate target to aim for? Perfect code is an illusion. Software engineering is an empirical discipline, and the software we create is only ever correct in a narrow context, from a technical and a social perspective. So if "perfect software” is the wrong target, what’s the right one? How do we make progress in a less-than-perfect world?
In this episode, Dave Farley, author of Continuous Delivery and Modern Software Engineering, explores the idea of perfection in software, the role of mathematics and what it takes tp create great software in the real world.
-----------------------------------------------------------------------------------
⭐ PATREON:
Join the Continuous Delivery community and access extra perks & content!
JOIN HERE ➡️ bit.ly/Continu...
-------------------------------------------------------------------------------------
🔗 LINKS:
Dave Thomas' talk is not currently publicly available.
Godel’s Incompleteness Theorem ➡️ www.quantamaga...
The case for mathematically probable software ➡️ www.infoq.com/...
“Computer Science is not Maths” ➡️ pron.github.io...
If you have a Netflix Account - “Downfall the Case Against Boeing” ➡️ www.netflix.co...
“Apollo 1201/2 Alarms Explained” ➡️ www.discoverma...
-------------------------------------------------------------------------------------
📚 BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is available as paperback, or kindle here ➡️ amzn.to/3DwdwT3
and NOW as an AUDIOBOOK available on iTunes, Amazon and Audible.
📖 The original, award-winning "Continuous Delivery" book by Dave Farley and Jez Humble ➡️ amzn.to/2WxRYmx
📖 "Continuous Delivery Pipelines" by Dave Farley
Paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd...
NOTE: If you click on one of the Amazon Affiliate links and buy the book, Continuous Delivery Ltd. will get a small fee for the recommendation with NO increase in cost to you.
-------------------------------------------------------------------------------------
CHANNEL SPONSORS:
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ bit.ly/3ASy8n0
Octopus are the makers of Octopus Deploy the single place for your team to manage releases, automate deployments, and automate the runbooks that keep your software operating. ➡️ oc.to/Dave-Farley
SpecFlow Behavior Driven Development for .NET SpecFlow helps teams bind automation to feature files and share the resulting examples as Living Documentation across the team and stakeholders. ➡️ go.specflow.or...
TransFICC provides low-latency connectivity, automated trading workflows and e-trading systems for Fixed Income and Derivatives. TransFICC resolves the issue of market fragmentation by providing banks and asset managers with a unified low-latency, robust and scalable API, which provides connectivity to multiple trading venues while supporting numerous complex workflows across asset classes such as Rates and Credit Bonds, Repos, Mortgage-Backed Securities and Interest Rate Swaps ➡️ transficc.com
LaunchDarkly is a first-of-its-kind scalable feature management platform that allows development teams to innovate faster by transforming how software is delivered to customers. We want to show you what we're all about. Book a demo to see our platform in action! ➡️ tinyurl.com/CD...
I am a musician, amateur mathematician and a bit of a hobbyist coder at this point. I write terminal applications that help me with music-making. I am thinking of taking my coding skills to the next level and it seems that this channel is a treasure trove of good advice.
6 years into my career... I wish I found this sooner
Haskell will make you a better amateur mathematician. Lisp will make you a coder wizard.
Re the Apollo flight computer, a similar story applies regarding the Golden Gate Bridge in San Francisco, CA. The designer first calculated the maximum stress predicted under the worst case of load, wind, etc. Then, he designed the bridge to safely handle 3x that amount of stress. And, it is still standing 85 years later.
My synopsis of yet another very fine video: Strive to write code that is both readable and robust.
The last minute contains great summary of this topic. Previous 15 minutes were about explaining why building software is not like math and thus can't be perfect.
I'm always in two minds about pragmatic and well-crafted code, code that just works and is extensible/maintainable. Eg. my new team tends to put a lot of things in controllers, skipping unit testing to integration tests. It's very pragmatic, and it works, but now I spent a lot of time adding one field in event and API, and the wrong database has been chosen. CQRS would probably be a good choice because data representation will change for sure, but we have a simple CRUD-like service instead. Maybe I'm usually overthinking my solutions before implementation but I believe that 'measure twice but cut once' is cheaper than taping and maintaining it. How to find balance?
I guess it depends on how you define "well crafted" and "maintainable", if the code is not tested and hard to change, I don't think it qualifies as either. The ability to change your code easily and safely defines its quality to me.
For game design - a simple 2D platformer provides multiple of values that you can tweak that will affect how the game will feel. I claim that you can't determine those values beforehand. They will be formed once you start to play test it. How fast your character accelerates to the maximum speed, when the character jumps after you press jump, how much gravity affects to the jump, how much you can control the character during the jump, what happens when you release the jump button, what happens when you're no longer pushing the right direction. Other channel - Game Makers Toolkit made a nice tutorial about this that anyone can try out.
The aeroplane argument can be refuted at an even more basic level: basic Navier-Stokes equations, the foundation for Computational Fluid Dynamics, cannot be solved analytically.
🤯 Grrrr.... This hits a NERVE!!! BIG TIME!!! I'm soooo very annoyed when talking to someone and I suggest that we follow the SOLID principles, and avoid the known pitfalls of software development... they are new to these concepts... so what inevitably happens is that it gets twisted up in their mind and they think that I'm suggesting writing "perfect" software that must be "properly designed"... in other words: "YOU MUST WRITE PERFECT SOFTWARE" and it wasn't at all what I was talking about: SOLID, Control Dependencies, avoid tight coupling, gain high cohesion by separating into modules, avoid global state, make testable, etc. For me (and most people I'd hope) this is basic design and self evident... but then some people, and they can be very seasoned programmers, does this take on such an uphill battle? I understand that people have their preferred way of working and this might be out of their comfort zone, but it seems like it's a bridge too far for many... This is a paradigm shift... I get that, but once you see it, you can't Un-see it! It is TRUE: We can never, and never should try, to write perfect software. While this revelation might be new to some, It was obvious to me many years ago, so I would not even suggest it, nor have I ever suggested we could. I have to correct that (why should I need to in the first place?) But what we can achieve is to write BETTER software that is more aligned with the GOAL... which is provide value to the end user.
I always write perfect code, Dave. 😥
Best channel ever (or at least a model of one)
P.S. Definitely going to show this one to my friend whose favourite quote is:
"All models are wrong but some are useful"
- George Box
I think TDD can be seen as sampling the nearly infinite volume of state space a program can find itself in. If you could test all inputs exhaustively, or prove that all unsampled inputs are equivalent to those you have sampled (which is impossible when you run into a halting problem), you would get a formal verification proof. TDD samples state space in a very sparse manner, and cannot offer proof, but in practice large swaths of state space produce the same behavior. If a function takes a 64-bit integer parameter, it is rather uncommon that such a function would fail *if and only if* the parameter is 2^42+1, while passing for all other possible values for a 64-bit int. Therefore it is sufficient in practice to only test a tiny fraction of the state space, which is fortunate as even 1 KiB of memory has more state space than the number of electrons in the Universe.
There is a practice called Domain Driven Design which includes a test method that applies a range of test inputs within a field of possibilities.
There are proper mathemathical tools that are way more effective than TDD exactly because they are not sampling, see TLA+ for a good example. It grants a much higher level of assurance than any TDD.
Great video! i've been struggling with this problem all year long and when i said, F**k it, it will never gonna be to my liking, then progress started to show up, i'm aware that my code has code smells, but i'm willing to understand that the product is not perfect and will be increased and upgraded as it goes on with time, being used by real users and learning more about this complex, almost headaching, but beautiful profession such as software engineering.
I fully agree with you. Can you comment or produce a video that covers the 'First time right' principle?
A good suggestion. thank you. I'll add it to my "to do" list.
Popper famously thought that all one can do is refute and "fail to refute". (Or falsify, etc.) His friend and fellow philosopher of science Mario Bunge points out that corroboration is also valuable and doable (and done!). Unit tests always struck me as being "Popper only", which makes them necessary but not sufficient. I wonder how automated versions of corroboration would work?
I guess I am more of a Popperian. I don't think that corroboration is on the same level as falsification, and so while I agree that good TDD is about falsification, I think that bad unit testing is about attempts to prove an implementation good.
The difference, for me, is that if all my tests pass, I don't assume that my system works, but I know that it works to the degree to which I thought of ways that I could check that it works. So that corroboration is valuable, but is not definitive. When a test fails there are two reasons. My test was wrong, or it is falsifying my system. Both of these are good things to know, and to explore. So, for me, Popper wins 😉
Great video Dave! Something different, and right on point IMHO. Thank you & Happy New year for 2023!
Thank you! Happy New Year!
One way I think of design is this: strive for maximal clarity, not ultimate perfection. TDD makes the prior very much easier to achieve.
'Sensible pragmatic solutions to problems.'
Well that's what we all work towards. I don't know of any other ideal.
Great Video! I think you explained a recent point Allen Holub was trying to make on twitter. Test on TDD are not for assurance, but they are for design?
I am a bit lost though, shouldn't we also add test for edge cases as we are developing the code, and any other tests that might offer more assurance? Shouldn't we include this guideline on the methodology? Maybe TDD+?
Test "edge cases" when they change the behaviour - "what should the system do when 'X' happens?", otherwise, don't really. This isn't really TDD+ this is TDD.
True, but because of some mathematical tools we have, people have no excuse for most memory mistakes anymore.
Hey Dave. I really liked your video. Is the Dave Thomas talk you mentioned open to watch?
I am afraid not, it is currently only available to registered attendees of YOW! I think it will be released at some point though.
TL;DW gather as many assumptions as possible, question those assumptions, and engage in meaningful conversation with customers BEFORE writing anything that is easy to change when the assumptions/use cases change and go by Murphy's law often as anything can/will go wrong
what is your opinion about akka and the actor model as well as the reactive manifesto ? when and where
should it be used ?
I am a big fan of Actor based systems, and think that they have a big future. Two of my co-authors of the reactive manifesto were the developers of AKKA.
I talk about these topics in these videos:
"Reactive Systems" th-cam.com/video/eRxLfUIMJwk/w-d-xo.html
"The Next Big Thing" th-cam.com/video/-IZlOPciSl0/w-d-xo.html
@@ContinuousDelivery I guess I am watching the right channel lol. Thank you so much for this content.
Excellent video to end the year.
Many thanks!
Thank you
"Maths is a modelling tool and not an universal truth" - great explanation in simple words.
Exactly I think the same and now I am the most idiotic person in my gang🙂.
Yet is is not how math works. Math is a truth. It can define things that are not modeled. Several things were discovered because math demanded them to exist.
Do you think your book is suitable for beginners?
It is not going to teach you to program, but I think it will help even beginners to understand what is important in software development. It is not a deeply technical book, in some respects, I guess it is about the "philosophy of software development" in part, but with a practical edge to it - not sure that helps! 😳🤔
@@ContinuousDelivery absolutely, thanks for that answer. to be sure I'll check it out myself. Just like software development we need to experience in practice to be sure 😊
Yes it is. I've read it. The sooner you can get good coding practices into your head the better. I would say that because it's not full of code examples it's a perfect book for a beginner. As Dave points out, there is a difference between coding and engineering and you need to know both.
Right now I'm trying to debug someone's code that has a function that is 500 Ines long. 🤦♂ To be fair, it may have been written in the late 90s where I believe there were some practical limitations. There are no tests. No documentation. Unintelligible variable names like "jskPr2". If the person who wrote this code read Dave's book (and cared) they would never have written this garbage.
While I agree for most software that it doesn't make sense to build perfect software, I think there are places where the perfect software (as possible) is the best. Those areas mostly the foundational software like Operating Systems, compilers and drivers. Those software has very well defined semantics most of the time and therefore it is possible to prove them. And when possible provable software is better because simply no useful program fits a human's head, and therefore may contain mistakes no matter how much we test it. A program that is proven to be correct is very valuable because there is simply no way that it will behave wrong.
In reality there is no software that has been proven down to the hardware level since we have lots of layers and at physical layer it's simply impossible to prove things, but nevertheless by proving certain aspects of a program we can get rid of certain class of errors completely, which helps with writing useful software because less things get in the way. A practical example of this how security vulnerabilities went down in Android when they started using Rust: (security.googleblog.com/2022/12/memory-safe-languages-in-android-13.html). Rust doesn't prove programs completely but it proves some aspects of it and it makes the language very pragmatic for system software.
I aim to write software that meets customer expectations...
I will always prefer to road test my code than to formally 'prove' it somehow
It's the opposite for me
That's one of the reasons "automated testing", specially in software engineering, doesn't exist.
There is testing with the help of tools that do automated checking, but the tools are just part of the testing activity.
Fuzzers are arguably a form of automated testing
I’m trying to fully automate JavaScript unit test creation because I think it’s possible to generate exhaustive tests, but there’s no way I could ever generate behavioral tests. I will only ever be able to automate what this channel calls wrong. And I fully understand why they are considered wrong too.
Tldr: you cannot build perfect software, because humans are imperfect. Accept imperfection and failures rnd deal with them.
I cringe every time I hear the British "maths" with an s for some reason.
I cringe every time I hear the American “math”, we are all victims of our culture.
I cringe because I'm not American nor English and I've never noticed the difference or that anyone cares about it :)
Great video! As a tech lead who used to struggle with the pressure to build perfect software, I can definitely relate to the points you made. It's important to remember that no software is ever truly perfect, and it's better to focus on delivering value and meeting the needs of your users rather than striving for perfection.
I love sharing mentorship advice on my channel, and I think it's important for aspiring tech professionals to understand that it's okay to make mistakes and learn from them. Building software is a collaborative process, and it's important to work with your team and stakeholders to find the best solution for the problem at hand.
so you force concepts up people's a** to exercise your title?
When you start a new project, you want it to be a success, and whether we can make it perfect or not, we usually want it to be better than the last project we worked on. I've put together a convenient and compact 'how-to' guide which you can DOWNLOAD FOR FREE here ➡ www.subscribepage.com/new-project-guide
Can we at least move toward software tools and programming languages that prevent the worst kinds of failures that cause costly damage, security nightmares, and sometimes deaths? Things like reading uninitialized memory, buffer overflows, race conditions, deadlocks, and null reference exceptions. The "things will never be perfect" argument is well stated, but we should not be complacent about mathematical rigor. Certain classes of software errors can and should be eliminated, and the best possible way to get there is to use development tools that make them impossible.
Agree, we could eliminate like 90 % of this problem.
Rust?
This is why many of us C++ developers are flirting with Rust. The language is not without faults. It's not perfect. The community has some signal/noise issues. The loud "functional" zealots are naive. Despite it all, the strong memory safety guarantees + no garbage collection, and opt-in low level access when you need it, makes a compelling value proposition for new projects that require C++ level performance.
It can be done, but the language would have to incorporate all of the concepts you've mentioned as primitives.
We have done all of these things many times, they have costs though. When Java was introduced., the company where I worked we were early adopters, the simple step of managed memory and garbage collection increased productivity on our, formerly C++ projects, but 25%, but that came at the cost of performance. There are lots of other languages that address these problems, but that isn't enough either. Software development is much more than programming languages and tools. It is about how we solve problems, and for that we need to work in ways that allow us to sometimes make mistakes, because we always will.
Software serves a purpose. The gripes I have so far are:
(1) premature optimization. You can't always know or anticipate where the journey goes
(2) testability. I have to deal with a lot of 3rd party software that is hard to test, i.e. you can't test functionalities/ business logic isolated from some frameworks/ 3rd party components. Some of our contractors still don't do TDD
(3) what users need. It's still very common to not deliver something that users need. The organization decides what users get. It gets worse when there are 3+ systems with overlapping features and users rather stick to excel sheets and reduce interactions with all systems to a necessary minimum. Law enforcement is prone to this.
Dirac mentioned that his engineering training taught him to tolerate approximations.
Thank you Dave for creating this type of videos. I love when you go into philosophy to argue your points. Highly enjoyable and thought-provoking content.
Glad you enjoyed it!
Never heard of "perfect software" term or target or aim
I sometimes see unachievable perfection mentioned to undermine actually achievable ways software can be more perfect. Like mathematical proofs can be built into languages to disallow out of range exceptions and seem like a nice thing we should have. Async operations could be typed in a way that ensures error handling with a non-empty list of error messages. So I agree that we iterate on imperfect things, but I worry at how often people dismiss tiny little safety mechanisms by labeling them perfectionistic.
And I can’t help but nitpick that something can be both “perfect” and a subset. Being only a subset doesn’t make the subset imperfect.
But I’m also driven mad by pipeline or architectural board guys occasionally who design checks that don’t allow smaller achievable incremental changes. Like requiring a huge refactor to remove preexisting duplications, or banning temporary comments to disable TypeScript when I’m increasing its strictness and trying to phase it in.
It depends on the tradeoffs.
And yes, you can have a perfect subset of your project if its designed a certain way, and requirements changed in a certain way. But requirements changes tend to make swiss cheese out of code, meaning it isn't a perfect subset, its many imperfect subsets.
If we look back in time then today nothing is perfect anymore, everything is bloated and slow.
We have PCs with 8+ cores and 5ghz and many websites are stuttering.
20 years ago we had games like FF7 or Diablo 1 that ran on 60 MHZ CPUs w 2 MB Ram and 1 MB VRam.
We have gotten very lazy in programming with our bloated frameworks.
Not being perfect is not an excuse for not getting the basics right, like writing clean code, and TDD. Getting the basics right doesn’t make the code perfect, it makes it maintainable. Neglecting the basics is just professional negligence, not pragmatism. Sadly the former (negligence) takes place under the auspices of the latter (pragmatism) far too often.
Depends on what your definition of perfection is. Sometimes the software must be exact to within certain limits.
very deep toughts
How does it apply to cryptocurrencies which are meant to be "mathematically proven" etc.? Or cryptograpy in general, can we be sure some algorithm is proven to be secure?
Cryptography is first proven on metaphorical chalk boards long before a code implementation. Both must then undergo serious rigours, attacks both theoretical and practical. And out the other side comes an implementation with a list of of caveats. E.g.: this only works while factoring large primes is difficult, with a definition of difficult being 2^n computer hours for a modern desktop machine, and be prepared to increase the key space when modern is 2^m faster than old modern.
And then someone finds a flaw on a metaphorical chalkboard and the whole thing goes away.
Cryptocurrency had similar history but then you just need everyone to agree to a block chain and apply a dollar value to the coin. A bad implementation or contact can be buoyed by good marketing and hype, and a good implementation can be conversely undervalued.
Actually both cryptography and cryptocurrency need consensus to be used, so no real difference.
Afaik cryptography in practice isn't generally proven to be secure. But what's often done is take some building blocks, which are not proven to be secure but are thought to be secure because they are well known and we can't find any published details of any insecurity, despite many researchers looking. Then we prove that *if* the building blocks are all secure, the combination is secure.
I rather think in terms of a different definition of perfect:
Perfect = Good Enough
We know that the program will change.
We know that there are thousands (milions? infinite?) solution variations that works almost equal.
We know that the scale of effectivity of solutions is exponential.
We know that the difference between two solution is the same scale is negligible.
So:
Any solution good enough presents no significant improvement compared against any other good enough solution.
So in a way, all the good enough solutions pertain to the perfect solution.
So, perfect is not one set of bits, perfect is a large humongous solution space region.
Working with some of these assumptions is really what I am talking about, but I don't think that "we" as an industry behave as though we know these things. Most orgs that I see find it hard to make change. Most work in ways that seem to assume that once the code is written, there will be no need to return to it and correct it as we learn more. These aren't signs of working in ways that suggest that we do really know that "things will change".
1:25 start
1:58 David Thomas: striving for perfection is an inappropriate goal for software
3:21
Thanks for keeping our thoughts wander
this year thought about this, and looking the etymology of word Perfect , we see that came from perfectus(Latin)
"completed, excellent, accomplished, exquisite"
A radical way to put it is , if it's Perfect its dead
I try to place my personal projects at edge, between impossibility and perfection
I just finished Java internship. This channel is awesome, even though I don't fully understand everything he talks about. But it gives me confidence by providing me with information by which I can evaluate my own progress and see where I need to improve and in which direction to move forward
Any application is capable of an infinite combination of behaviors and conditions, any of which might result in something undesirable to the user or business. Risk comes from the unanticipated more so than the anticipated. We test for the anticipated, but that is usually simple testing which is only the start of understanding product risk. Very good testing applies techniques and methods that help us observe things we did not anticipate. It exacerbates conditions that amplify mistaken assumptions. It targets areas where thinking seems unclear, uncertain. It focuses on patterns of error and accident in human judgment, analysis and performance.
🤟
Question - I have never worked on a project that measured test coverage. Is it important to a project? It never seems to be a priority.
I know you talk about how it can be gamified by Devs.
Test coverage is important, but using test coverage scanners as a rule for a project can often be more of a problem than a solution. Test publicly exposed functions. Set your test coverage rules for that. Once you start requiring tests for private methods, you're probably doing more harm than good
Principia is pronounced "pruhn·si·pee·uh". Even if you try to go classical latin it is "prin-chee-pia" and in no way it was like you said "prin-kee-pia".
One of the wonderful things about programmers is our affinity for pedantry, especially in respect of incidental issues that neither add to, nor detract from, our understanding. There are many pronunciations of principia, depending on whether we are applying the classical Greek or Latin, later Italian or common English use of the word. I am happy for you to use your version 😉
Coincidentally I just started reading Clean Architecture by Bob Martin. He talks about this in the early chapters. About how maths can be proved. But that software is science, where you can only test that it doesn't work. And when you aren't able to find a test that shows it does work, it's probably good enough and working. But there's no way to "prove" that it correct or right.
Interesting
This feels like a critique of functional programming lol
Well I think that some functional programmers are amongst the group of people who attempt to, in my view wrongly, mathematecise programming. It certainly has its place, but that isn't all there is to programming.
Interestingly, Dave, I chatted to Tim Mackinnon about TDD and he talked about using that to test the hypothesis developed with CRC cards. I liked that. I've wondered whether, through mock objects, we could converge to the hypothetical architecture using, say, fake it til we make it, triangulation etc.
I hope you get to interview Tim Mackinnon - he has lots of great ideas worth chatting about. He became my mentor in 2014 after a talk by Jason Gorman on the subject of mentoring. It made a huge difference to my career.
Yes, I think that is a reasonable take. I think it goes a bit deeper than only validating the CRC assumptions, but it certainly does that. I think that TDD in general amplifies our experience of our design, at all levels. So by driving the development from tests, we end up with better designs.
I have a long list of people to talk to, but Tim is certainly on it, thanks for the suggestion.