Your advice about making sure we are measuring what's important rather than what's easy to measure reminds me of the page in your book where you advise that optimizing for the number of lines of code is optimizing for the wrong thing. It's advice I took to heart recently when making the choice between 2 different frameworks. One was more succinct in its syntax but more limited in its functionality. I feel like I made the right decision because of this advice. Thanks Dave.
Thanks, very important points here - speed is everything, if you are not moving fast enough you might not be even able to spot some problems. Since they will be strached in time. Forsing people to change speed is extremely hard indeed.
I would argue that the ONLY time to improve our approach is during development. The idea that we're gonna' do everything right next time all at the beginning isn't just a fallacy, it's physically impossible. Also, your videos are the only ones I can't share with my own thoughts and context, because having any thought whatsoever on these topics like, "Obviously! We need to be able to learn and change as we go!" is very controversial.
You'd be surprised how many times I had this discussion as a consultant with software companies where these middle/program/project/whatever managers simply don't get it. Surprise, surprise, usually in 6-8 months the project would be massively rewritten, or how it happened in the most recent case, thrown away and re-done a second time, of course, with the same managers. I am guessing the result will be very similar as the first time.
In part 'Get People To Think' Dave talks about making change in people's thinking. In my experience to make such a change you have to put in effort. Most often then not large amount of effort! There are situations where you would find too much hustle to make such a change. That scenario is farily common in projects running for a long time. So in cases like these the question is how do you determine whether its worth putting in the effort to change?
I am a theoretical fan of TDD... I just don't know how to implement it in the domain of Infrastructure-as-code or the types of small bash scripts that I tend to write that are primarily device facing. How can I get implement more testing in such an environment? I tend to eat my own dogfood and release it into the development environment so the code gets exercised a bit before going on to production, and I want something better than that. EDIT: I am using terraform for my IAC and have all my scripts in version control, as well as being deployed via automation. I just don't have any tests, other than monitors which are a bit after-the-fact.
You'll need to define test workloads that should test everything you need from your infrastructure. If they work your infrastructure can probably handle the real workload to. Deploying it all in a virtualize environment first as a test makes sense a well. If your target is e.g. AWS, deploy the tests to a different account and then clean it all up after you get your test results.
I've not worked with Infrastructure as code so not sure about the specific problems you might have. Releasing into the development environment is just fine if you can't mock out various devices, services or dependencies, that's what I do all the time with specflow acceptance tests. Obviously, if you can try writing your code so it's abstracted from the underlying hardware then you could get some unit tests around that, but maybe that's not possible or not worth the effort. I'd stick with releasing it into the dev environment for now and try and figure out what you're looking for to know you're done. What are you looking for in your monitoring to know that nothing has gone wrong? Could you assert that in a test? Sometimes it can be really difficult to know what you're actually trying to assert. Even if its just a simple smoke test like is my service up could you write a test for that?
In TDD you test all logic, but not the underlying frameworks such as a SQL database working as you expect. IaC basically only has configuration of real systems, no business logic at all. Testing it in TDD style is pointless. You can validate that configurations lead to the outcome you expect only after deploying the configuration. You can validate that security is handled with automated testing setups and some deterministic config and policy validation scripts. I would run those in step with the development lifecycle your team uses early in the pipeline and use a infra dev environment to test your own work in your own “works on my virtual machines” step of “local” development that does not impact other team members. For example, we’ve usually worked in a way that any commit on an open pull request would do linting checks on the IaC, spin up a new environment with the PR-ID in the name, test deployment of code and apps, run all integration and unit tests and then destroy the environment again. This is a bit heavyweight for code changes, but we found it to be an effective commit check for infra code changes. HTH
@@WouterSimonsPlus Thanks. I had an intuition that TDD would be useless because of that fact, but it is nice to have validation from someone else! I'm doing the other steps already (running in a local test environment, and keeping it in sync with the development pipeline so it goes out at the same time as relevant code changes). It just felt inappropriate to make the claim that TDD is great when I'm not actually doing it myself professionally.
I have done this a few times. One of my ex-colleagues wrote a simple version of xUnit for Linux shell scripts - worked fine, same as any other xUnit variant. The trick, as ever, is to abstract the actual I/O so that you can test the logic without real I/O. That gets you so far. The next step is to write Acceptance tests for infrastructure changes. I once built a deployment pipeline for producing deployment pipelines. It had a simple simulation of a working pipeline with a tiny handful of tests of various kinds and a very very simple app that it deployed.
With continuous integration having only one main branch, how then do you ensure that everyone follows good coding standards, like making sure that their function does one thing and not 10 things, naming variables correctly and also not breaking architecture?
My preference is to assert this on commit. If you write a function that doesn't meet the team's coding standards, it is rejected. More that 4 params in a method, more than 15 lines of code in a method or function, you commit fails. etc.
Brilliant as usual. I'm paraphrasing but Reinertsen states that 'queues are the greatest source of economic waste in product development.' When you look closely this comes down to batch sizing. My take on this is that optimizing for small batches happens at multiple levels in an organization, all the way from IC to portfolio. I would be really curious to hear your thoughts on this, as your thinking always describes familiar things in ways I haven't considered. Thanks!
I was watching a conference talk about using serverless "function as a service" style architecture for long running problems. The solution was the same, reduce the batch size and work in smaller steps. The result was more efficient, and each step was simpler. It's interesting how the solutions to "information problems" are the same in so many different contexts.
Dave, your timing is suspicious. As in "simulation hypothesis-confirmingly suspicious". This month I am transitioning from a green field project on which I introduced a CD approach and moving to a team which I know has two things: Pull Requests and bugs. Wish me luck.
Your advice about making sure we are measuring what's important rather than what's easy to measure reminds me of the page in your book where you advise that optimizing for the number of lines of code is optimizing for the wrong thing. It's advice I took to heart recently when making the choice between 2 different frameworks. One was more succinct in its syntax but more limited in its functionality. I feel like I made the right decision because of this advice. Thanks Dave.
The tshirt this episode needs to be mass produced! I want one with my name on it!!
Thanks, very important points here - speed is everything, if you are not moving fast enough you might not be even able to spot some problems. Since they will be strached in time. Forsing people to change speed is extremely hard indeed.
Thank you very much for everything you do in this channel Dave.
This is exactly what I needed, I will probably re-watch this video a few times..
Thanks!
I would argue that the ONLY time to improve our approach is during development. The idea that we're gonna' do everything right next time all at the beginning isn't just a fallacy, it's physically impossible.
Also, your videos are the only ones I can't share with my own thoughts and context, because having any thought whatsoever on these topics like, "Obviously! We need to be able to learn and change as we go!" is very controversial.
You'd be surprised how many times I had this discussion as a consultant with software companies where these middle/program/project/whatever managers simply don't get it. Surprise, surprise, usually in 6-8 months the project would be massively rewritten, or how it happened in the most recent case, thrown away and re-done a second time, of course, with the same managers. I am guessing the result will be very similar as the first time.
@@ForgottenKnight1 My whole career has been rewrites and being hated by middle managers. I am not surprised in the slightest.
This is exactly what I needed too. Very timely.
In part 'Get People To Think' Dave talks about making change in people's thinking.
In my experience to make such a change you have to put in effort. Most often then not large amount of effort!
There are situations where you would find too much hustle to make such a change. That scenario is farily common in projects running for a long time.
So in cases like these the question is how do you determine whether its worth putting in the effort to change?
You're reminding me of my reading of The Goal
Hi Dave, what are your thoughts on ChatGPT?
I am a theoretical fan of TDD... I just don't know how to implement it in the domain of Infrastructure-as-code or the types of small bash scripts that I tend to write that are primarily device facing. How can I get implement more testing in such an environment? I tend to eat my own dogfood and release it into the development environment so the code gets exercised a bit before going on to production, and I want something better than that.
EDIT: I am using terraform for my IAC and have all my scripts in version control, as well as being deployed via automation. I just don't have any tests, other than monitors which are a bit after-the-fact.
You'll need to define test workloads that should test everything you need from your infrastructure. If they work your infrastructure can probably handle the real workload to. Deploying it all in a virtualize environment first as a test makes sense a well. If your target is e.g. AWS, deploy the tests to a different account and then clean it all up after you get your test results.
I've not worked with Infrastructure as code so not sure about the specific problems you might have. Releasing into the development environment is just fine if you can't mock out various devices, services or dependencies, that's what I do all the time with specflow acceptance tests. Obviously, if you can try writing your code so it's abstracted from the underlying hardware then you could get some unit tests around that, but maybe that's not possible or not worth the effort.
I'd stick with releasing it into the dev environment for now and try and figure out what you're looking for to know you're done. What are you looking for in your monitoring to know that nothing has gone wrong? Could you assert that in a test? Sometimes it can be really difficult to know what you're actually trying to assert. Even if its just a simple smoke test like is my service up could you write a test for that?
In TDD you test all logic, but not the underlying frameworks such as a SQL database working as you expect.
IaC basically only has configuration of real systems, no business logic at all. Testing it in TDD style is pointless. You can validate that configurations lead to the outcome you expect only after deploying the configuration. You can validate that security is handled with automated testing setups and some deterministic config and policy validation scripts. I would run those in step with the development lifecycle your team uses early in the pipeline and use a infra dev environment to test your own work in your own “works on my virtual machines” step of “local” development that does not impact other team members. For example, we’ve usually worked in a way that any commit on an open pull request would do linting checks on the IaC, spin up a new environment with the PR-ID in the name, test deployment of code and apps, run all integration and unit tests and then destroy the environment again. This is a bit heavyweight for code changes, but we found it to be an effective commit check for infra code changes.
HTH
@@WouterSimonsPlus Thanks. I had an intuition that TDD would be useless because of that fact, but it is nice to have validation from someone else! I'm doing the other steps already (running in a local test environment, and keeping it in sync with the development pipeline so it goes out at the same time as relevant code changes). It just felt inappropriate to make the claim that TDD is great when I'm not actually doing it myself professionally.
I have done this a few times. One of my ex-colleagues wrote a simple version of xUnit for Linux shell scripts - worked fine, same as any other xUnit variant.
The trick, as ever, is to abstract the actual I/O so that you can test the logic without real I/O. That gets you so far. The next step is to write Acceptance tests for infrastructure changes. I once built a deployment pipeline for producing deployment pipelines. It had a simple simulation of a working pipeline with a tiny handful of tests of various kinds and a very very simple app that it deployed.
Could development activities not using tools like chatGPT and copilot be called manual development?
With continuous integration having only one main branch, how then do you ensure that everyone follows good coding standards, like making sure that their function does one thing and not 10 things, naming variables correctly and also not breaking architecture?
My preference is to assert this on commit. If you write a function that doesn't meet the team's coding standards, it is rejected. More that 4 params in a method, more than 15 lines of code in a method or function, you commit fails. etc.
Brilliant as usual. I'm paraphrasing but Reinertsen states that 'queues are the greatest source of economic waste in product development.' When you look closely this comes down to batch sizing. My take on this is that optimizing for small batches happens at multiple levels in an organization, all the way from IC to portfolio. I would be really curious to hear your thoughts on this, as your thinking always describes familiar things in ways I haven't considered. Thanks!
I was watching a conference talk about using serverless "function as a service" style architecture for long running problems. The solution was the same, reduce the batch size and work in smaller steps. The result was more efficient, and each step was simpler. It's interesting how the solutions to "information problems" are the same in so many different contexts.
❤️❤️❤️
Dave, your timing is suspicious. As in "simulation hypothesis-confirmingly suspicious".
This month I am transitioning from a green field project on which I introduced a CD approach and moving to a team which I know has two things: Pull Requests and bugs.
Wish me luck.
Best T-Shirt yet by far! :D + the talk is amazing as always @dave farley