I don't like "magic" like this, where they hide actual implementation. I would consider it good if it generated dockerfile the way like source generators do so I can view it and really understand what's going on.
isn't the point of abstraction in programming to "hide magic", so that you can worry about more important things? If we had it your way, we would still be wasting time looking at zeroes and ones
@@dandayne i did not say they should be no more higher or easier abstraction, I simply wanted them make generated docker config accessible like source generator does
Super cool to see my team's feature on a Nick Chapsas video. Tons of engagement in the comments that I'll go through and respond to, but to cover the common questions: * No, we don't want to replace docker files for everyone! We want to reduce friction and get rid of the 80% docker files that had no value. If you have specific needs (native libraries, integration with tooling that expects Docker files), then definitely still keep that! Just like NoSQL means 'not only' SQL, there's a broad ecosystem of container-adjacent tools that work like this new tool does (e.g. ko for golang, and jib for Java). This is just that, for .NET! * It's really really fast in comparison :) makes me happy every time I build a container. * It does work with alternative container runtimes like podman and nerdctl. I have explicitly manually tested these, and they remain part of our goals. I'd like to expand our support for them in the future. * I think if project files had better intellisense it would be a nicer experience overall. I have plans in this space (I'm also the MSBuild PM), but nothing concrete yet. Glad you had a good time with it, Nick, and thanks for the highlight :)
The purpose of having the dockerfile in the project directorx is that you can build multiple projects from the same build context. Say you have two projects in a solution, you can now build both using docker build -f project1 . and docker build -f project2 .
@@errrzarrr This isn't a step backwards. Like you said, just because you can, doesn't mean you should. You don't HAVE to do this, you can still have your dockerfile if you want to. This will make docker easier to get into for new devs and smaller projects, especially microservices will benefit from this. We have multiple microservices that are all in different repositories, we're using the basic dockerfile that builds basic aspnet images. If we use this we don't have to have an extra file that most developers won't even know how to edit, we can just manage it via the csproj. If any project gets to a point where the basic build doesn't work anymore we can just add the dockerfile back in and everything will work fine. A step backwards would be if Microsoft forces this down our throat and somehow breaks any builds that run through a dockerfile
@@deondupreez4713 "most developers" don't know how to edit a csproj file as "most developers" don't use c# but that's besides the point. This still requires knowledge of docker, it's just a different syntax and location for the configuration and is utterly unnecessary. It will only lead to more Visual Studio ignorance/dependence.
Whether you are managing the csproj file or a docker file, I see no difference. I do like what MS is doing and I appreciate it, but I am a fan of having a separate Dockerfile, also sometimes you might have 2 or more Dockerfiles in a project for different environments that you'd like your application to run in.
Yeah, when a .NET developer jumps to another stack. They might get shocked because there are so many things they didn't see in .NET, like Dockerfile because it's hidden. What if I want to install a package before or run something else before starting the app? How can I do that without the Dockerfile? Creating a base image?
@@viacheslavsamodelkin6944 It's about increasing dependency on Microsoft, or more so strengthening their reputation of heading in a more innovative direction. Plus, this may end up being a more customizable and native solution to container management that Docker is not able to provide since it is specific to .NET. It is a smart move on their part from a business perspective in making .NET more cloud native. You don't see other languages doing this. For Microsoft it is a priority to become cloud native, in-house.
This is one of those things that I hope don’t really catch on. I’m very positive about dotnet’s other changes, but I think a dotnet specific way to interact with Docker isn’t going to be good long term.
Genuine questions II have to deal with in my company: - What if I want to duplicate base docker file configs for other stacks that are not in C# but are shared between projects? - What if I want separate dev config from deploy config (ie. sqlite on local and postgres in prod) (docker-compose)? - What if I want to inject from .env files config like keys/secrets or native templates that I rather have seprate, how is the support for this (docker-compose)? - If Docker gets updated, do I now have to wait on Microsoft's release schedule to add an interface to bind to this property in their own file? - Why do you start writing docker config in a .cs file but with more characters (xml-ish vs yaml), I thought your point was to write less? - How does this work with docker-compose in general? - How much of Docker & Docker Compose config can it bind to? - What if my company takes on a legacy project where they work with exising Docker files in an *Existing team* , why should I now make things harder for the collab team without any to little advantage. Dockerfile seems an odd one to optimize in general. I re-use base files I get from DevOps, they hardly grow to a size that is unreadable, and they also rarely change. Unless u visually don't like the look of it in ur file system, it seems over optimising for little benefit, as you don't work on them in day-to-day operations anyway
I think you can just add this to the "when you wanna microsoft turnkey all the things because you're a budget contractor and don't have time to do what you need to do let alone learn how to do it so you learn how to string templates together and never learn how any of it actually works so that microsoft will own you forever as an incompetent boob" portfolio as well as the "i write tutorial-level code for enterprise production systems" portfolio.
Hey there! PM for the feature here, let me see if I can answer some of the point you raise (or at least add a bit of context to them): * If you have a shared base image that you want to work off of, you definitely can use it here - set the `ContainerBaseImage` property to explicitly control that (though you are then responsible for keeping that image up to date, as you are probably aware). * Compose is an example of where we'd suggest one of two pathways for users - the first would be to stay on Dockerfiles, the second would be to build your image with us then use that image as the `image:` in the compose file. * You're describing a compose feature here, not a Dockerfile-building feature here, so this new feature has zero impact on that. * When you say 'Docker gets updated' here, do you mean the base image that we infer for you? If so, we by default use image tags and fetch the latest of that tag, so you'll always get the latest 7.0 (or 6.0 if you target that, 8.0 if you target that) image to match. This is part of how we hope to help you stay up-to-date. * You don't have to keep the information in your project file - you can also use publish profiles (dedicated XML chunks that won't pollute your project file), or just raw CLI parameters for these MSBuild properties if that makes you feel better. We think many folks will be satisfied with the defaults we're inferring here, so any customizations would still be less than creating an entire new Dockerfile. * I touched on the compose interactions above, does that adress it? * You can check github.com/dotnet/sdk-container-builds/blob/main/docs/ContainerCustomization.md for the full details, but at this point basically only User/Group metadata is missing. We have zero ability to do RUN command equivalents, and nothing here is about docker-compose - that's an entirely separate feature set that we are not looking into at all. This is about container _generation_ (which is technically not very complex or interesting), not about _orchestration_ (which is muuuuuch more varied and opinionated). * Again, this is a _suggestion_, a new feature. If your team doesn't want to use it then no one will force you to. VS/CLI/SDK will continue to support Dockerfile-based builds because it would be foolish to drop that - Dockerfiles are everywhere. Hope this helps!
You can keep the Dockerfile in the project folder and run it from the solution directory using -f option in docker command, the context of the Dockerfile while be the solution directory. In this way you can use project reference (dotnet add reference) inside the container, you juste need to tel to your Dockerfile which project to copy from Host to the container (you can because the context is the solution dir not the project one)
i don't know, perhaps i'm old fashion but i prefer the docker image approach. For example in my use case i have to install the datadog agent and the grpc-health-probe via curl and i don't want go crazy on the csproj configuration
@@Eldest139 I mean... Docker is a different "system". The dotnet SDK has no bearing on how it behaves, this new feature is just "interacting" with the docker daemon. Docker by itself is still functioning as it did before 😅
If this would also produce a dockerfile, I'd be more into it I think. Then it would be more of a syntactic sugar which is perfectly fine. Removing it from view completely is like to cause more problems/complexity than it solves for larger organizations.
Thanks for sharing. I still prefer using Docker Compose to configure everything in one place and for multiple applications as in the Clean Architecture Example you mentioned. I think Docker Init makes it very easy to getting started with autogenerating the docker and docker compose files.
Actually the docker file generated by .NET have valid URLS. Basically .NET designed it in such a because the build commands will be run from solution level because some of the files needed for your project will/can also be present at solution level. So basically dotnet explicitly provides dockerfile path from solution file location for docker build.
seems to me like its similar to top level statements - some magic is happening behind the scenes, but it is all hidden behind some abstraction layer. can be useful as a short term solution, but for production i would stick to dockerfiles and docker composes
Ok it has a lot of customization but at that point it's so much text you might as well use the Dockerfile! Why learn a new syntax and lock in to the microsoft feature when you can just use Dockerfile?
Would you consider making a video about TPL with threadpool interactions, execution context, async state machine and all things related and how you can easily run into thread pool starvation if you misuse these techniques? It's easy to find superficial explainations about these topics on the web, but an indepth guide to everything async would be really valuable I think!
And what about extra lib in base image (Docker RUN command)? Don't know but the first impression is, the docker file feels more convenient & standard if you use Docker otherwise may be Microsoft is trying to be generic here.
How many times have we all seen a project which has fallen behind on a few things, and now the .csproj file is doing something different from the Dockerfile (or, the Dockerfile doing something different than the CI pipeline build agent, etc). The fewer descriptions we have about how to build a project, the fewer opportunities there are for things to get out of sync and bugs to be introduced. Overall I like it for that reason, but I'm not sure I'm in love with all the syntax that's being used.
@nickchapsas can you put a page up on your website that links all of the cool tools, extensions and shells you use in your videos? I am finding I want to use some things you use, but don't know where to find them to try them out!
One nice thing about this, if they implemented it correctly of course from the MS side, is that this way layering can be done for you and you are less likely to mess up. Wrong layering can fill up an artifact storage quite quickly.. I like the idea, not sure if I like how it is split between Item and Property groups. Maybe this should actually not be part of the csproj at all or optionally can be pulled out 🤔 Imho, don’t want this to become a mess like the poms in Java projects..
this package almost had me excited... but it is a frontend for docker/dockerfile as a dependency. Ive been working for days to get a linux self-hosted AzDevops build agent that runs in k8s going. And since Docker-inn-Docker is painful (and problematic for security reasons), I have used Kaniko instead (rootless container building). I so much wanted to hear that this was an OCI compliant builder of some form that would have made all my efforts wasted, and allow for build agents without having docker infrastructure installed.
This looks great. But how would you build a stack of containers like an API, a Website and a Database all at once? Is there support for docker-compose?
From what I could find, atm there is no way to emit/generate the Dockerfile that is implicitly generated behind the scenes, so you can't use it easily in docker compose. This should be the issue tracking this: github.com/dotnet/sdk-container-builds/issues/146. Given that "dotnet build-image" has this feature already with the "--as-file", I don't see why MS won't add it in publish too.
@@nickchapsas Minor note - there's no Dockerfile being generated at all behind the scenes. We deal with the raw elements of containers directly (layers, metadata, configuration), which is what Docker does as well, just they do it by way of interpreting your Dockerfile and we do it by way of MSBuild items and properties. This issue is specifically to emit a Dockerfile that attempts to do what we do (which may not 100% work due to custom NuGet configurations, etc).
TO ME, lol, before people start hating on my comment. Sounds good, doesn't work. As always, Microsoft is trying to reinvent the wheel and you'll end up with a lot of magic that needs lots of workarounds in a real life scenario. nice video, as always. short, direct. noice.
It's great and all, but how does it deal with dependencies between projects? Let's say I have 2, 3 or more dependencies for that API project. Will they be automatically built in into the image? If yes that's amazing and I would love for this to work under .NET6
@@nickchapsas This is actually one of the big motivations for the work - authenticating to private Nuget feeds is a huge pain in Dockerfiles, and this feature completely removes that pain. If you can `dotnet build` locally, you can create a container locally now.
I am struggling for days trying to containerize a Blazor app for ARM64 format via a dockerfile and now I just learnt that... :O It made me that ARM64 image in matter of seconds... :O Is there a way to have this pushed to Docker Hub ? From what I see, it does not support login/password so we cannot use it yet but maybe I'm not up to date.
I think this is an interesting and useful fearure for developing applications fast. However I am not sure about the thing that is seperation of concern. A Dockerfile does what it should do, and I am not sure that it should be mixed in with the csproj file. I think this would be useful for the first development stages when starting a new project, but in the long run I think I'd still prefer a seperate Dockerfile.
I couldn't figure out what to do with the Healthcheck call which is in my deleted Dockerfile nor how to change the my docker-compose.yml to not include a reference to the Dockerfile. None of the microsoft official documentation addresses those either, so not entire sure if this is mature enough to use.
Dockerfile on the beginning of your video is nice example of multi stage image, how Microsoft approach support multi stage images? I don't see that, what about running multiple containers on the same node using docker compose (this is very common scenario for testing purposes) I think Microsoft is reinventing the wheel with even more verbose syntax and tailored only for .NET apps how we even call this "cloud native"
then how we know what is copied from "base" stage to "final" using Microsoft approach? In Dockerfile you have those things specified (which is good thing because it gives you control what you want in docker final image)
Now you do not have to manage a separate docker file... now you have to manage csproj properties while few other can. Nay to standards! Yay for job security!
So Dockerfile will be in project file? Good to know, it is possible. I'm good tho. Prefer applying already built up docker knowledge vs learning a new tool. At least right now.
I wouldn't ever type this or docker build, coz it's just for CI Lemme think 1. I have to install smth and know it's syntax 2. I have to change my csproj 3. I have to rebuild my ci 4. ??? 5. profit
Hi Nick, Nice video and cool presentation. How will does work in a microservice scenario. with Shared Library projects and say Envoy gateway? Thank. you.
I don't know other people experience but in my scenario, where we want to be able to use a docker-compose deubg experience, we need a dockerfile. I'd like to see source generators being used to generate for example a partial dockerfile to handle multiple projects dependencies (that's the main "problematic" part of having to handle our project dockerfile actually), but I want to be able to edit the dockerfile, not having it be used implicitly by some "magical" tool. (and I see no pros on having to move part of my dockerfile into the csproj)
This is really amazing! How to refer the private nuget packages from controlled internal registry which needs feedurl, username and password to pull during application build?
I would love to see a way to host full webapp using some serverless solution in the cloud. What would be the easiest way? Azure? GCP? AWS? What would you recommend? Can we take advantage of docker?
Now each building tool has to create a pain in the ass about "how to manage this hell" in a multiple ways (DCPROJ, CSPROJ, docker-compose and something else), break "Debugger attach", force to rebuild everything because of naming rules (when you open MSVS with project already started through docker-compose) and other
I don't like that or I'm confused! Because what happened if one part of our services use .Net and others use Docker file and if want to mange whole system we have to deal with .Net 7 part separately! Right? We don't want that!
From what I could find, atm there is no way to emit/generate the Dockerfile that is implicitly generated behind the scenes, so you can't use it easily in docker compose. This should be the issue tracking this: github.com/dotnet/sdk-container-builds/issues/146. Given that "dotnet build-image" has this feature already with the "--as-file", I don't see why MS won't add it in publish too.
What has that got to do with this? This is for building your docker images. I would assume once they are built and stuck in a registry you can reference them in a docker compose in exactly the same way as you did before.
Kind of a weird idea in my perspective, tbh. In a team usually there's more than one microservice and to manage a CI/CD pipeline for them id usually make a generic building block (be it gitlab template or jenkins shared library or w/e) for building my container image to later deploy it to some kubernetes cluster. it'd be weird if one microservice for some unknown reason would use a "dotnet build" stage instead of an "image build" stage. also the build stage & the "push" stage (where u push ur image to some image registry) r often coupled for performance reasons and i dont think making a specific "dotnet image build" stage would be good practice unless you only work with dotnet and are willing to commit to dotnet forever
This not an improvement. There is lot of space to improve the framework, but this not one. Not removing this file, it makes it less compatible with other environments, the cloud and modern apps in general.
If you want to keep using a Dockerfile, you can keep using it. You can't assume that you are the representive example of every userbase they are trying to appeal to.
@@nickchapsas that's my point precisely. Is not a matter of whims or I want, is a matter of greater compatibility and convenience which is lost this way. Doesn't mean if you can do _cool-new-feature-x_ you should. As you can see in the comments this way brings more doubts than solutions, as Many things are lost removing this file. MS first was reluctant to containers and docker, now it adopts it in a way that makes it difficult to deal with. Developers, DevOps, architects, please DO NOT adopt this way, do not remove the dockerfile
@@errrzarrr Look, there is no denying that the vast majority of dockerfiles look exactly the same. Sure there will be a 10-25% of usecases where this can't be sufficient and in that case, obviously don't use it, but if it's a drop-in replacement for what you can do now, then why not simply use it? It's a win-win
@@nickchapsas not a win-win. The file exists for a reason, first MS was reluctant to adopt containers and docker, now adopting it in a way everyone else will hate us.
Looks promising! How would this work in a GitHub action chain? We currently use the generated Dockerfile to build and publish the containers with the git tags. I like the fact that it is using the same Dockerfile, so it doesn't matter if I'm manually performing 'docker build .' or GitHub actions is doing this. The result will be exactly the same. So do I just run the dotnet command as a GitHub action step or is there a more integrated option that does build & push?
I use auto generated dockerfile per project into a solution having all of my projects (microservices) and I use dockercompose debugging this way the MS I want to trigger. It works pretty well, not sure to use this new feat.
Yeah, don't do this on a real project or your "devops" will not like you. Also when build machine has docker only installed and no dotnet then you won't be able to use this, right? For proof of concepts and sandboxing this might look promising, but why bother to learn all these new bells and whistles when the Dockerfile solution is already working just great and simple. Maybe if you are totally new to containerization you could do this to kickstart your learning but still you would start on this path that is different and you will be just confused I guess.
Exactly this. With dockerfiles, you can have a ci/cd template that builds whichever language with kaniko, the only requirement is a container runtime. With this approach, you would need a separate ci setup only for dotnet7+ projects. No thanks!
Hi. What are true benefits of that approach? Only "no need to learn docker"? I think it's an illusion, it's only hiding important details. Isn't it a step away from kind of standard? We have a billion ways to do everything. Now, instead of expanding what is already there, there is new way to do the same(with limited possibilities).
MS embracing Cloud Native Container Apps is the only right move! Love how they work on making it easier to use! Are there any relation between this nuget package and Project Tye?
I'm not sure - this feature is right decision. This is so "Microsoft way" feature. they built in support of build of Docker images into the dotnet sdk. It does tons of "magic" behind the scene. This is great for the simple cases like you showed in this video, however this feature complete isolated from the rest of the world. lets say we have CI/CD pipeline in Gitlab/github/GCP/Azure Dev OPS. those tools already supports build od Docker images, Podman containers, native K8s images. So Microsoft decided to introduce build of docker images using the cproj files. What if we want to use podman instead? What if we want to use k8s native containers? What if I have custom security Auth process for building Docker containers? I'm glad that Microsoft adding new features to the sdk, but I have some doubts as for correctness of this particular one.
@@itmeurdad Thanks for your feedback folks! Would it change your mind if I pointed you to the ko project for golang and the Jib project for Java, both of which use this model and are authored by Google? In any case, we think that more ways to make containers are great, but the way _you_ want to make containers is the way that works best for your team/environment.
Nice step in right direction, but useless (for now) in real life ci/cd automation set ups, for various reasons, one of them would be that we for example use docker-daemon-less builds
The other one that in real life you will have templates and especially in cloud projects you have different programming languages and all of them could use docker so you’re building your domain tailored docker pipeline template and then java, dotnet, whatever can use it. This on the other hand can’t be used by anything else than dotnet
@@romanhrytskiv8845 This does do daemonless builds - that's one of our driving use cases. WE just also support pushing the generated container to a local container daemon for use in things like docker-compose, etc. Having said that, definitely continue to use Dockerfiles if you rely heavily on automation that needs them!
For development sure it simplifies things but containers aren't really built on dev machines. Even tho this might get by with Azure GitOps I can't really see how to build a container in kubernetes where there is no Docker daemon instead you get Buildah or Kaniko or similar builders. I only suspect this approach you demonstrated is actually talking to Docker daemon in the background but then again I might be wrong.
Nope, we're more like ko for golang or Jib for Java - we build the images by constructing the JSON manifests, config, and TAR file system layers in a spec-compliant way. This is what lets us a) build without a container builder engine, and b) reuse existing MSBuild metadata, properties, etc to remove the boilerplate and work so much faster.
Nice! I would love to know how to move away from using the dockercompose because that is my biggest headache in trying to move to vscode as my primary editor. (I know you like Jetbrains) lol
Microsoft is really pushing towards DevOpsbut there is a long way to go. On leaving the DockerFile, I would still go with the DockerFile. And I saw a lot of people build the Docker file via Docker-compose in the build context which I alos do't find it convient. so I always have My DockerFile and Docker-Compose files seperately. I'm planning to explore more deep via Github Actions and Kubarnets so my perspectives may change along the way.
This is indeed very cool, but on a production level, when you want to also include a database or other dependencies, you will still need to work with docker files, no ?
@@nickchapsas I guess what I meant was that you can't really forego Docker and it's ecosystem while constructing the solution as a whole. Thanks and great content!
It's a nice feature, but it's not accurate to say it works for all situations. There are lots of cases where you would want to customize your image for security reasons for instance, where this *probably* wouldn't be enough. Take Microsoft's new chiseled images. They don't come with CultureInfo support out of the box. How would you customize that image to add it in (the way Microsoft says you can) if using this approach?
I didn’t say that it works in all situation. I said most situations. It’s always a win when you can make adoption more accessible to people who only use the tool for basic stuff. When they need more advanced usage they can more to a custom dockerile
@@nickchapsas It might interest you that you can probably do this in .NET 6 as well (but you probably do need the .NET 7 SDK). At least it did last time I tested this out.
I don't like "magic" like this, where they hide actual implementation. I would consider it good if it generated dockerfile the way like source generators do so I can view it and really understand what's going on.
isn't the point of abstraction in programming to "hide magic", so that you can worry about more important things? If we had it your way, we would still be wasting time looking at zeroes and ones
@@dandayne i did not say they should be no more higher or easier abstraction, I simply wanted them make generated docker config accessible like source generator does
You are my source for keeping up with my tech stack. Thank you for showing me this cool new .NET 7 feature!
Super cool to see my team's feature on a Nick Chapsas video. Tons of engagement in the comments that I'll go through and respond to, but to cover the common questions:
* No, we don't want to replace docker files for everyone! We want to reduce friction and get rid of the 80% docker files that had no value. If you have specific needs (native libraries, integration with tooling that expects Docker files), then definitely still keep that! Just like NoSQL means 'not only' SQL, there's a broad ecosystem of container-adjacent tools that work like this new tool does (e.g. ko for golang, and jib for Java). This is just that, for .NET!
* It's really really fast in comparison :) makes me happy every time I build a container.
* It does work with alternative container runtimes like podman and nerdctl. I have explicitly manually tested these, and they remain part of our goals. I'd like to expand our support for them in the future.
* I think if project files had better intellisense it would be a nicer experience overall. I have plans in this space (I'm also the MSBuild PM), but nothing concrete yet.
Glad you had a good time with it, Nick, and thanks for the highlight :)
Will it work with Windows Containers on Win10 PRO 21H2 (latest updates) with `--isolation=process`?)))))
The purpose of having the dockerfile in the project directorx is that you can build multiple projects from the same build context. Say you have two projects in a solution, you can now build both using docker build -f project1 . and docker build -f project2 .
Yes, exactly my point. Because you can do _cool-new-feature-x_ doesn't mean you should. This is a step backwards
@@errrzarrr This isn't a step backwards. Like you said, just because you can, doesn't mean you should. You don't HAVE to do this, you can still have your dockerfile if you want to. This will make docker easier to get into for new devs and smaller projects, especially microservices will benefit from this. We have multiple microservices that are all in different repositories, we're using the basic dockerfile that builds basic aspnet images. If we use this we don't have to have an extra file that most developers won't even know how to edit, we can just manage it via the csproj. If any project gets to a point where the basic build doesn't work anymore we can just add the dockerfile back in and everything will work fine. A step backwards would be if Microsoft forces this down our throat and somehow breaks any builds that run through a dockerfile
@@deondupreez4713 "most developers" don't know how to edit a csproj file as "most developers" don't use c# but that's besides the point. This still requires knowledge of docker, it's just a different syntax and location for the configuration and is utterly unnecessary. It will only lead to more Visual Studio ignorance/dependence.
@@itmeurdad How will this lead to Visual Studio dependence? He's not even using Visual Studio in the video?
@@deondupreez4713 .csproj is how?
Whether you are managing the csproj file or a docker file, I see no difference. I do like what MS is doing and I appreciate it, but I am a fan of having a separate Dockerfile, also sometimes you might have 2 or more Dockerfiles in a project for different environments that you'd like your application to run in.
Isn’t that a declarative vs imperative approach?
MS again on its thing. Trying to offer another way of doing things that already exists and suits people. We all know how it's gonna end.
Yeah, when a .NET developer jumps to another stack. They might get shocked because there are so many things they didn't see in .NET, like Dockerfile because it's hidden.
What if I want to install a package before or run something else before starting the app? How can I do that without the Dockerfile? Creating a base image?
@@viacheslavsamodelkin6944 It's about increasing dependency on Microsoft, or more so strengthening their reputation of heading in a more innovative direction. Plus, this may end up being a more customizable and native solution to container management that Docker is not able to provide since it is specific to .NET. It is a smart move on their part from a business perspective in making .NET more cloud native. You don't see other languages doing this. For Microsoft it is a priority to become cloud native, in-house.
This is one of those things that I hope don’t really catch on. I’m very positive about dotnet’s other changes, but I think a dotnet specific way to interact with Docker isn’t going to be good long term.
Excellent new feature in net 7 Thanks for this explanation Nick 👍
Genuine questions II have to deal with in my company:
- What if I want to duplicate base docker file configs for other stacks that are not in C# but are shared between projects?
- What if I want separate dev config from deploy config (ie. sqlite on local and postgres in prod) (docker-compose)?
- What if I want to inject from .env files config like keys/secrets or native templates that I rather have seprate, how is the support for this (docker-compose)?
- If Docker gets updated, do I now have to wait on Microsoft's release schedule to add an interface to bind to this property in their own file?
- Why do you start writing docker config in a .cs file but with more characters (xml-ish vs yaml), I thought your point was to write less?
- How does this work with docker-compose in general?
- How much of Docker & Docker Compose config can it bind to?
- What if my company takes on a legacy project where they work with exising Docker files in an *Existing team* , why should I now make things harder for the collab team without any to little advantage.
Dockerfile seems an odd one to optimize in general. I re-use base files I get from DevOps, they hardly grow to a size that is unreadable, and they also rarely change. Unless u visually don't like the look of it in ur file system, it seems over optimising for little benefit, as you don't work on them in day-to-day operations anyway
I think you can just add this to the "when you wanna microsoft turnkey all the things because you're a budget contractor and don't have time to do what you need to do let alone learn how to do it so you learn how to string templates together and never learn how any of it actually works so that microsoft will own you forever as an incompetent boob" portfolio as well as the "i write tutorial-level code for enterprise production systems" portfolio.
Hey there! PM for the feature here, let me see if I can answer some of the point you raise (or at least add a bit of context to them):
* If you have a shared base image that you want to work off of, you definitely can use it here - set the `ContainerBaseImage` property to explicitly control that (though you are then responsible for keeping that image up to date, as you are probably aware).
* Compose is an example of where we'd suggest one of two pathways for users - the first would be to stay on Dockerfiles, the second would be to build your image with us then use that image as the `image:` in the compose file.
* You're describing a compose feature here, not a Dockerfile-building feature here, so this new feature has zero impact on that.
* When you say 'Docker gets updated' here, do you mean the base image that we infer for you? If so, we by default use image tags and fetch the latest of that tag, so you'll always get the latest 7.0 (or 6.0 if you target that, 8.0 if you target that) image to match. This is part of how we hope to help you stay up-to-date.
* You don't have to keep the information in your project file - you can also use publish profiles (dedicated XML chunks that won't pollute your project file), or just raw CLI parameters for these MSBuild properties if that makes you feel better. We think many folks will be satisfied with the defaults we're inferring here, so any customizations would still be less than creating an entire new Dockerfile.
* I touched on the compose interactions above, does that adress it?
* You can check github.com/dotnet/sdk-container-builds/blob/main/docs/ContainerCustomization.md for the full details, but at this point basically only User/Group metadata is missing. We have zero ability to do RUN command equivalents, and nothing here is about docker-compose - that's an entirely separate feature set that we are not looking into at all. This is about container _generation_ (which is technically not very complex or interesting), not about _orchestration_ (which is muuuuuch more varied and opinionated).
* Again, this is a _suggestion_, a new feature. If your team doesn't want to use it then no one will force you to. VS/CLI/SDK will continue to support Dockerfile-based builds because it would be foolish to drop that - Dockerfiles are everywhere.
Hope this helps!
You can keep the Dockerfile in the project folder and run it from the solution directory using -f option in docker command, the context of the Dockerfile while be the solution directory. In this way you can use project reference (dotnet add reference) inside the container, you juste need to tel to your Dockerfile which project to copy from Host to the container (you can because the context is the solution dir not the project one)
i don't know, perhaps i'm old fashion but i prefer the docker image approach. For example in my use case i have to install the datadog agent and the grpc-health-probe via curl and i don't want go crazy on the csproj configuration
And you can still do that of course. This new feature will just make it a lot simpler for 90% of developers.
@@weebatheart Oh wow, any documents of that?
@@weebatheart Exactly - as a person who has yet to try Docker, I like the fact that I can learn as I go more easily!
@@Eldest139 I mean... Docker is a different "system". The dotnet SDK has no bearing on how it behaves, this new feature is just "interacting" with the docker daemon. Docker by itself is still functioning as it did before 😅
@@weebatheart ok, so I think that the correct flow cold be create a custom base image with all that you need and use that as default container image.
Another nugget of Nick Chapsas GOLD! Thanks Nick!
If your dockerfile is anemic, yes. But in my case a run a lot of things like tests, private nuget and format
If this would also produce a dockerfile, I'd be more into it I think. Then it would be more of a syntactic sugar which is perfectly fine. Removing it from view completely is like to cause more problems/complexity than it solves for larger organizations.
It probably did in the obj folder. That’s how most source generators work.
Thanks for sharing. I still prefer using Docker Compose to configure everything in one place and for multiple applications as in the Clean Architecture Example you mentioned. I think Docker Init makes it very easy to getting started with autogenerating the docker and docker compose files.
Actually the docker file generated by .NET have valid URLS. Basically .NET designed it in such a because the build commands will be run from solution level because some of the files needed for your project will/can also be present at solution level.
So basically dotnet explicitly provides dockerfile path from solution file location for docker build.
seems to me like its similar to top level statements - some magic is happening behind the scenes, but it is all hidden behind some abstraction layer. can be useful as a short term solution, but for production i would stick to dockerfiles and docker composes
Great tutorial!! Best on youtube about this topic.
Ok it has a lot of customization but at that point it's so much text you might as well use the Dockerfile! Why learn a new syntax and lock in to the microsoft feature when you can just use Dockerfile?
Thanks for this content!
I Love it! Ty for Sharing!
Would you consider making a video about TPL with threadpool interactions, execution context, async state machine and all things related and how you can easily run into thread pool starvation if you misuse these techniques?
It's easy to find superficial explainations about these topics on the web, but an indepth guide to everything async would be really valuable I think!
that'd be totally great
Love the feature
And what about extra lib in base image (Docker RUN command)? Don't know but the first impression is, the docker file feels more convenient & standard if you use Docker otherwise may be Microsoft is trying to be generic here.
How many times have we all seen a project which has fallen behind on a few things, and now the .csproj file is doing something different from the Dockerfile (or, the Dockerfile doing something different than the CI pipeline build agent, etc). The fewer descriptions we have about how to build a project, the fewer opportunities there are for things to get out of sync and bugs to be introduced. Overall I like it for that reason, but I'm not sure I'm in love with all the syntax that's being used.
Nick this video made me subscribe.
I’m glad 😃
I recently struggle with building container of my clean architecture app i hope that this method will work for my solution.
That is so freaking cool!!
@nickchapsas can you put a page up on your website that links all of the cool tools, extensions and shells you use in your videos? I am finding I want to use some things you use, but don't know where to find them to try them out!
One nice thing about this, if they implemented it correctly of course from the MS side, is that this way layering can be done for you and you are less likely to mess up. Wrong layering can fill up an artifact storage quite quickly..
I like the idea, not sure if I like how it is split between Item and Property groups. Maybe this should actually not be part of the csproj at all or optionally can be pulled out 🤔
Imho, don’t want this to become a mess like the poms in Java projects..
How can I use cross-container-communication using dotnet publish? I mean how can I create a network and make a .net 7 app part of it ?
Hi Nick, great video as usual. Can you please share the official documentation ?
😊 life isn’t hard anymore, thank you 🙏
this package almost had me excited... but it is a frontend for docker/dockerfile as a dependency. Ive been working for days to get a linux self-hosted AzDevops build agent that runs in k8s going. And since Docker-inn-Docker is painful (and problematic for security reasons), I have used Kaniko instead (rootless container building). I so much wanted to hear that this was an OCI compliant builder of some form that would have made all my efforts wasted, and allow for build agents without having docker infrastructure installed.
That's incredible
This looks great. But how would you build a stack of containers like an API, a Website and a Database all at once? Is there support for docker-compose?
From what I could find, atm there is no way to emit/generate the Dockerfile that is implicitly generated behind the scenes, so you can't use it easily in docker compose. This should be the issue tracking this: github.com/dotnet/sdk-container-builds/issues/146. Given that "dotnet build-image" has this feature already with the "--as-file", I don't see why MS won't add it in publish too.
I'm using dotnet publish with docker-compose with no issue. Docker-compose must not compile images. It can use ready images
@@nickchapsas Minor note - there's no Dockerfile being generated at all behind the scenes. We deal with the raw elements of containers directly (layers, metadata, configuration), which is what Docker does as well, just they do it by way of interpreting your Dockerfile and we do it by way of MSBuild items and properties. This issue is specifically to emit a Dockerfile that attempts to do what we do (which may not 100% work due to custom NuGet configurations, etc).
TO ME, lol, before people start hating on my comment. Sounds good, doesn't work. As always, Microsoft is trying to reinvent the wheel and you'll end up with a lot of magic that needs lots of workarounds in a real life scenario. nice video, as always. short, direct. noice.
It's great and all, but how does it deal with dependencies between projects? Let's say I have 2, 3 or more dependencies for that API project. Will they be automatically built in into the image? If yes that's amazing and I would love for this to work under .NET6
That will work fine. The tool knows to resolve dependencies
@@nickchapsas have you tried dealing witb private nuget feeds?
@@Downicon3 As long as the feeds are authed in the agent level, you'll be fine
@@nickchapsas This is actually one of the big motivations for the work - authenticating to private Nuget feeds is a huge pain in Dockerfiles, and this feature completely removes that pain. If you can `dotnet build` locally, you can create a container locally now.
I am struggling for days trying to containerize a Blazor app for ARM64 format via a dockerfile and now I just learnt that... :O
It made me that ARM64 image in matter of seconds... :O
Is there a way to have this pushed to Docker Hub ?
From what I see, it does not support login/password so we cannot use it yet but maybe I'm not up to date.
I think this is an interesting and useful fearure for developing applications fast. However I am not sure about the thing that is seperation of concern. A Dockerfile does what it should do, and I am not sure that it should be mixed in with the csproj file. I think this would be useful for the first development stages when starting a new project, but in the long run I think I'd still prefer a seperate Dockerfile.
I love this
really nice video , How can we Create a CI-CD pipelien in Azure Devops wth this approach to publish Lambda function on AWS ?
Very cool feature like in Java Gradle/Maven Builds. Thx
I couldn't figure out what to do with the Healthcheck call which is in my deleted Dockerfile nor how to change the my docker-compose.yml to not include a reference to the Dockerfile. None of the microsoft official documentation addresses those either, so not entire sure if this is mature enough to use.
Dockerfile on the beginning of your video is nice example of multi stage image, how Microsoft approach support multi stage images? I don't see that, what about running multiple containers on the same node using docker compose (this is very common scenario for testing purposes) I think Microsoft is reinventing the wheel with even more verbose syntax and tailored only for .NET apps how we even call this "cloud native"
The backing dockerfile is the exact same multi stage image
then how we know what is copied from "base" stage to "final" using Microsoft approach? In Dockerfile you have those things specified (which is good thing because it gives you control what you want in docker final image)
What about restoring from private nuget feeds? We had to generate a PAT and pass it into our dockerfile as an argument.
Thoughts on Rider/Intellij Platform's built-in http client (the HTTP scratch) over postman?
I prefer using http scratch when I’m doing actual work. I’m using postman in videos because it’s better visually for the viewer
Now you do not have to manage a separate docker file... now you have to manage csproj properties while few other can. Nay to standards! Yay for job security!
So Dockerfile will be in project file?
Good to know, it is possible.
I'm good tho. Prefer applying already built up docker knowledge vs learning a new tool.
At least right now.
Sooo weird. Why somebody needs this instead of explicit Dockerfile?
Why would you have one extra thing to keep up to date if your Dockerfile is simple enough?
I wouldn't ever type this or docker build, coz it's just for CI
Lemme think
1. I have to install smth and know it's syntax
2. I have to change my csproj
3. I have to rebuild my ci
4. ???
5. profit
@@qweqwe123qewweqwe I'd be pretty worried if you didn't know the syntax of "dotnet publish" ngl
A video on customizing MsBuild will be nice
Pretty cool, will like to see how auth and small gotcha's are handled
One more responsibility for the csproj, that then may become messy and cluttered. Is there a way to split a csproj into multi files ?
Microsoft doing right moves 🎉
Will it work in a scenario where I have a monolith git repo and want to build a single solution among others?
Hi Nick,
Nice video and cool presentation.
How will does work in a microservice scenario. with Shared Library projects and say Envoy gateway?
Thank. you.
So if you install that package, you can no longer publish it as a non container package? Or how would I still be able to use a regular publish?
I don't know other people experience but in my scenario, where we want to be able to use a docker-compose deubg experience, we need a dockerfile. I'd like to see source generators being used to generate for example a partial dockerfile to handle multiple projects dependencies (that's the main "problematic" part of having to handle our project dockerfile actually), but I want to be able to edit the dockerfile, not having it be used implicitly by some "magical" tool. (and I see no pros on having to move part of my dockerfile into the csproj)
Great Video, Thanks a lot
This is really amazing!
How to refer the private nuget packages from controlled internal registry which needs feedurl, username and password to pull during application build?
Nick, hi. Thanks for the video. Can I find out what VM options you use for the rider?
I would love to see a way to host full webapp using some serverless solution in the cloud. What would be the easiest way? Azure? GCP? AWS? What would you recommend? Can we take advantage of docker?
Now each building tool has to create a pain in the ass about "how to manage this hell" in a multiple ways (DCPROJ, CSPROJ, docker-compose and something else), break "Debugger attach", force to rebuild everything because of naming rules (when you open MSVS with project already started through docker-compose) and other
I have no
Issues with docker files, BUT I like the way Microsoft is going! Nice
I don't like that or I'm confused! Because what happened if one part of our services use .Net and others use Docker file and if want to mange whole system we have to deal with .Net 7 part separately! Right? We don't want that!
Not sure if offloading everything to .csproj file is a good idea. It is probably useful for automated tasks that they use internally at Azure.
It's an AWFUL idea.
thoughts on buildpack? we started playin' with it and it's been a nice way to get rid of our dockerfiles as well.
What about docker-compose, networking and volumes?
From what I could find, atm there is no way to emit/generate the Dockerfile that is implicitly generated behind the scenes, so you can't use it easily in docker compose. This should be the issue tracking this: github.com/dotnet/sdk-container-builds/issues/146. Given that "dotnet build-image" has this feature already with the "--as-file", I don't see why MS won't add it in publish too.
What has that got to do with this? This is for building your docker images.
I would assume once they are built and stuck in a registry you can reference them in a docker compose in exactly the same way as you did before.
@@briumphbimbles Realistically yeah, but I assume they are referring to docker compose execution for local develoment
@@nickchapsas Why would that prevent you from using a docker compose for local development?
@@briumphbimbles Because docker compose would need a dockerfile to build your local codebase, and there is no dockerfile anymore
But what if I want to customize the environment inside a container? Seems pretty useless for a lot of actual use cases.
Kind of a weird idea in my perspective, tbh. In a team usually there's more than one microservice and to manage a CI/CD pipeline for them id usually make a generic building block (be it gitlab template or jenkins shared library or w/e) for building my container image to later deploy it to some kubernetes cluster. it'd be weird if one microservice for some unknown reason would use a "dotnet build" stage instead of an "image build" stage. also the build stage & the "push" stage (where u push ur image to some image registry) r often coupled for performance reasons and i dont think making a specific "dotnet image build" stage would be good practice unless you only work with dotnet and are willing to commit to dotnet forever
You could use the directory.build.props and .targets and share similar settings across multiple services in the same solution pretty cleanly.
It's possible to configure enviroment, like running apt-get update or setting certificates for https ?
This not an improvement. There is lot of space to improve the framework, but this not one. Not removing this file, it makes it less compatible with other environments, the cloud and modern apps in general.
If you want to keep using a Dockerfile, you can keep using it. You can't assume that you are the representive example of every userbase they are trying to appeal to.
@@nickchapsas that's my point precisely. Is not a matter of whims or I want, is a matter of greater compatibility and convenience which is lost this way. Doesn't mean if you can do _cool-new-feature-x_ you should.
As you can see in the comments this way brings more doubts than solutions, as Many things are lost removing this file. MS first was reluctant to containers and docker, now it adopts it in a way that makes it difficult to deal with.
Developers, DevOps, architects, please DO NOT adopt this way, do not remove the dockerfile
@@errrzarrr Look, there is no denying that the vast majority of dockerfiles look exactly the same. Sure there will be a 10-25% of usecases where this can't be sufficient and in that case, obviously don't use it, but if it's a drop-in replacement for what you can do now, then why not simply use it? It's a win-win
@@nickchapsas not a win-win. The file exists for a reason, first MS was reluctant to adopt containers and docker, now adopting it in a way everyone else will hate us.
sensacional !😉
Nick, what's the terminal setup u use here? It's win terminal?
It's Microsoft Terminal with oh-my-posh
Looks promising! How would this work in a GitHub action chain?
We currently use the generated Dockerfile to build and publish the containers with the git tags. I like the fact that it is using the same Dockerfile, so it doesn't matter if I'm manually performing 'docker build .' or GitHub actions is doing this. The result will be exactly the same. So do I just run the dotnet command as a GitHub action step or is there a more integrated option that does build & push?
They show how to use it with GitHub actions here: devblogs.microsoft.com/dotnet/announcing-builtin-container-support-for-the-dotnet-sdk/
ok, but how to deploy blazor webassembly hosted (front and back) to raspberry pi?
I use auto generated dockerfile per project into a solution having all of my projects (microservices) and I use dockercompose debugging this way the MS I want to trigger. It works pretty well, not sure to use this new feat.
Which tool are you using to generate the docker file?
@@nickchapsas I built my own dockerfile once and put it into a dotnet template that I use to create my microservices
Yeah, don't do this on a real project or your "devops" will not like you. Also when build machine has docker only installed and no dotnet then you won't be able to use this, right? For proof of concepts and sandboxing this might look promising, but why bother to learn all these new bells and whistles when the Dockerfile solution is already working just great and simple. Maybe if you are totally new to containerization you could do this to kickstart your learning but still you would start on this path that is different and you will be just confused I guess.
Exactly this. With dockerfiles, you can have a ci/cd template that builds whichever language with kaniko, the only requirement is a container runtime. With this approach, you would need a separate ci setup only for dotnet7+ projects. No thanks!
Pass, seems like once you get beyond a beginner-level, you are just putting the docker properties into XML in the csproj, which is very limiting
Hi. What are true benefits of that approach? Only "no need to learn docker"? I think it's an illusion, it's only hiding important details. Isn't it a step away from kind of standard? We have a billion ways to do everything. Now, instead of expanding what is already there, there is new way to do the same(with limited possibilities).
So everything just moved from the dockerfile to the csproj file, but with less structure and much more wordy. Gotit
Yeah, you're right. Yaml is the minacle of structure and XML with its explicit fields is such a downgrade. Gotit
Wow 👌
Aren't you in fact ending up with moving some of the content from dockerfile to the csproj?
The timing of this video couldn’t have been more ideal, one of my upcoming tasks at work is to start switching away from Dockerfile to this method 😁
How did you color your terminal? 🙄🙄
MS embracing Cloud Native Container Apps is the only right move! Love how they work on making it easier to use! Are there any relation between this nuget package and Project Tye?
I'm a bit confused about this since no profile named DefaultContainer exists but dotnet still knows to publish to docker
Awesome ty. It seems it could be used in a dotnet 6 project with how it was implemented. Do you know?
I haven't tried to run it in .NET 6. Official support is from .NET 7 onwards
@@nickchapsas It can work with projects that _target_ .NET 6, but you have to use a .NET 7 SDK for the functionality to 'light up'.
I assume the configuration will also work from a Directory.build.props file instead of a csproj file?
I know you can run it on the solution level so I assume so yeah
I'm not sure - this feature is right decision. This is so "Microsoft way" feature. they built in support of build of Docker images into the dotnet sdk. It does tons of "magic" behind the scene. This is great for the simple cases like you showed in this video, however this feature complete isolated from the rest of the world. lets say we have CI/CD pipeline in Gitlab/github/GCP/Azure Dev OPS. those tools already supports build od Docker images, Podman containers, native K8s images. So Microsoft decided to introduce build of docker images using the cproj files.
What if we want to use podman instead?
What if we want to use k8s native containers?
What if I have custom security Auth process for building Docker containers?
I'm glad that Microsoft adding new features to the sdk, but I have some doubts as for correctness of this particular one.
Glad to see people thinking critically about these types of blatant vendor lock in measures.
@@itmeurdad Thanks for your feedback folks! Would it change your mind if I pointed you to the ko project for golang and the Jib project for Java, both of which use this model and are authored by Google?
In any case, we think that more ways to make containers are great, but the way _you_ want to make containers is the way that works best for your team/environment.
How about CI/CD piplines. Will this works??
Sure it does. I tried it in GitHub actions and it works
Nice step in right direction, but useless (for now) in real life ci/cd automation set ups, for various reasons, one of them would be that we for example use docker-daemon-less builds
The other one that in real life you will have templates and especially in cloud projects you have different programming languages and all of them could use docker so you’re building your domain tailored docker pipeline template and then java, dotnet, whatever can use it. This on the other hand can’t be used by anything else than dotnet
So yeah, i would say it’s good for noob :)
@@romanhrytskiv8845 This does do daemonless builds - that's one of our driving use cases. WE just also support pushing the generated container to a local container daemon for use in things like docker-compose, etc.
Having said that, definitely continue to use Dockerfiles if you rely heavily on automation that needs them!
Will this work with other containers engines? (e.g podman)
I haven't tested that so you'd need to give it a go
For development sure it simplifies things but containers aren't really built on dev machines. Even tho this might get by with Azure GitOps I can't really see how to build a container in kubernetes where there is no Docker daemon instead you get Buildah or Kaniko or similar builders. I only suspect this approach you demonstrated is actually talking to Docker daemon in the background but then again I might be wrong.
Nope, we're more like ko for golang or Jib for Java - we build the images by constructing the JSON manifests, config, and TAR file system layers in a spec-compliant way. This is what lets us a) build without a container builder engine, and b) reuse existing MSBuild metadata, properties, etc to remove the boilerplate and work so much faster.
Nice! I would love to know how to move away from using the dockercompose because that is my biggest headache in trying to move to vscode as my primary editor. (I know you like Jetbrains) lol
Has anyone else tested ContainerImageTags property? It seems not working and I am wondering if it's a known issue or I'm making a mistake somewhere?
Microsoft is really pushing towards DevOpsbut there is a long way to go.
On leaving the DockerFile, I would still go with the DockerFile. And I saw a lot of people build the Docker file via Docker-compose in the build context which I alos do't find it convient. so I always have My DockerFile and Docker-Compose files seperately. I'm planning to explore more deep via Github Actions and Kubarnets so my perspectives may change along the way.
This is indeed very cool, but on a production level, when you want to also include a database or other dependencies, you will still need to work with docker files, no ?
You need those in docker compose, but do you need them in your dockerfile?
@@nickchapsas I guess what I meant was that you can't really forego Docker and it's ecosystem while constructing the solution as a whole. Thanks and great content!
Do you really need docker or containers?
It depends on your setup
It's a nice feature, but it's not accurate to say it works for all situations. There are lots of cases where you would want to customize your image for security reasons for instance, where this *probably* wouldn't be enough.
Take Microsoft's new chiseled images. They don't come with CultureInfo support out of the box. How would you customize that image to add it in (the way Microsoft says you can) if using this approach?
I didn’t say that it works in all situation. I said most situations. It’s always a win when you can make adoption more accessible to people who only use the tool for basic stuff. When they need more advanced usage they can more to a custom dockerile
@@nickchapsas It might interest you that you can probably do this in .NET 6 as well (but you probably do need the .NET 7 SDK). At least it did last time I tested this out.
Isn't this thing became obsolete with .net aspire?