Great explaination, as always! This simplified analogy between restaurant and IDP becomes extremely useful for those who needs to have a high level idea about what an IDP is and what added values brings to the company productivity, while still hiding the complexity and the necessary integrations to implement a solid solution. The Waiter example clearly highlights the boundary that separates users point of view from the backend details Keep it up! 💪🏼💯
Thank you for this video, I really liked the comparison between platform engineering and restaurants, it really clarifies the concepts behind it. I bet that while many people already know how to implement the other parts (service catalog, services, ...) at least up to a certain point it's still unclear how to get the feedback part done right. Maybe it could be an idea for an upcoming video?
@DevOpsToolkit. Great mental model of Platform Engineering - thanks a lot for this! In this mental model, and in your opinion, would be a YAML file a proper API for stream aligned teams to "order" things from Platform engineering? (like a simple definition of a Hosted Zone in AWS, defined as custom YAML and parsed / processed from some Platform Engineering tool). And, again, would it fit the restaurant mental model, if developers define this 2-4 lines in YAML by themselves and create a PR for that, which will be reviewed and merged afterwards by Platform Engineering team?. Or are we speaking about a Platform only, if we have a UI for devs or kind of CLI for "ordering" things, without the need for devs to write code (even if it is a simple YAML) and create a PR?
Yes. YAML is the most common "language" to talk to APIs and is the format people should use to "order" resources. Why would platform engineers review those PRs? They created services that devs are using. Right? That's similar to AWS creating services you're using. AWS folks do no review your PRs. The reason i don't like that is because platform engineers keep being a bottleneck. People would still need to wait for those reviews. Instead, I would suggest creating policies that would validate that everything is okay.
@@DevOpsToolkitin general, fully agree with you. But it becomes then difficult in companies with super strong 4-eyes principle policy for anything pushed into the main branch (automatic validation of the YAML, for instance with CUE is still possible, clear).
One thing I’ve been a bit confused about is how GitOps practices interact with this kind of platform. When you provide your users with an API to order their own infrastructure, would the API be interacting with a git repo, or is this a different thing that falls outside the scope of GitOps?
Assuming that those resources are managed by kubernetes, it's up to you to choose how you interact with it's API. You can use gitops, or kubectl, or helm, or anything else. That is the beautify or APIs. You are not concerned how people consume it. Personally, i tend to use gitops to interact with kube API.
Ah, okay. I was imagining that the API was something injected between the user and Kubernetes, but it sounds like you would recommend letting Kubernetes be the API in this case. From there, it follows that it’s on the user to decide how they interact with Kubernetes. I suppose to stick with the restaurant analogy, the user decides whether to go to the restaurant and sit at a table or to order to go or to use an app to order delivery.
Hi, great video, i didnt understand the part about observability, how can one filter and transform the logs (technically) which frameworks recommend? Also how would one expect the logs to be?
Logs can be transformed by services themselves. They can output logs that matter to end users and keep those for service providers separate. That's what, for example, AWS does. It does not give us, the users, all the information but only parts that matter to us. As an example, I am trying to push crossplane project to enable some kind of filtering so that service providers, people building compositions, can choose what matters to end users. Alternatively, you can use log shippers like fluentd or promtsil to filter logs based on whichever criteria you define.
Very nice comparison and very well explained! Is there a simple tool for creating menus from APIs, besides kubectl explain or IDE's Kubernetes plugin (which are great but kinda "low level")?
Unfortunately, as far as I know, there are no such tools that are user-friendly. That is the part that is very surprising. I do not fully understand why none of "service catalog" tools use Kube API to discover what's available instead of asking the API. That being said, even though there is no plugin in Backstage that can do that, it should be trivial to develop one and I know that at least a few companies are doing it (without open sourcing it). Hoopefully, someone will create a plugin. I do not work with TS so I can't help with that one myself. Port, my favorite tool in that area is working on it. That was one of my complaints and they took it to heart. I can't say when it will be release except to say that I believe we're close to getting it.
I've spent years of literal pain, suffering and stress until I've realized that I wasn't the problem, just -at most- a minor part of the problem when it comes to culture, management and horrible practices, even if we don't take in count the poor ownership/governance which it's seems to be a "global pandemic" hahahaha, regards from Chile!
I have been thinking about this many many times and you bring it to the point! Awesome!
Glad to hear it!
Great explaination, as always! This simplified analogy between restaurant and IDP becomes extremely useful for those who needs to have a high level idea about what an IDP is and what added values brings to the company productivity, while still hiding the complexity and the necessary integrations to implement a solid solution. The Waiter example clearly highlights the boundary that separates users point of view from the backend details
Keep it up! 💪🏼💯
Thank you for this video, I really liked the comparison between platform engineering and restaurants, it really clarifies the concepts behind it. I bet that while many people already know how to implement the other parts (service catalog, services, ...) at least up to a certain point it's still unclear how to get the feedback part done right. Maybe it could be an idea for an upcoming video?
Adding it to my to-do list...
@DevOpsToolkit. Great mental model of Platform Engineering - thanks a lot for this! In this mental model, and in your opinion, would be a YAML file a proper API for stream aligned teams to "order" things from Platform engineering? (like a simple definition of a Hosted Zone in AWS, defined as custom YAML and parsed / processed from some Platform Engineering tool).
And, again, would it fit the restaurant mental model, if developers define this 2-4 lines in YAML by themselves and create a PR for that, which will be reviewed and merged afterwards by Platform Engineering team?. Or are we speaking about a Platform only, if we have a UI for devs or kind of CLI for "ordering" things, without the need for devs to write code (even if it is a simple YAML) and create a PR?
Yes. YAML is the most common "language" to talk to APIs and is the format people should use to "order" resources.
Why would platform engineers review those PRs? They created services that devs are using. Right? That's similar to AWS creating services you're using. AWS folks do no review your PRs. The reason i don't like that is because platform engineers keep being a bottleneck. People would still need to wait for those reviews. Instead, I would suggest creating policies that would validate that everything is okay.
@@DevOpsToolkitin general, fully agree with you. But it becomes then difficult in companies with super strong 4-eyes principle policy for anything pushed into the main branch (automatic validation of the YAML, for instance with CUE is still possible, clear).
You're my mentor now!!
One thing I’ve been a bit confused about is how GitOps practices interact with this kind of platform. When you provide your users with an API to order their own infrastructure, would the API be interacting with a git repo, or is this a different thing that falls outside the scope of GitOps?
Assuming that those resources are managed by kubernetes, it's up to you to choose how you interact with it's API. You can use gitops, or kubectl, or helm, or anything else. That is the beautify or APIs. You are not concerned how people consume it.
Personally, i tend to use gitops to interact with kube API.
Ah, okay. I was imagining that the API was something injected between the user and Kubernetes, but it sounds like you would recommend letting Kubernetes be the API in this case. From there, it follows that it’s on the user to decide how they interact with Kubernetes. I suppose to stick with the restaurant analogy, the user decides whether to go to the restaurant and sit at a table or to order to go or to use an app to order delivery.
@michaeljuliano8839 that's correct. I believe that kubernetes API is the best bet we have when building API for a platform.
Hi, great video, i didnt understand the part about observability, how can one filter and transform the logs (technically) which frameworks recommend? Also how would one expect the logs to be?
Logs can be transformed by services themselves. They can output logs that matter to end users and keep those for service providers separate. That's what, for example, AWS does. It does not give us, the users, all the information but only parts that matter to us. As an example, I am trying to push crossplane project to enable some kind of filtering so that service providers, people building compositions, can choose what matters to end users. Alternatively, you can use log shippers like fluentd or promtsil to filter logs based on whichever criteria you define.
Greate compare, thank you! Interesting who is sommelier in this restaurant?
That would be the mainframe guy.
@@DevOpsToolkit or mainframe AI :)
Very nice comparison and very well explained!
Is there a simple tool for creating menus from APIs, besides kubectl explain or IDE's Kubernetes plugin (which are great but kinda "low level")?
Unfortunately, as far as I know, there are no such tools that are user-friendly. That is the part that is very surprising. I do not fully understand why none of "service catalog" tools use Kube API to discover what's available instead of asking the API.
That being said, even though there is no plugin in Backstage that can do that, it should be trivial to develop one and I know that at least a few companies are doing it (without open sourcing it). Hoopefully, someone will create a plugin. I do not work with TS so I can't help with that one myself.
Port, my favorite tool in that area is working on it. That was one of my complaints and they took it to heart. I can't say when it will be release except to say that I believe we're close to getting it.
@@DevOpsToolkit oh that is indeed surprising. Thanks a lot for your answer!
My videos often contain "sublimal" messages to other projects to influence their roadmaps. I'm sure we'll have that soon 🙂
I've spent years of literal pain, suffering and stress until I've realized that I wasn't the problem, just -at most- a minor part of the problem when it comes to culture, management and horrible practices, even if we don't take in count the poor ownership/governance which it's seems to be a "global pandemic" hahahaha, regards from Chile!