Cody, thank you so much for your content. I feel like you're one of the few content creators that is actually providing content for mid-level to senior+ engineers, which I deeply appreciate. Enough is enough with all this "create your Netflix/Twitter/Instagram clone" and similars. You're actually helping me getting my senior promotion (hopefully next year). Please keep it up.
Logging is huge, especially in larger orgs where uptime is mission critical. I am not a dev but work with the operations/ reliability team and logging allows our 24/7 team to identify issues faster and able to engage the correct team more accurately. We work with the devs and SREs to make the logging more accurate, create alerts and dashboards to make it easy for less tech operators to react to alerts that matter. Logging is boring but very useful lol
We use glitchtips self hosted version and since it’s compatible with all of Sentry’s SDKs there is a lot you can do with it. Also quite a bit easier to setup then then solution mentioned above
I would love a video explaining how to secure user data from devs having access to the server. For example, let's you're encrypting phone numbers, nothing stops a dev to access the server, take the encryption key and decrypt the fields. What are the different approach to solve this issue?
I've heard of this so many times but never actually implement it (I suppose as a web dev knowing that it exists is enough right?) There is one point that confuses me: Why ElasticSearch when Kibana is already a search + dashboard ? Isn't that redundant?
Hey Dev Cody, I wanna setup my custom back-end. I saw your videos and you're astonishing. Can you please create a video on how to setup a scalable application with laravel and docker. Or you can create a course for that! It'll be an ace of course.
Could you use ELK stack with a service like Sentry? How do they compare? It feels like Sentry is used for catching errors in a detailed manner (stacktraces) where as the ELK stack would catch errors and general system information. I would like to hear your input because this seems like a tool my team could use
So this type of logger; do they typically send the longs to the centralized logger as they happen, or do they store the logs locally then send them in batches, or is this dependent on the framework / project / config.
I work at Axiom (6:14 in the video). Our SDKs batch by default, but also have the option not to (as well as a lot of other config options to override the default behaviour. I don't know all the details about every other service, but in general I would expect them to not be too different in this regard.
Thanks for all the replies guys, I guess batching would usually make the most sense, cause you could just adjust the batch intervals if you need really up to date logs anyway. I can imagine there might be some systems where concurrent logging is required.
How would you implement client-side logging? Winston is only used for Node.js logging. Seem like I'll have to implement an API backend that I'll make a post request to when errors are caught.
yeah, you'll client will need to add logic to catch all exceptions and forward them to a authenticated backend endpoint so you know what errors happen for which users.
Dude you rock! Also mate, would you mind sharing the brand/model of your main monitor, please? I am looking for a good monitor for my Mac mini. Thanks a lot for all the shared knowledge!
@@WebDevCody it's getting too big and too slow to deploy, and iac seems better. Also the plugin ecosystem is kind of dead, we already have some patches on the serverless-offline family of plugins (dynamo, sqs...)
@@G0NZA11 I’d consider serverless framework as an IaC tool. our project started with serverless framework, but we found aws cdk is too slow and was brittle 4 years ago. Now we deploy everything using terraform, including all our lambda function (which kind of sucks to be honest), but I’ve found terraform to be a much better IaC tool. SST uses aws cdk under the hood, so I’d also be a bit cautious migrating to it without testing stuff out
For clarity, this would NOT be a solution for browser client side logging. This sounds more for server side Node systems or even lower level software/hardware configurations. For client side debugging, having tests run is also a great proactive solution.
Cody, thank you so much for your content. I feel like you're one of the few content creators that is actually providing content for mid-level to senior+ engineers, which I deeply appreciate. Enough is enough with all this "create your Netflix/Twitter/Instagram clone" and similars. You're actually helping me getting my senior promotion (hopefully next year). Please keep it up.
Glad to hear it!
I agree with you. Cody is doing something different here. Nice job. Really
I agree, even though im a junior this stuff really sets a high bar for me on what I should know and learn, etc
Logging is huge, especially in larger orgs where uptime is mission critical. I am not a dev but work with the operations/ reliability team and logging allows our 24/7 team to identify issues faster and able to engage the correct team more accurately.
We work with the devs and SREs to make the logging more accurate, create alerts and dashboards to make it easy for less tech operators to react to alerts that matter.
Logging is boring but very useful lol
Thank you for this kind of content that gives newcomers an idea what the industry is using outside of common stuff like React, NodeJS, etc
If possible please make a deep dive video on ELK stack, really love how simplify things...
I like the overview videos a lot. It'd be really nice to have a video on how to set this stuff up also
Thanks for the last few videos Cody, I’ve recently added logs on my prod apps and they’ve picked up so many things I missed! Really appreciate it
Thanks for the video 😊 Looks similar to Graylog which we use at my current job for application side logging.
You make me a better developer. Thank you :)
Even though I don't understand much of these terms i come here just to expose myself to these new vocabulary. And later on learn them.
Can we get an tutorial on elk stack please
We use glitchtips self hosted version and since it’s compatible with all of Sentry’s SDKs there is a lot you can do with it. Also quite a bit easier to setup then then solution mentioned above
Love your videos and learning a lot. thank you ❤️
Oooo a centralized logger huh lol you’re doing great babe! ❤u
Love you! 😘
I would love a video explaining how to secure user data from devs having access to the server. For example, let's you're encrypting phone numbers, nothing stops a dev to access the server, take the encryption key and decrypt the fields. What are the different approach to solve this issue?
Idk if you can other than encrypt on the ui and never let any dev know the private keys used to decrypt. You don’t trust your devs?
@@WebDevCody I trust my dev, but I've always been wondering how big companies with a lot at stakes do
Nice!! Thanks 🙏
I've heard of this so many times but never actually implement it (I suppose as a web dev knowing that it exists is enough right?)
There is one point that confuses me: Why ElasticSearch when Kibana is already a search + dashboard ? Isn't that redundant?
Kibana just communicates directly with elasticsearch. Elasticsearch is where all the data is stored
@@WebDevCody ah I see I see ty for the reply
Hey Dev Cody,
I wanna setup my custom back-end. I saw your videos and you're astonishing.
Can you please create a video on how to setup a scalable application with laravel and docker.
Or you can create a course for that! It'll be an ace of course.
I don’t really use laravel, but I’d just find an existing docker compose file, I’m sure there are a lot
@@WebDevCody Oh Thank you so much
Could you use ELK stack with a service like Sentry? How do they compare? It feels like Sentry is used for catching errors in a detailed manner (stacktraces) where as the ELK stack would catch errors and general system information. I would like to hear your input because this seems like a tool my team could use
So this type of logger; do they typically send the longs to the centralized logger as they happen, or do they store the logs locally then send them in batches, or is this dependent on the framework / project / config.
I work at Axiom (6:14 in the video). Our SDKs batch by default, but also have the option not to (as well as a lot of other config options to override the default behaviour. I don't know all the details about every other service, but in general I would expect them to not be too different in this regard.
we batch our logs, usually writes 100 at a time or if 10 seconds have passed
I’m assuming it’s based on configuration, but batching would be the most performant for large amounts of logs I’d think
Thanks for all the replies guys, I guess batching would usually make the most sense, cause you could just adjust the batch intervals if you need really up to date logs anyway.
I can imagine there might be some systems where concurrent logging is required.
How would you implement client-side logging? Winston is only used for Node.js logging. Seem like I'll have to implement an API backend that I'll make a post request to when errors are caught.
yeah, you'll client will need to add logic to catch all exceptions and forward them to a authenticated backend endpoint so you know what errors happen for which users.
Dude you rock! Also mate, would you mind sharing the brand/model of your main monitor, please? I am looking for a good monitor for my Mac mini. Thanks a lot for all the shared knowledge!
I use my laptop monitor
@@WebDevCody thanks a lot Cody!
great video thanks
Whats the price
Have you used sst with kafka or msk? How is the support for sst to work offline? (Not only live lambda but db and others).
Thanks!
I only use sst for deploying, not for running anything locally
@@WebDevCody gonna have to give a little try to see if I can replace serverless offline. Thanks for your time and your content, have a good one!
@@G0NZA11 what issues are you having with serverless offline? I've used the serverless framework as well, it seems to work ok
@@WebDevCody it's getting too big and too slow to deploy, and iac seems better. Also the plugin ecosystem is kind of dead, we already have some patches on the serverless-offline family of plugins (dynamo, sqs...)
@@G0NZA11 I’d consider serverless framework as an IaC tool. our project started with serverless framework, but we found aws cdk is too slow and was brittle 4 years ago. Now we deploy everything using terraform, including all our lambda function (which kind of sucks to be honest), but I’ve found terraform to be a much better IaC tool. SST uses aws cdk under the hood, so I’d also be a bit cautious migrating to it without testing stuff out
Great video! What log service do you use in your saas application?
ELK and aws cloudwatch
@@WebDevCody from your videos i understood that you don't use vps, how do you deploy your elk stack? thanks again!
@@LawZist we use aws opensearch which has kibana built in
For clarity, this would NOT be a solution for browser client side logging. This sounds more for server side Node systems or even lower level software/hardware configurations. For client side debugging, having tests run is also a great proactive solution.
There are ways to ship any front end exceptions to your api so that are also indexed into you logging system
Express? The dark side?
Good 👍🏻
How is this different from a splunk logs please explain
It’s the same, but splunk isn’t free and I hear it’s better
just build a monolith