Meet Grafana LOKI, a Log Aggregation System for EVERYTHING

แชร์
ฝัง
  • เผยแพร่เมื่อ 7 ธ.ค. 2024

ความคิดเห็น • 355

  • @TechnoTim
    @TechnoTim  3 ปีที่แล้ว +68

    OK, nerd talk: What's your favorite logging system? (would this ever come up in a normal conversation???)

    • @axelolaussonholtenas2970
      @axelolaussonholtenas2970 3 ปีที่แล้ว +6

      LibreNMS

    • @camerontgore
      @camerontgore 3 ปีที่แล้ว +12

      For prod systems I really love Splunk

    • @bruceritchie7613
      @bruceritchie7613 3 ปีที่แล้ว +9

      I've tried graylog, elk, datadog, cloud watch and a few others. Most are overkill, some are crazy expensive and none are perfect (or even great) Loki was on the 'check out at some point' list

    • @VoislavVasiljevski
      @VoislavVasiljevski 3 ปีที่แล้ว +11

      ELK (elasticsearch, logstash, kibana)

    • @michaelkasede1489
      @michaelkasede1489 3 ปีที่แล้ว +9

      @@VoislavVasiljevski ELK is definitely on the "one of the best" list. It's only down side is, it's resource intensive.

  • @forzaracer
    @forzaracer ปีที่แล้ว +21

    These guides are insane. No one goes to the detail that you do. You explain why and how commands and config files work rather that just reading them. Love your channel!

  • @thegeogian
    @thegeogian 3 ปีที่แล้ว +65

    "Make sure you're in a directory... I dont know where else you'd be..." Made me laugh so hard xD

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว +22

      haha! I was going to re-record that part but I was like "just roll with it"

    • @thegeogian
      @thegeogian 3 ปีที่แล้ว +12

      @@TechnoTim best decision

    • @HTOP1982
      @HTOP1982 ปีที่แล้ว

      Root?

  •  3 ปีที่แล้ว +25

    Great tutorial! One thing to note if you want to log specific docker containers you don't need to install promtail or change the deamon file. Just need the docker plugin for Loki and extra configuration in docker-compose for your specific containers that should use dedicated Loki logging driver.

    • @ClaudeAlexandreRochatProfile
      @ClaudeAlexandreRochatProfile ปีที่แล้ว +3

      Hello, this sound interesting can you give an example or link ? Thanks

    • @chelo-homelab
      @chelo-homelab 9 หลายเดือนก่อน

      Yes please, that will help a ton!

  • @prasaddotcom
    @prasaddotcom 2 ปีที่แล้ว

    Hi , I tried to extract logs from mssql server using promtail , loki grafana I window environment, logs are coming to grafana dashboard. Problem here is in logs words are displaying as I n s t a n c e instead of instance. Can you please help with solution to resolve my issue.

  • @resetToDefaultSettings
    @resetToDefaultSettings 2 ปีที่แล้ว +2

    I've watched so many deployment videos trying to get a better all around understanding on all the NMS out there, what they can do, how they operate under the hood, etc. This by far is the absolute best I have seen and definitely earned you a sub. Most deployment videos tend to be quite dry, very un-explainative, and usually feel like a 'just do this real quick and it works' video. Thank you!!!

  • @rupert8606
    @rupert8606 2 ปีที่แล้ว

    Fantastic tutorial... everything explained really well, and worked perfectly.. great way to get up and running with Loki / Promtail / Grafana quickly!!

    • @TechnoTim
      @TechnoTim  2 ปีที่แล้ว +1

      Glad it helped!

  • @brightly9368
    @brightly9368 2 ปีที่แล้ว +2

    IDK may be I am wrong fix me if so. The man talks as if he is talking inside my brain. Awesome job

  • @AshfaqueAhmed-b8d
    @AshfaqueAhmed-b8d ปีที่แล้ว +2

    Yo bro, I like your video and the way explained every single bit and steps are just beyond awesome. Thanks for this quality content. God bless you my friend.

  • @toshy50
    @toshy50 3 ปีที่แล้ว +2

    Thanks man! This is just so much easier and less troublesome then configuring a proper ELK stack.

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      Glad it helped!

  • @Harry-hi3kr
    @Harry-hi3kr 3 ปีที่แล้ว +7

    Been using Loki for over 2 years for work and play. Even out performed some enterprise level logging we tried too!

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว +1

      Nice! Thanks for sharing!

  • @paulnieuwkamp8067
    @paulnieuwkamp8067 3 ปีที่แล้ว +3

    Hi Tim, you look like the kind of guy to keep (their system) up to date, but for those that may not know, there was a high-severity, zero-day vulnerability for Grafana a few days ago that enabled remote access to local files. Anybody that could access the site could access any file on the system, like for instance /etc/passwd. There are proof-of-concepts publicly available so it takes hardly any skill for script kiddies to start scanning.
    Grafana released a fix right away. If you have not already, install the latest versions / patches!

    • @toshy50
      @toshy50 3 ปีที่แล้ว +1

      +1 I've noticed this as well, but by using the "latest" tag the fixed version should be pulled next time you up the containers with the "--build" argument. Just want to add a small note for others as well, that using "latest" is not a best practice, and you should specify the version you want to use (especially in production), else you could end up pulling a vulnerable version without even realizing.

  • @UzielBueno-xd2ud
    @UzielBueno-xd2ud ปีที่แล้ว +2

    Thanks! This was informative! It's worth mentioning that Promtail is not required if you are setting Loki as the Docker daemon default logging driver

  • @ThomasWilent
    @ThomasWilent 2 ปีที่แล้ว

    5 min in and I can already tell your my new favorite tech guide guy. The detail is really welcome!

  • @renobodyrenobody
    @renobodyrenobody 2 ปีที่แล้ว

    Excellent! I am a long time user of Grafana and do a lot with it, but now the job is to manage the logs: here we are! Loki is the tool we want to deploy. Thanks a lot for your video, brillant, will be very helpful!

  • @therus000
    @therus000 ปีที่แล้ว

    Good day
    i got this error when install plugin
    docker plugin enable loki
    Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/etc/resolv.conf" to rootfs at "/etc/resolv.conf": stat /etc/resolv.conf: no such file or directory: unknown
    ls -l /etc/resolv.conf
    lrwxrwxrwx 1 root root 32 Dec 26 16:15 /etc/resolv.conf -> /run/systemd/resolve/resolv.conf

  • @weitanglau162
    @weitanglau162 3 ปีที่แล้ว +36

    Please make a video for Tempo as well!
    Then we can see how you monitor logs, traces and metrics using grafana and setting alerts!
    That would be cool!

    • @annanayagarwal9836
      @annanayagarwal9836 3 ปีที่แล้ว +2

      Tempo Maintainer here and YES! Would love to see that!

  • @TrueCharts
    @TrueCharts 3 ปีที่แล้ว +2

    We went ahead and added Grafana and Loki to our TrueNAS SCALE Apps as well :)
    Took some work, but well worth it ^^

  • @slothchunk
    @slothchunk ปีที่แล้ว

    thank you for validating my use of nano. we are nano brothers, brother.

  • @jaygreentree4394
    @jaygreentree4394 3 ปีที่แล้ว +4

    Before watching this video I was thinking to myself I wish Tim had a grafana tutorial.

  • @leagueoflegendswildriftnep2236
    @leagueoflegendswildriftnep2236 3 ปีที่แล้ว

    I also used 24:24 and its really awesome, but now there's issue i could not fix, in kubernetes 1.22.x chart is deprecated and solution is to update clusterrole to /v1 from /v1beta1 but no idea in helm how to?

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      We'll have to wait for grafana to update this. This is just a deprecation warning, it still works.

  • @pivertd
    @pivertd ปีที่แล้ว

    Now, because of you, I want it ! Thanks for the great work.

  • @itskagiso
    @itskagiso 2 ปีที่แล้ว +2

    This was a lengthy but worth every second. Managed to get this up and running and Im happy but I would love a guide on getting syslog integrated

    • @TechnoTim
      @TechnoTim  2 ปีที่แล้ว +2

      Nice work! Noted!

  • @JohnMatthew1
    @JohnMatthew1 ปีที่แล้ว

    GREAT VIDEO, got me through Loki and Syslog, thank you Tim

  • @belbono
    @belbono 2 ปีที่แล้ว +1

    The docker loki plugin had a serious problem, so it was not an option for production use. I don´ t know if this issue has been solved. The last time I was reading through the related issues the developers said, that they might not be able to provide a solution anytime soon. Let me describe: If your container host has high IO load and/or the loki server is not reachable for some reason the whole docker daemon hang up, because it wasn´t able to write the logs. The worst possible situation is when shutting down multiple containers. It wasn´ t possible to define some timeout. It was required to kill the daemon process and remove the container files (/var/lib/docker/containers) the daemon normally manages.
    I never used the plugin again. I´m happy with configuring the log property of a container and have promtail to scrape this. The result is the same, but without the possibility of a blocked daemon.

    • @TechnoTim
      @TechnoTim  2 ปีที่แล้ว

      Thanks for sharing your expertise and experience!

  • @wildflowers465
    @wildflowers465 2 ปีที่แล้ว

    Remap Caps Lock to Ctrl if your control key is broken! Easier to reach, I do this by default. Great tutorial.

  • @MarkDodgson
    @MarkDodgson 2 ปีที่แล้ว +4

    Great video, I have a question about the promtail adding docker section. when you set up the docker loki driver, the daemon config was sending logs directly to loki so I am not sure what the promtail configuration actually did in this setup. Additionally, the promtail config was set to push from /var/lib/containers.... but this would have ben for the folder within the container of promtail only and not host. Have I missunderstood something?

    • @WoBuGs-yt
      @WoBuGs-yt 9 หลายเดือนก่อน +1

      You are correct, I believe the loki driver method is completely different form the promtail one with the docker pipeline stage
      See the other comment I just posted about this for more details :)

  • @GeorgeKowalski
    @GeorgeKowalski 3 ปีที่แล้ว

    Thanks!

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      Wow! Thank you!

  • @kokizzu
    @kokizzu 3 ปีที่แล้ว +1

    you can use ./bla (relative path) for binding volume

  • @dstdg18
    @dstdg18 3 ปีที่แล้ว

    What plugin causes the yellow, green, and purple stripes in Tim's text editor to show which indent he's at? Time code 4:00 as a reference.

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      It's indent-rainbow and I love it!

  • @johnconnor7778
    @johnconnor7778 2 ปีที่แล้ว +2

    Awesome video, Tim! Could you share your dashboard config from 0:47? It looks like exactly what I need.

  • @royalcanadianbearforce9841
    @royalcanadianbearforce9841 2 ปีที่แล้ว +2

    Just found your channel. I freaking LOVE these deep dives. Thank you so much for your hard work and insights!

    • @TechnoTim
      @TechnoTim  2 ปีที่แล้ว

      Thank you and welcome!

  • @vigscodes
    @vigscodes ปีที่แล้ว +2

    Is there a GitHub repository showing the various configuration files? That would be helpful.

  • @magnoliaraoul
    @magnoliaraoul 3 ปีที่แล้ว +2

    I was waiting for this video. Very well explained as always, thanks Tim !
    To take a quick glance at my docker logs, i like to use Dozzle, a very small and straight to the point tool, but it does the job

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      Thank you! Glad you liked it!

  • @lolikpof
    @lolikpof 7 หลายเดือนก่อน +1

    Can this be used in a closed source commercial product? The AGPL license is concerning. Even though no changes are made to the source code, and it would only be used for internal purposes, section 13 of the license is concerning. The software is a web application, so it would be accessible over a network, and while logs would obviously not be shown to end users, and technically aren't even necessary for the application to work, they are necessary for development and debugging.. Can't figure out whether it can be used in my company or not

  • @jharding65
    @jharding65 3 ปีที่แล้ว +1

    This is great and timely! Awesome vid! You make logging look fun! Gonna try it out right now!

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      Thank you! Let us know how it goes!

  • @Deffcolony
    @Deffcolony 2 ปีที่แล้ว

    Hi Techno Tim, Docker compose does not work. Services are not exposed... I get a 404 page not found at port 3100 the port 3000 is not exposed at all... What do i have to do now?

    • @TechnoTim
      @TechnoTim  2 ปีที่แล้ว

      Not sure, possibly check the config again?

  • @simonostendorf6280
    @simonostendorf6280 3 ปีที่แล้ว +3

    New monitoring:
    error and alert monitoring: CheckMK
    performance monitoring: Prometheus + Grafana
    log monitoring: Loki + Grafana
    status monitoring: uptime kuma

    • @shekharnandkoemarsing158
      @shekharnandkoemarsing158 3 ปีที่แล้ว

      Only need a trace management solution and you are all set up

    • @MrNecrotik
      @MrNecrotik 3 ปีที่แล้ว

      Starting to get into k8s but wouldn't your alerting and status all be in Grafana?

    • @simonostendorf6280
      @simonostendorf6280 3 ปีที่แล้ว +1

      @@MrNecrotik Yes its inside K8S. I think i will use two different grafana instances

  • @larsvontrierpung9337
    @larsvontrierpung9337 ปีที่แล้ว

    Thanx a lot. Something that would be cool is to have a guide howto setup loki,mimir,tempo with grafana and prometheus in k8. Thanks!

  • @nada176
    @nada176 3 ปีที่แล้ว +2

    Messing with influxdb but will check out Loki next. Wondering how Loki performs with TBs of logs without switching to cloud storage. Thanks for the video!

  • @MadChristianX
    @MadChristianX 2 ปีที่แล้ว +1

    Thank you for your great work, i think i have nearly a copy of your homelab in my home

  • @michaelganesan4578
    @michaelganesan4578 2 ปีที่แล้ว

    Thank you for the demo. I really appreciate it. I am in the process of setting up a syslog server.

  • @michaelcooper5490
    @michaelcooper5490 ปีที่แล้ว

    Awesome tutorial Tim, as usual you nailed thank you for your hard work.

  • @cpt_Ruckus
    @cpt_Ruckus 3 ปีที่แล้ว +2

    Loki is great, very versatile. Be warned tho if you have limited storage space the loki database can grow in size quick!
    Edit: loki doesn't trim old logs by default.. I only keep logs up to two weeks old. Huge data saver!

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว +1

      Good tip!

    • @djdubd
      @djdubd 3 ปีที่แล้ว +1

      What config items do you need to set to change the default storage locations? I'm looking at their config reference and having trouble making sense of it

    • @SebastianSastre
      @SebastianSastre ปีที่แล้ว

      This is the first I though when Tim was explaining the setup.

  • @fretbuzzly
    @fretbuzzly 7 หลายเดือนก่อน +10

    I swear software is annoying. If you look at the official docs to install this thing using docker-compose there's no mention of most of what Tim talks about. "Creake a loki folder and run this wget. Done." And if you look at the yaml file from the repo it's completely different than Tim's. If software developers knew how to communicate they'd be dangerous.

    • @oxJake
      @oxJake 5 หลายเดือนก่อน +1

      It’s because of the age of the video software has updates things change

  • @chrisumali9841
    @chrisumali9841 3 ปีที่แล้ว

    Thanks for the demo and info, have a great day

  • @TheApeMachine
    @TheApeMachine 3 ปีที่แล้ว +2

    You know it's like really simple to develop plugins for Grafana? In my recent job we had to develop some very custom dashboards on the data we were collecting and it turns out Grafana has this whole eco-system to develop not only panel plugins, or data source plugins, but even app plugins, which integrate directly into the Grafana UI. It's really nice, since you get all user management and auto-reloading of data mechanisms for free :) And when I say easy, I mean really hard if you don't know it's basically React (and I didn't know React). But once you figure it out, it's easy.

    • @TheApeMachine
      @TheApeMachine 3 ปีที่แล้ว

      It was an interesting video though, I had been wondering about this logging stack for a while and if I should chase it. The thing is, Kibana is just too powerful. Like even having unstructured data and being able to structure it on the fly is just everything.

  • @hassanad94
    @hassanad94 5 หลายเดือนก่อน

    Thanks for this tuto. :) I get a lot of value from it :D

  • @Stoney_Eagle
    @Stoney_Eagle 3 ปีที่แล้ว

    You can't start logging this professional now dude, I stayed hidden so far from you but now it's impossible 🤣
    You have so many docker tutorials and you're actuality running them in your house that... I have to...
    I hereby declare you the docker King of TH-cam!

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว +1

      Thanks 😅. I did get a little enterprise on this one, did I? 😆

    • @Stoney_Eagle
      @Stoney_Eagle 3 ปีที่แล้ว

      @@TechnoTim This is what I want in my feed, some good knowledge instead of that Tiktok crap. 😉

  • @pantatzz
    @pantatzz 2 ปีที่แล้ว

    Gratefull i found this channel, thanks sir. Greeting from indonesia

  • @rupakrath637
    @rupakrath637 ปีที่แล้ว

    Thank you so much for the knowledgeable session

  • @sevanchandra1
    @sevanchandra1 2 ปีที่แล้ว

    You are my inspiration!! Great job!!

  • @uzairnizamuddin1837
    @uzairnizamuddin1837 2 ปีที่แล้ว +1

    I would like to reach a position where my ctrl and enter breaks. How can I do it?

  • @grouper1596
    @grouper1596 6 หลายเดือนก่อน

    Thank you for the video. Helped a lot!

  • @NFvidoJagg2
    @NFvidoJagg2 ปีที่แล้ว

    for the syslog, see if you can point the logs towards a dummy ip address. this way if someone gets into you system, they won't be able to find and mess with the logging server

  • @abdulrahmanmohammed1382
    @abdulrahmanmohammed1382 ปีที่แล้ว

    Awesome video @TechnoTim . Around the 15:24 mark, he was talking about how you would usually check Docker logs then scrolled down his container names, how did he do that? I'm assuming it is a particular shell, maybe? Anyone knows?

    • @TechnoTim
      @TechnoTim  ปีที่แล้ว

      I am using zsh with oh-my-zsh and the docker plugin enabled. Search my website for zsh. I have examples.

  • @9910313742
    @9910313742 2 ปีที่แล้ว

    I want to send only Error Logs and need to define that in promtail configuration file , so i do not need to send all logs to loki before sending logs it will filtered and send only error logs that is required , pls help ???

  • @JasonTurner
    @JasonTurner 3 ปีที่แล้ว

    Great presentation of some solid content! Thanks for sharing!

  • @lucianopacione8755
    @lucianopacione8755 ปีที่แล้ว +1

    Hi! Before all, amazing video!!
    But, I'm having an issue when I try to do the first query, I see "No logs volume available". I already checked /ready and /metrics and looks good. Do you know which can be the problem?
    Thanks a lot

    • @udaydito721
      @udaydito721 18 วันที่ผ่านมา

      Hi, is this fixed?
      I'm also facing similar issue
      Thanks,

  • @mvaldes
    @mvaldes 3 ปีที่แล้ว +1

    i usually do Elasticsearch, i know its heavy in use and maintenance but also plugs into other things like my wiki so its great. Loki does look promising tho.

  • @gr4yweb
    @gr4yweb 3 ปีที่แล้ว

    Thank you for your video. I enjoyed it very much and I know what I will do when I have some more free time around christmas this year.

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      Thank you! Me too!

  • @Nikzww187
    @Nikzww187 3 ปีที่แล้ว

    this tutorial was awesome, appreciate you!

  • @Travis94ZX
    @Travis94ZX 2 ปีที่แล้ว

    Unfortunately, I found that the syslog functions do not work with my firewall (SonicWALL) and promtail. Looking at some dashboards on Grafana related to SonicWALL it says something about setting up rsyslog in front of promtail to get it to work.

  • @elmoustaphamalick9925
    @elmoustaphamalick9925 25 วันที่ผ่านมา

    Thanks man you made my day :)

  • @dustinjorden6594
    @dustinjorden6594 ปีที่แล้ว

    receiving an error when running docker-compose up -d
    I did try doing this on my own before and thought i cleared everything from that but not sure what i did.
    any help would be appreciated
    ERROR: Named volume "home/serveradmin/docker_volumes/grafana:/var/lib/grafana:rw" is used in service "grafana" but no declaration was found in the volumes section.

  • @MinhHaiNehehe
    @MinhHaiNehehe 22 วันที่ผ่านมา

    Hi , i would like to say thank you !

  • @rajuhs3086
    @rajuhs3086 3 ปีที่แล้ว

    1. Is Loki-Promtail stack suitable for aggregating and shipping logs that are generated at microsecond level?.
    2. When logs are available in multiple files, promtail ships the logs in round robin fashion, spoiling the order of the logs. (Let's say logs generated through rolling file appender fashion).
    Any comments will be helpful.

  • @ameenjohnson1061
    @ameenjohnson1061 3 ปีที่แล้ว +2

    Great video! I installed grafana, prometheus, loki and promtail on my Proxmox host and I am really liking it. One question: is there an easy way to get promtail to get logs from my Proxmox LXC containers and VMs? Or do I need to install promtail into each LXC/VM in order to get the logs into loki?

    • @reyastaroth
      @reyastaroth ปีที่แล้ว +1

      Hello, I have LXC containers (15) with dockers running inside, have you found a way to monitor them without installing the agent on all of them?

  • @grunkasvunka2988
    @grunkasvunka2988 2 หลายเดือนก่อน

    Awesome video!

  • @halo64654
    @halo64654 2 หลายเดือนก่อน +2

    Your documentation (as far as copy & pasting goes) does not work anymore. Docker containers will not be able to find directories, or run without root, and will be stuck in a restart loop.

  • @AshishGupta-oc2cz
    @AshishGupta-oc2cz 2 ปีที่แล้ว

    Very Nicely Explained, keep it up.

    • @TechnoTim
      @TechnoTim  2 ปีที่แล้ว

      Glad you liked it

  • @project_mini_hero
    @project_mini_hero 4 หลายเดือนก่อน

    Hey Tim, great content as per usual!! I just saw your video and been struggling to set up logging from a firewall with this especially since promtail will be deprecated next year and they are switching to alloy. I wanted to know what are you using today in 2024 and if your still use this setup, have you migrated to alloy instead ?

  • @ThorstenS-linux
    @ThorstenS-linux 3 ปีที่แล้ว

    really nice content dude!

  • @scottezinn8555
    @scottezinn8555 2 ปีที่แล้ว +1

    Hey Tim - great information in your videos! Through lots of tinkering, I've managed to get k3s, cert-manager, rancher, traefik with tls certs up and running under Proxmox (I backed up my whole server, reformatted with Proxmox, created a VM and restored the server which runs 35+ containers in docker for my homelab). I want to get Grafana Loki but also want the prometheus and alerting too from your monitoring video. It looks like installing monitoring via the marketplace gets charts etc but do you just add loki rather than the loki stack? Adding loki stack seems to add things but there were no charts by default installed. Again, thanks for your awesome work. Your homelab machinery is sweet!

    • @TechnoTim
      @TechnoTim  2 ปีที่แล้ว +1

      Hi! Thank you! Nice stack! Yeah, if you just want loki and grafana use the lokie stack (I have the commands in my docs) but you can also tailor the helm command if you already have grafana installed. This is probably what you want so that you can use your existing install.

    • @scottezinn8555
      @scottezinn8555 2 ปีที่แล้ว +1

      @@TechnoTim So I installed the Rancher Monitoring package (brought in prometheus and customized grafana), installed standalone grafana and added rancher monitoring prometheus (worked), then installed loki-stack with just loki (with persistent storage), Promtail, and prometheus (could probably use the rancher monitoring one but couldn't figure out how to configure Promtail to use it vs the one it links to) enabled and added loki as a datastore to standalone grafana. Worked! So I have alerts now going to Slack, loki collecting and filtering logs, and grafana to view either prometheus or loki logs. Now just need to add in my Unifi data source and teslamate datastore once those containers are moved from docker-based system to k3s. Now back to the day job...

  • @fasooly1992
    @fasooly1992 ปีที่แล้ว

    Is there a video on how to set up the dashboard?

  • @joeripelgrims8900
    @joeripelgrims8900 ปีที่แล้ว

    I seem to be missing something with the syslog portion. I set it up with the listening ports. I then went into one of my network devices and pointed it to loki on port 1514. Now what? I don't see anything in grafana indicating that any syslogs are coming in. I must be missing something really basic.

  • @squalazzo
    @squalazzo 3 ปีที่แล้ว +2

    excellent, man! We are evaluating right now a custom logging solution as the ones offered by major cloud providers are quite expensive... alternatives would have been an ELK stack, or something preconfigured as graylog... what do you think? Thanks

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว +1

      really depends on your infra! This is great for cloud, containers, kuerbernetes, and even traditional syslog. If you are already sing grafana that's a plus too!

  • @smeuse
    @smeuse 3 ปีที่แล้ว +3

    Used graylog for a while, hated it. This looks much nicer, going to give it a whirl. Currently just aggregating into a rsyslog server with some scripts to compress and logrotate.

  • @raymondvanderwerf
    @raymondvanderwerf 3 ปีที่แล้ว

    Pfff ...this is great!! 😀
    Thx man! Will definitely have a play with this! ✌️

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      Hope you enjoy it!

  • @pyhoff
    @pyhoff ปีที่แล้ว

    Stupid question, what do you use for SIEM solution in this setup, LOKI is not one or do they plan on it as per github. What would you forward logs to? Graylog or ELK?

  • @alexandrumaer
    @alexandrumaer 3 ปีที่แล้ว

    Tried Loki, because it's easy to correlate it with Jaeger so you could check out traces based on trace ids found in logs. But to have advanced features on your logs, there's nothing else like elk+kibana. It only depends on what your needs are. Cheers and thumbs up for your videos ;) keep up the good work!

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      Good call! Thank you!

  • @Jose__Manuel
    @Jose__Manuel ปีที่แล้ว

    Hi mister, my name is jose manuel, I hope you can help me to confugure this grafana-loki-promtail to check any file of the system if it is possible, is it possible? I want to mean , we creat a file in home called file with text, simple text and can I see its content with grafana-loki-promtail? With out docker o something like that, can you help me?
    thanks a lot.

  • @Chinna_Chintu2023
    @Chinna_Chintu2023 2 ปีที่แล้ว

    Hi friends,
    Note : logs are from ubuntu server....
    I'm looking for a log retention for 3 months in grafana loki .... How can we extend the retention period from the default 30 days to 90 days ??

  • @tyaprak
    @tyaprak 3 ปีที่แล้ว

    I love grep, awk, wc, uniq and sort commands :)

  • @bret44
    @bret44 3 ปีที่แล้ว +1

    This looks nice but a bit of a confusing nightmare to setup. I couldn't get syslog to work the way you outlined, I decided to use rsyslog to a local target in promtail but for some reason it get's angry if there are two local job names. Also the plugin for loki causes the docker container to hang if you attempt to restart it. Assuming I got all this running, I have no idea how to add sensors from remote servers. There must be a simpler way :/. Thanks for the effort though!

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว +3

      Thank you. This should work. Be sure you don't have scrape_configs: listed twice! Only once! I should probably add a note to the docs. I did this so you could comment one out and uncomment another but if you uncomment both you end up with a bad config.

    • @bret44
      @bret44 3 ปีที่แล้ว

      @@TechnoTim Thanks! I got it working. I had tried only using scrape_configs just once but apparently it was picky about the syntax (I may have not tabbed over correctly or something). Now, please make a guide on how to add load/temps/Ethernet traffic from remote hosts. :)

  • @majorgear1021
    @majorgear1021 ปีที่แล้ว

    I got this working well for standard logs . I'm curious if I can send other logs, like from ansible pull or from application that I write, to promtail as well.
    Google wasn't useful for answering this question, so maybe I'll hit up reddit and Grafana forums.

  • @b9944236
    @b9944236 ปีที่แล้ว

    Great explanation, thanks a lot.

  • @exact-estimate
    @exact-estimate 2 ปีที่แล้ว

    Great video, just set it up. One question though: Retention, what are the limits, and where can you set them to stop it taking up all the space on my host?!

    • @TechnoTim
      @TechnoTim  2 ปีที่แล้ว

      Log retention is configurable

  • @camerontgore
    @camerontgore 3 ปีที่แล้ว

    Great thumbnail! 😁

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว +1

      Haha! Thanks! I figured I would get creative on this one!

  • @reyastaroth
    @reyastaroth ปีที่แล้ว

    Hello, I have LXC containers (15) with dockers running inside, have you found a way to monitor them without installing the agent on all of them?

  • @superworstje
    @superworstje 3 ปีที่แล้ว

    Hi, I wan't to setup promtail to get all syslog from my mikrotik routers. they send it over udp. If I'm right I have to setup a rsyslog forwarder. Can you explain how to do this?

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      I think you can just enable syslog on promtal and the send them there!

  • @yuliamorzhova4516
    @yuliamorzhova4516 3 ปีที่แล้ว

    If I got 500gGB logs in HDD as syslog.logs (txt) , what is better to use: loki + graphana or collect these logs in elasticsearch (graylog) ? As I know elasticsearch not very good with HDD...

    • @TechnoTim
      @TechnoTim  3 ปีที่แล้ว

      I don't have the IOPS per comparison at this point, sorry.

  • @MikeCarr07
    @MikeCarr07 11 หลายเดือนก่อน

    Would you deploy Loki in Kubernetes and use it only for that or would you send logs from other hosts to it as well?

    • @TechnoTim
      @TechnoTim  11 หลายเดือนก่อน +1

      You can do either if you need the logs

    • @MikeCarr07
      @MikeCarr07 11 หลายเดือนก่อน

      @@TechnoTim What are you doing?

  • @mahabubulhasan7827
    @mahabubulhasan7827 ปีที่แล้ว

    Thanks, great explanation indeed

  • @ThangMinh-ul3cs
    @ThangMinh-ul3cs ปีที่แล้ว

    Hi, I am setting up Grafana, Loki, and Promtail for an upcoming project, and I am using syslog-ng to receive logs from external devices. Can you create a video tutorial on using syslog-ng with Grafana Loki?

  • @mahimaverma3131
    @mahimaverma3131 2 ปีที่แล้ว

    @Techno Tim, can we see more than a month old log along with a description(full stack trace) through Grafana Loki. Would this be an efficient tool for maintaining too old logs in application's Eco system.

    • @TechnoTim
      @TechnoTim  2 ปีที่แล้ว +1

      I believe you can. Check out the storage retention options!

  • @deviouschu
    @deviouschu ปีที่แล้ว

    Simply amazing

  • @angelosnm
    @angelosnm 2 ปีที่แล้ว

    Great video Tim as always! How do you add remote docker hosts on the promtail/loki configuration?

    • @TechnoTim
      @TechnoTim  2 ปีที่แล้ว +1

      The same way but then point them to the remote server with promtail to ingest!

  • @frankihk
    @frankihk ปีที่แล้ว

    Hi , may I know how to add the prometheus , snmp exporter also include in the docker?
    My purpose is monitor linux system, windows system , and network deivce e.g Cisco switch , FortiGate firewall . Thank you

  • @wilcochris
    @wilcochris ปีที่แล้ว

    Just come across your video. Great tutorial. One question: what dashboard are you using for this? It's something I would be interested in using. Thanks