![Prometheus Monitoring with Julius | PromLabs](/img/default-banner.jpg)
- 13
- 285 715
Prometheus Monitoring with Julius | PromLabs
Germany
เข้าร่วมเมื่อ 2 พ.ค. 2020
Prometheus co-founder Julius Volz shares Prometheus monitoring fundamentals, tutorials, tips and best practices, and more via his company PromLabs. See also promlabs.com/ and training.promlabs.com/.
Understanding Prometheus Histograms | Motivation and Concepts, Instrumentation, Querying in PromQL
In this video, I explain Prometheus histograms (for now only the "classic" ones that have been in Prometheus for around a decade - I will make a separate video about the new "native" histograms once they are stable): What are histograms, why are they useful, how can you instrument your service code with histograms, how are histograms exposed as metrics to Prometheus, and how can we query them in PromQL to get quantiles/percentiles, heatmaps, request rates, or average request durations?
Link to the Prometheus histograms best practices:
prometheus.io/docs/practices/histograms/
Also check out my other Prometheus training courses if you want to learn Prometheus in a structured way from the ground up:
training.promlabs.com/
Chapters:
00:00 Introduction
00:56 Motivation and histogram basics
01:22 Need to measure request durations / latency
01:37 Downsides of using event logging
01:56 Why a single gauge doesn't help us
02:28 Downsides of using Prometheus summary metrics
03:09 Prometheus histogram example for tracking request durations
04:32 How can we expose histograms as time series to Prometheus?
05:11 Cumulative histogram representation
05:40 The special "le" (less-than-or-equal) bucket upper bound label
06:12 Time series exposed from a histogram metric
07:30 Instrumentation - adding histograms to your code
07:44 Adding histograms without additional labels
09:07 Adding histograms with additional labels
10:06 Querying histograms with PromQL
10:44 Querying all bucket series of a histogram
11:23 Querying percentiles / quantiles using histogram_quantile()
13:32 Using rate() or increase() to limit a histogram to recent increases
14:42 Controlling the smoothing time window
15:05 Aggregating histograms and percentiles over label dimensions
17:58 Errors of quantile calculation and bucketing schemas
19:27 Showing histograms as a heatmap
20:36 Querying request rates using _count
20:57 Querying average request durations using _sum and _count
21:28 Outro & PromLabs Trainings
---------------------------------------------------------------------------
CREDITS: "Subscribe Button" by MrNumber112 th-cam.com/video/Fps5vWgKdl0/w-d-xo.html
Link to the Prometheus histograms best practices:
prometheus.io/docs/practices/histograms/
Also check out my other Prometheus training courses if you want to learn Prometheus in a structured way from the ground up:
training.promlabs.com/
Chapters:
00:00 Introduction
00:56 Motivation and histogram basics
01:22 Need to measure request durations / latency
01:37 Downsides of using event logging
01:56 Why a single gauge doesn't help us
02:28 Downsides of using Prometheus summary metrics
03:09 Prometheus histogram example for tracking request durations
04:32 How can we expose histograms as time series to Prometheus?
05:11 Cumulative histogram representation
05:40 The special "le" (less-than-or-equal) bucket upper bound label
06:12 Time series exposed from a histogram metric
07:30 Instrumentation - adding histograms to your code
07:44 Adding histograms without additional labels
09:07 Adding histograms with additional labels
10:06 Querying histograms with PromQL
10:44 Querying all bucket series of a histogram
11:23 Querying percentiles / quantiles using histogram_quantile()
13:32 Using rate() or increase() to limit a histogram to recent increases
14:42 Controlling the smoothing time window
15:05 Aggregating histograms and percentiles over label dimensions
17:58 Errors of quantile calculation and bucketing schemas
19:27 Showing histograms as a heatmap
20:36 Querying request rates using _count
20:57 Querying average request durations using _sum and _count
21:28 Outro & PromLabs Trainings
---------------------------------------------------------------------------
CREDITS: "Subscribe Button" by MrNumber112 th-cam.com/video/Fps5vWgKdl0/w-d-xo.html
มุมมอง: 3 188
วีดีโอ
Relabeling in Prometheus | Relabeling Architecture and Flow, Configuration, Examples, Debugging
มุมมอง 7K11 หลายเดือนก่อน
In this video, I explain the concept of relabeling in Prometheus, a powerful tool for filtering and transforming your flows of labeled objects in the Prometheus server. I go into the motivation for relabeling, explain how relabeling fits into the main Prometheus configuration file, and then show how the relabeling steps and flow works. I also discuss the structure of a single relabeling rule, g...
Understanding Counter Rates and Increases in PromQL | Reset Handling, Extrapolation, Edge Cases
มุมมอง 14Kปีที่แล้ว
In this video, I explain the exact value calculation behaviors of the rate(), irate(), and increase() functions in PromQL for computing rates of increase for counter metrics, show how they are different, how they deal with counter resets, how and when they extrapolate values, and which one you should use. Check out my Prometheus training courses if you want to learn Prometheus in a structured w...
Exposing Custom Host Metrics Using the Prometheus Node Exporter | "textfile" Collector Module
มุมมอง 8Kปีที่แล้ว
In this video, I explain how to use the Node Exporter's "textfile" collector module to expose custom host metrics. The textfile collector example scripts repository mentioned in the video: github.com/prometheus-community/node-exporter-textfile-collector-scripts Check out my Prometheus training courses if you want to learn Prometheus in a structured way from the ground up: training.promlabs.com/...
7 Things You Didn't Know About Prometheus | Little-Known Features and Implementation Details
มุมมอง 1.8Kปีที่แล้ว
In this video, I cover seven little-known features, behaviors, and implementation details in the Prometheus monitoring system that I find mildly interesting. How many of these did you know about? Check out my Prometheus training courses if you want to learn Prometheus in a structured way from the ground up: training.promlabs.com/ Chapters: 00:00 Introduction 00:32 Fact 1: Scrape Offsets 02:07 F...
PromQL Data Selection Explained | Selectors, Lookback Delta, Offsets, and Absolute "@" Timestamps
มุมมอง 12Kปีที่แล้ว
In this video, I explain how Prometheus / PromQL data selection works in detail. I go into instant and range vector selectors, label matchers, the 5-minute lookback delta for instant vector selectors, staleness markers and staleness handling, and then explain relative offsets and absolute "@" timestamps for selectors. Check out my Prometheus training courses if you want to learn Prometheus in a...
Don't Make These 6 Prometheus Monitoring Mistakes | Prometheus Best Practices & Pitfalls
มุมมอง 13Kปีที่แล้ว
In this video, I go through some of the biggest pitfalls you can run into when you're new to Prometheus, and I also explain how to avoid them. Check out my Prometheus training courses if you want to learn Prometheus in a structured way from the ground up: training.promlabs.com/ Related PromLabs blog post about these pitfalls: promlabs.com/blog/2022/12/11/avoid-these-6-mistakes-when-getting-star...
Monitoring Linux Host Metrics with Prometheus | Node Exporter (Setup, Scrape, Query, Grafana)
มุมมอง 12Kปีที่แล้ว
In this video, I show you how you can use the Prometheus Node Exporter to monitor Linux (or Unix) host metrics. We first download and run the Node Exporter and then we monitor it with Prometheus. Then, we explore some of the available metrics in Prometheus and show how to import an exhaustive Node Exporter dashboard into Grafana. Check my Prometheus training courses if you want to learn Prometh...
Understanding "up" and Friends in Prometheus | Synthetic (Auto-Generated) Scrape Metrics
มุมมอง 8Kปีที่แล้ว
In this video, I explain the various synthetic (or auto-generated) metrics that Prometheus records for every target scrape: up, scrape_duration_seconds, scrape_samples_scraped, scrape_samples_post_metric_relabeling, and scrape_series_added. Where do they come from, what do they mean, and what can you do with them? Check my Prometheus training courses if you want to learn Prometheus in a structu...
Understanding Prometheus Metric Types | Meaning and Usage (Gauge, Counter, Summary, Histogram)
มุมมอง 34Kปีที่แล้ว
This video explains the four different metric types in Prometheus: Gauges, Counters, Summaries, and Histograms. It goes into the meaning of each type, explains how to use each type in instrumentation, and how they show up in the exposition format. It also touches on what you have to watch out for when querying each metric type with PromQL. Check my Prometheus training courses if you want to lea...
Creating Grafana Dashboards for Prometheus | Grafana Setup & Simple Dashboard (Chart, Gauge, Table)
มุมมอง 71Kปีที่แล้ว
This video shows you how to use Grafana to build dashboards for Prometheus. The video explains how to download and run Grafana, how to create a Prometheus data source in Grafana, as well as how to create a dashboard with three basic panel types (time series chart, gauge, and table). Check my Prometheus training courses if you want to learn Prometheus in a structured way from the ground up: trai...
Getting Started with Prometheus | Minimal Setup (Download, Config & Run)
มุมมอง 44Kปีที่แล้ว
This video shows you how to download, configure, and run Prometheus in a minimal way, only scraping itself and some demo targets. Check my Prometheus training courses if you want to learn Prometheus in a structured way from the ground up: training.promlabs.com/ Chapters: 00:00 Introduction 00:44 Downloading Prometheus 01:16 Unpacking and Inspecting the Tarball 02:03 Configuring Prometheus 04:22...
Introduction to the Prometheus Monitoring System | Key Concepts and Features
มุมมอง 57Kปีที่แล้ว
Get a quick high-level overview of the key concepts of the Prometheus monitoring system straight from the co-founder of Prometheus. This video explains what Prometheus is, what the system architecture looks like, and what the main features and concepts are that make Prometheus-based monitoring so powerful: the dimensional data model, the text-based metrics transfer format, the PromQL query lang...
Hi bro, how to backup and restore Prometheus Data ?
You can use the snapshot API (prometheus.io/docs/prometheus/latest/querying/api/#snapshot) to create a snapshot of the entire TSDB that can be copied / backed-up to somewhere else. Just drop it back in place as Prometheus' data directory when you want to restore it. Triggering a snapshot via the API requires the admin API to be enabled via the "--web.enable-admin-api" command line flag.
Simple and concise. Great video once again.
Great video. I wanted to setup prometheus with my jellyfi setup. This video helped.
Hi, just want to say that i chanced upon using prometheus while trying to conduct a load test. This is really interesting tool! I deployed a postgres exporter to expose certain metrics from my postgres instance. Encountered some issues as there wasnt too many tutorials about how to do this in azure managed prometheus. But nonetheless managed to figure it out. I watched most of your video in one ago just afew hours ago, and these are really useful knowledge, including those tips in grafana. thank you!
I'm watching your tutorial at 3:50 AM and having my head racked for 2 weeks thanks 🧍
I just finished watching the video series and would like to congratulate you. They are awesome. Especially the animations to explain how prometheus works. Really brilliant. Thank you very much for your effort.
Hi! This video is an excellent example of how to share knowledge in the best possible way. By the way, are you a native German speaker? You sound so close to a colleague of mine from Frankfurt. I love your video and appreciate that you make such impressive recordings. Cheers!
Thanks for the kind words! Haha yes, I'm German :)
Great stuff
Wow, thank you for this excellent content and your work on this project. I lwatched the entire playlist and was able to get a good overview. 😀🎉
I checked your excellent github profile and now I see why you should take lessons from the Creator. Despite my strong understanding of golang, do you have any recommendations for learning prometheus-specific golang?
Histogram from the horse's mouth ❤, understood and luved it ! Thank you! Any code examples of enabling this for Python and Java?
Thanks! You can find Python examples at prometheus.github.io/client_python/instrumenting/histogram/ and Java at prometheus.github.io/client_java/getting-started/metric-types/#histogram
The gauge one helped me a lot :D thank you.
Very informative videos! Kudos! I'm looking for easy to understand and implement tutorials/resources about "prometheus at scale for k8s clusters in production" either using thanos/cortex or prometheus federation. It would be great if you can make few videos on this topic. Because I think if someone can make this topic easy to understand, it has to be none other than you.
Great playlist, however it would be helpful if you can fix the order of the playlist from getting started to more advance topic like histogram and counter rates, etc
Thanks for the suggestion, I reordered the videos a bit. They're not really meant to be building on each other in any particular order or curriculum, so not sure if the new order is that much better :)
amazing video!!! please continue!!!
Very happy that i came across your channel. I am quite new to Prometheus and your content has already helped me out a bunch. Thanks Julius
I have some python service files , in that I need to identify memory leakages.For this ,prometheus may helps me?? To monitor how memory is increasing...sudden increase in memory may leads to memory leakage.
Hi, for such specific usage scenarios and questions, the public community channels like the users mailing list (see prometheus.io/community/) are usually a great place to ask!
Julius. Your videos are great. But can you please slow down just a bit!! 🙏🏾
I don't want to bore anyone either though 😬
Amazing
I am really hyped to find your channel, been working with prometheus for a few years now, it was my primary reason to begin learning golang also instantly subscribed, I'll be lurking here on my weekends! ♥
Damn, got already lost at 1:20......sue downloading the file yes so far so good and then suddenly you change to Terminal comand lines....How the heck do you conect the downloaded file which is on the PC at that moment to the make it be able to get unpacked from within Terminal????? or is this only possible with Macs??
I'm on Linux (that's why I'm downloading the Linux binary), so I just switched from my browser to my terminal, both locally on the same computer. That'll look similar on macOS and probably also on Windows, although Windows is not a Unix environment (so the available terminal commands will differ unless you use something like WSL, Windows Subsystem for Linux). Generally I assume for my videos that people have a decent level of IT/admin experience already, so I only focus on the Prometheus-specific questions.
How to add DB connections dashboard?
❤
Thanks a lot. It's really helpful.
unable to open web-interface, have tried my server adress, localhost on 9090, checked my firewall, nothing seems helping
Hi Sir, I have a question. Does this tools suitable for Maintenance Planning use? If yes, would you please explain a bit on how this Prometheus helping with the planning for maintenance work? Many thanks.
Maintenance planning sounds like a mostly unrelated task, so not really what Prometheus was designed for (systems monitoring). However, you can temporarily silence alerts in the Alertmanager when there are known maintenance windows going on, so you don't send notifications in that case.
@@PromLabs another question, does Prometheus able to integrate with Pronto. I can't find the support article on this. So far, only found Prometheus integrated with SAP, MAXIMO and Oracle. Would you please advise. Cheers.
@@HaziHaznidaUsolludin Sorry, I've never heard of Pronto before. If you have specific usage and integration questions like this one, I would recommend using one of the open community channels like the Prometheus Users mailing list (see prometheus.io/community/).
A good teacher can make you fall in love with anything. Thank you for making this video ❤
Currently learning prometheus, this helps, thank you very much!
it's really helpful for me.👏
Thank you for the video! :) Very good content!
Great video! How do I display histograms in Grafana?
Good question, I'll make a quick follow-up video about using heatmaps in Grafana soon!
very very helpful, ty very much Sir. I will take a time in your channel to watch more staff like this :)
hello Julius, Thanks for this great explaining. I actually have a problem when scrape any port except the main port for prometheus (9090), for example: port 9100 for node-exporter or any other port and it always has down state. could you please tell me how could i fix this problem
Not sure what the problem is here if you checked that the network connectivity from Prometheus to the other process works and that the port is specific in the target. But generally it's a good idea to ask with more details on the users mailing list (see prometheus.io/community/).
Hey Julius, quick question, I'm currently pursuing the understanding-promql training in promlabs, but I'm especially curious about PromLens as an interface. I can't find any tutorial, just a live demo and some querying guidance from the understanding-promql training. Its there any material, or any sort of walkthrough the interface more than the landing page? I would like to understand how to take the most of promlens. Thank you!
Saw your email and replied there :)
Hi, may I ask for some help? I am using a kube-prometheus-stack. Federating 2 remote prometheus alerts into a central prometheus. I use externalLabels and they appear just fine. Problem is, the metrics/ alerts do not have on central prometheus the set externaLabels. How to fix that?
How do I select data by month?
So helpful! Thanks so much!
thanks for the video but the subscriber button has a very different kind of inc() method it substracts one if you are already subscribed :)
Haha yes, this is very dangerous 🤣
hey julius... your way of explaining is super simple and on-point. the way you speak steadily and clearly..it made the concepts so easy to understand. keep bringing more videos on grafana and prometheus.
How to enable authentication in prometheus
There's several ways, but take a look at prometheus.io/docs/guides/basic-auth/ and prometheus.io/docs/prometheus/latest/configuration/https/ for a start.
thank you , great video`
very very effective, thanks a lots...
Julius, as a long-time explainer and trainer on OSS, you're doing some seriously great work here, simply explaining things so others can have a good experience and follow along until they can fly themselves, thank you!
Thanks for the video! Very informative
did you use any AI software for the eye contact, because you didn't even blink in 44 seconds?
Haha, no! I was reading off a teleprompter at fixed speed. This should be much better in the more recent videos, where I used a remote control to be able to pause the script, breathe, and blink :)
Do you have any video related to building Prometheus using source code, if you have please share it
I don't have any video on that yet, as I'm focusing more on basic Prometheus usage rather than development / building from source.
I have a datetime value I want to track with metrics and prometheus. What would be the ideal metric type for that job?
A common pattern in Prometheus is to store Unix timestamps in gauge metrics. Then you can use the time() function (e.g. "time() - my_timestamp_seconds") to calculate how old the timestamp is. Example: demo.promlens.com/?l=e2HGvJ2PJgl
Hi, just now started using prometheus. I am trying to set rate function for counter.. so i have a counter metric and the values is 1.. when i use rate for counter rate of first entry a at that time starts with 0..why for the rate of first new single instrumwntation entry starts with 0. How to resolve that..i want rate /changes/ increases should starts with non zero and then it should become 0 when its idle how to achieve this
Thanks for writing and open-sourcing such great software, Julius.
turly amazing video! I have taken at least 3 courses on prometheus+grafana but non of them are properly detailed like your videos