Monitoring Proxmox VE With Prometheus And Grafana
ฝัง
- เผยแพร่เมื่อ 29 มิ.ย. 2024
- Monitoring computers is very important and even more so for hypervisors like Proxmox VE
Because if something goes wrong it will likely affect all of the virtual computers that are being run on that physical computer
Now an interesting open source combination of monitoring tools that's available for free is Prometheus and Grafana
So in this video, we show how to configure Prometheus and Grafana to monitor Proxmox VE, when you're using Docker
=============================
SUPPORT THE CHANNEL
Donate through Paypal:
paypal.me/DavidMcKone
Donate through Buy Me A Coffee:
buymeacoffee.com/dmckone
Become a monthly contributor on Patreon:
/ dmckone
Become a monthly contributor on TH-cam:
/ techtutorialsdavidmckone
==============================
=============================
MY RECORDING HARDWARE:
Blue Yeti USB Microphone
amzn.to/3IfL3qm
Blue Radius III Custom Shockmount for Yeti and Yeti Pro USB Microphones
amzn.to/3G3f89P
RØDE PSA1 Professional Studio Arm
amzn.to/3Z3lPBF
Aokeo Professional Microphone Pop Filter
amzn.to/3VuZl9H
Sony Alpha ZV-E10L Mirrorless Camera
amzn.to/3ITHCoU
Elgato Cam Link 4K Capture Card
amzn.to/43CzQaT
Neewer NP-FW50 Dummy Battery Charger Kit
amzn.to/3qp9Q4s
Elgato Key Light Air - Professional 1400 lumens Desk Light
amzn.to/3G81OB9
Neewer 2 Packs Tabletop LED Video Light Kit
amzn.to/3CcuN5O
Elgato Green Screen
amzn.to/3CoJBOL
=============================
==============================
MEDIA LINKS:
Website - www.techtutorials.tv/
Twitter - / dsmckone1
==============================
For more technical information, including commands used, check out our blog post
www.techtutorials.tv/sections...
Useful links:
github.com/prometheus-pve/pro...
grafana.com/grafana/dashboard...
Chapters
00:00 Intro
00:33 Assumptions
00:50 Create Monitoring Account
05:07 Install PVE Exporter
07:07 Configure PVE Exporter
09:37 Configure Prometheus
12:46 Install Grafana Dashboard
14:59 Summary
monitor proxmox with prometheus,monitor proxmox with grafana,monitor proxmox with prometheus and grafana,monitoring proxmox,monitor pve,proxmox prometheus grafana dashboard,proxmox monitoring prometheus,proxmox monitoring grafana,proxmox monitoring api,proxmox server monitoring - วิทยาศาสตร์และเทคโนโลยี
![[MIRRORED] LISA - ROCKSTAR 🎸 MV version #LISA #BLACKPINK #ROCKSTAR](http://i.ytimg.com/vi/-U58DuCCtZk/mqdefault.jpg)
Please be aware there has been a path change for the pve-exporter
When I first set this up, the suggested volume mapping was
/path/to/pve.yml:/etc/pve.yml
That''s still working for me, although my container hasn't been updated recently
After checking the dev's Github webpage, the suggested volume mapping is now
/path/to/pve.yml:/etc/prometheus/pve.yml
NOTE the extra prometheus folder in the container path
Thanks again for your great work! I'm up and running...
I am finding the combination of Prometheus and Grafana very useful
So good to know this video helped
Just what I have been looking for. Can't wait to watch.
Hope you find the video useful
Fairly straightforward to setup
Thank you, great on sharing the rationale of your choices as each environment is different and it may not apply to each individual use case, so you taking the time to explain allows the end user to take an informed decision. Keep it up!
Thanks for the feedback, really appreciated
Great stuff! About to setup Grafana for the first time on my new Proxmox server, and this is going to help!
I like that it can be run in a container.
And having community dashboards makes setting this up a lot easier
Thanks for making and sharing this video. I did some exporting of metrics in a different method a while back. Will have to lookup what I did and compare to you video to see the differences.
Thanks for providing feedback, always appreciated
Great video! Made this easy to setup.
I do like this combination for monitoring
So good to know the video was useful
Thank you! This was very useful!
Good to hear the video was helpful
Great Video. Thank you!
Thanks for the feedback, always appreciated
That's interesting. I didn't realized that we need to use this node exporter. I don't remember others creators mentioned it. I though in the end will be running only 2 container and not 3 (Exporter, Grafana and Prometheus). As always, thank you David!!!!
There are other ways but I think this exporter is very useful
No agent needed on the nodes so less things to go wrong
Thanks David, much appreciate and on my list. This weekend its guacamole docker to nginx proxy manager to cloudflare tunnel with email authentication. 👍
Sounds like quite the weekend
Amazing, Would install in my HomeLab
Definitely a good open source monitoring solution and it's free to use
Interesting, I monitor my proxmox using observium via snmp but this looks like a cool alternative to that setup with some advantages. Thank you
I've tried a lot of monitoring tools over the years but I do like how much easier this is to setup
A backup and restore should be a lot easier too as I don't really need the data, I just need the docker compose and config files to get back up and runnning
So far, the devs for the exporters I've been using provided good examples to at least get started and the community dashboards for grafana saved a lot of time
SNMP can be a pain to setup if you don't already know the mibs but for now that's just going be needed for the physical switches
Thank you!
Good to know the video was helpful
Thanks
Thank you very much
I followed these exact steps, but for some reason my prometheus only shows Prometheus up, it shows neither the nodes nor the exporter
Check the status of the containers to see if they are up and running, for example
docker ps -a
Also check the configs, for example, the IP addressing will probably be different for your network
And check the logs of the containers to see what they're reporting, for example
docker container logs node_exporter
Nice video. Had some headaches around the new file path but got it working. Very nice dashboard with some very good metrics. One item to note: if you are using a drive point for your VM storage you will need give permissions to it as well. Also I had to use the username and password for the connection in the pve yaml file since I had multi Proxmox servers. Couldn't get the different token keys to work.
It's unusual for a path change like that
It would be good if you could edit a video to make a slight change, like add some text, but the only option is to edit your own copy of the original file and upload it as a replacement
I've tried a few monitoring solutions over time though and I do like this Prometheus and Grafana combo
@@TechTutorialsDavidMcKone Agree. It is a good centralized dashboard for monitoring all my Proxmox servers and able to get a quick over view. Love the simple but easy to read metrics to able to look for any potential issues.
@@cyberjohn44 Hi, John.
I've been monitoring my Proxmox using influxDB, because, well, it has build in feature (metrix server). And I use dashboard 18621, but it lacking something, so I copy-paste from 15356, so I get its storage pool and modified it to using gauge.
But, data from metric server is still missing something. 2 things.
1. temperature sensors. and I use telegraf on LXC to get CPU temperature.
2. Proxmox actual free memory, which is memfree - (cache+buff), which crucial in my opinion.
is dashboard 10347 in the video have that (2 things)? is it possible to do that?
@@Anuitu2u Here are the 3 dashboards I am using. Node Exporter Full - 1860 Proxmox via Prometheus - 10347 Docker monitoring with node selection - 8321
Thank you! It was informative and very easy to follow. How can we apply this to a second pve server? Do we need to have a cluster or should we create another node-exporter and pve-exporter in the second pve server?
pve-exporter only needs to run on your docker server. It connects to the nodes over the network through an API
Add each node you want to monitor to the prometheus job that you setup for prometheus to connect to the pve-exporter
Bear in mind, when you have a cluster, pve-exporter will gather the same information from each node
So you'll see the same details in Grafana
But you still want to monitor all nodes in a cluster for redundancy reasons
If you want details at a Linux level, then that's where node-exporter comes in
But it needs to be installed on the host you want to monitor
So if you want to use it to monitor your PVE nodes then install it on each node and configure prometheus to gather the data from each exporter
Excellent video, I used your tutorials on setting up Docker, Heimdall, Prometheus, and Grafana. I was attempt to add the ProxmoxVE monitor but I'm running into an issue with the docker container restarting after ~1minute. Any thoughts or tips on where to look?
Good to know the videos have been helpful
If a container is restarting then check the container's log file, for example
docker container logs prometheus
It should tell you what the problem is and which line to check
But chances are there's a typo in the script, even if it's just a space missing or an extra one added
Or maybe it's pointing to something which doesn't exist e.g. a folder that needs mapping
Excellent video.
I experienced a similar issue. This may help. I ran >docker compose up which enables me to see all the detail of the containers starting. I noticed this error under pve-exporter
FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml'
I change the volume entry for pve-exporter to :
volumes:
- ./pve/pve.yml:/etc/prometheus/pve.yml
This was successful. Good luck!
For anyone that hits the issue with node-explorer container not starting due to error "pve-exporter | FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml'" i did the following to fix it:
-> edit docker-compose.yml and change pve-exporter volumes path from "/etc/pve.ym" to "/etc/prometheus/pve.ym". It should look as this:
volumes:
- './pve/pve.yml:/etc/prometheus/pve.yml'
I take it, you're monitoring PVE with node-exporter?
I ask because you're referencing a folder called /etc/pve which I haven't referred to but would exist on a PVE server
I didn't use that method in this video, instead I set up monitoring through the API to avoid software installation on PVE
Although node-exporter would make sense for additional monitoring
Hi David!@@TechTutorialsDavidMcKone Yes, i followed your guide and got all the containers up yesterday but i got stuck later, at grafana not importing the dashboard profile. I will try to fix it later today. But, first of all: thank you so much for this guide, it helped me a lot, your brought me quite some time with this great guide.
Regarding the error mentioned above, i hit it after configuring the node-explorer container (see instructions around minute 6:45 of your video): the external path is ./pve/pve.yml which your guide said it will map it internally to /etc/pve.yml. But in my case, with your config instructions, the pve-exporter container did not start and was complaining with the error "pve-exporter | FileNotFoundError: [Errno 2] No such file or directory: '/etc/prometheus/pve.yml' (docker compose logs). So, my workaround was to edit docker-compose.yml change the internal path from "/etc/pve.yml" to "/etc/prometheus/pve.yml"; the pve-exporter contaner ran without other issues after this change.
@@bogdandotct4863 That seemed odd as I checked my pve-exporter config and it's
- ./pve/pve.yml:/etc/pve.yml
I've had a look on the Github page and the path is now
/path/to/pve.yml:/etc/prometheus/pve.yml
So the devs must have changed the internal folder at some point, grrr
Thanks for sharing as like others, I'll run into this myself the next time my container is updated
Hi David, thanks for the interesting video. I tried to follow your instructions, but I have issues to start the pve-exporter. I always get the error message: IsADirectoryError: [Errno 21] Is a directory: '/etc/pve.yml' . In which directory is the file pve.yml stored? Where is my error?
Thanks for your support David.
Sounds like the mapping isn't lining up
In the folder where the docker-compose.yml file is and where docker is being run from, I created a sub-folder called pve
In that folder I created a file called pve.yml
The docker compose file maps /etc/pve.yml to that file I created by referencing ./pve/pve.yml
The period or dot is important because it references the current folder and it's a relative path i.e. relative from where docker is running from
So the full line for mapping the config file is
- ./pve/pve.yml:/etc/pve.yml
Do check the syntax, which folder you're in and which folder you're running docker from
Otherwise you could try an absolute path such as /home/docker/pve/pve.yml for instance if you are running docker from a home directory for a user called docker
Hello David,
thank you very much for your help. Now it works.@@TechTutorialsDavidMcKone
I am very surprised that InfluxDB cannot be used ;-(
I did look into exporting metrics from Proxmox VE and I know it can send to InfluxDB and Graphite but I'm more familiar with pull models
What puts me off push models is it requires punching a hole through the firewall to allow access to the monitoring server which I wouldn't been keen on from a security perspective
I think Telegraf would make more sense than Prometheus for InfluxDB, although there is Prometheus exporter for it
github.com/prometheus/influxdb_exporter
@@TechTutorialsDavidMcKone thank you. I have used Home Assistant integration that was pulling data from Proxmox directly to InfluxDB and then to Grafana
This is a REALLY BAD how to video!! REDO IT
Can you be more specific please?
I need at least some points and suggestions
Show or tell file path (/home/username/xyz.abc)
..
link to your doc/blog about the video then
Chapters OR
Chapters than link to blog before asking for money aka support is always better.
@@TechTutorialsDavidMcKone
Very nice tutorial. Despite havin followed along, have been unable to get it working. prometheus exporter is able to scrape data from prox mox (can check the exporter pve target and see scraped metrics), but prometheus does not load the scrape_config for pve, nor do I see any pve endpoint target in prometheus.
I would suggest checking the config for Prometheus as it sounds like it isn't aware of the exporter
It also needs to be reloaded if the config has been changed
Otherwise it would be best to check the log file as any errors would show up there
@@TechTutorialsDavidMcKone Hey Dave, my bad, incorrectly mounted volume for prometheus. Going into running prometheus container and checking prometheus.yml showed me it was using the default, not my custom one from host. All good, thanks again!
@@TechTutorialsDavidMcKone oh, and subbed!