Following the previous JSON API video, we created this detailed tutorial making it possible to work with JSON API open data sets for everyone. Check out the blog post for more details: volkovlabs.io/blog/json-api-data-sources-in-grafana-20230104/ Now we are working on the business dashboard tutorial. Leave a comment below for what to include.
@@vinodmahant5486 Do you have any specific options on your mind? Are you using or planning to use JSON API or Infinity Data source? Infinity supports many authentication options out of the box.
@@volkovlabs Thanks for quick response. I have created a dashboard with JSON API datasource, Now I want to secure my rest api, It works with basic auth but I want to use token based authentication for my use case like Oauth2 or header based authentication (where the auth token periodically changes) (JSON API options: Forward Oauth identity, Custom Http Headers) Please suggest ?
@@vinodmahant5486 JSON API configuration displays native Grafana Authentication options, including "Forward OAuth Identity", but we have not tested it in our projects. JSON API is maintained by Grafana, and I would recommend asking on Community Forum (community.grafana.com/new) if you have any issues.
Hi, Can you please make a video on using json api to fetch json data from an url using Path in Json Api. Please add a global variable in the path too which we can select from a drop down.
Thank you very much for the video. Can you please help me to understand more about variables. Like how can we pass multiple value variables to JSON API params.
Which plugin are you talking about? Both of them has excellent documentation explaining variables: grafana.com/docs/plugins/yesoreyeram-infinity-datasource/latest/
Hello guys! Thanks for the video. Did you manage to install Infinity datasource plugin in Grafana 11 (I use 11.1.4)? I am in an air-gapped environment, so i download zip file from plugins website, unzip it and copy the folder to /var/lib/grafana/plugins/ inside the container and then I restart the container. The "grafana cli plugins ls" command shows that "yesoreyeram-infinity-datasource @ 2.9.5" plugin is installed. However, I cannot see the corresponding datasource in the "Add new connection" section of the Grafana menu. I did the same on Grafana 10.2.1 instance and everything worked as I expected.
It's a great question to ask on the Community Forum: community.grafana.com/new. Infinity is popular data source and Community can guide you if logs are not helpful. I would recommend to look at the Grafana's startup logs to check that plugin was loaded or not. You may see the reason which prevented data source to load. It may be permission issues.
@@volkovlabs Thanks for the link. I actually managed to install it from zip file using grafana cli. No idea why it didn't work after copying the unzipped folder
Does the JSON API and Infinity Data Source fetch the data continuously from the rest end point(URL) provided. If not how to fetch data in regular interval of time continuously?
It's called Streaming in Grafana terms and I have not heard about it in both plugins. Looking at Infinity there is no mention for Streaming in Docs and Issues. You can use Dashboard refresh to refresh on interval. Other option is to use the WebSocket data source, which support streaming.
Can you please help me on how to retrieve two specific fields from the JSON payload of 15 health check URLs and display the data in a single table. please note that each of the 15 health check URLs contains the same two specific fields in their JSON payloads. Also if possible please let me know how to add variables to dashboard for querying the data
You can add 15 queries to the panel and then use Merge transformation to combine them together. JSON API and Infinity data sources support dashboard variables, check out the documentation on how to use them.
@volkovlabs how to add the startyear and endyear dynamically, For example, whatever I select from the time picker it should consider that range? is there any way?
@@praveennaik63 I just guessed the answer, because the questions is still not clear. What Data Source do you use and how do you specify start and end year? -- Mikhail
Nice explanations and visualizations! One thing I noticed is that both panels switch alternatively between ascending and descending curves (the max and the min change places). I guess it has something to do with the .reverse() method that was called on the data. But why doesn't it stay fixed?
Good catch! I don't remember the details, it was something wrong in the script. Let me know if you able to reproduce it using the latest version of the plugins.
@@volkovlabs I am using a specific release of Grafana (v9.2.1) because of the project I am working in. Perhaps I can check that later in a latest release.
Have you tried the Elastic Search Data Source? grafana.com/docs/grafana/latest/datasources/elasticsearch/. We recommend to use specific data source is available instead of using custom solution using API Data Sources. -- Mikhail
@@volkovlabs Thanks for the reply, actually I need to perform generate some dynamic fields based on the existing fields in my response data. For this reason I cannot go ahead with the existing ES data source. Anyways I am able to able to achieve the requirement with Infinity DS but is there any such limitation of why it cannot process more than 10000 records ?
@@LakshayChopra-h6v I am not aware of this limitation. If it's not mentioned in the documentation, please submit an issue in the project repository: github.com/grafana/grafana-infinity-datasource.
How do I create a dashboard using these plugins on grafana installed in the Kubernetes cluster. I want to automate so that every time a new pod is created, the dashboard should be already configured.
thank you for detailed explaination. I have one question , I am using garfana for visualization i want to do a task in which data is filtered by using garfana tool. and send to the algorithm written in python which finds the clusters and send back processed data to garfana (store in database). is there any way to do such task? thanks. looking forward to hear from youl
i have a API which has historical data and i can get history/5d history/1h history/1w for example. Is it possible to use the grafana timefilter and dynamically request the correct history endpoint? (i get 20 data points out)
Grafana's Time Range To and From values can be accesses using Global Variables `${__from}` and `${__to}`. You can use them in the URL or Payload to request the data.
Thank you for the video, I am using grafana to display Wazuh REST api data into tables or charts. By using both plugins I manage to connect and pull data from api. But all data (json format) is coming into one line in grafana. Is there any way to put data into table format or other charts?
Both plugins support parsing JSON data into separate columns and rows. Check the documentation for Infinity how to parse JSON: sriramajeyam.com/grafana-infinity-datasource/docs/json
Thank you very much for your video, one quick question I am using json api to fetch the incident count from Pagerduty to display it in Grafana dashboard everything is working fine but i am not able to give default value for last 24hours data. As of now i am using "since" in param and giving last day date. Tried adding now() also but its fetching last month data. Any suggestion please 🥺
Good question. I am not familiar with Pagerduty API and can't comment on the parameters. I would recommend to ask in Community Forum: community.grafana.com -- Mikhail
Hello, I am a starter grafana developer, After install Infinity or JSON API i can't use and there are message "Alerting not supported" Please help or suggestion me many thanks.
Alerting not supported means that the JSON API is a frontend only plugin and you can't create Alerts based on it. Infinity has a backend and it can be used for Alerts. What do you mean you can't use?
Are you able to assist me in parsing XML data using Infinity as the datasource. I can GET the raw data using the URL in Grafana using Infinity, but I have not been successful at parsing any of it.
Infinity Data source is well documented in the Documentation: grafana.com/docs/plugins/yesoreyeram-infinity-datasource/latest/. Also, Grafana Forum is a great place to get advice for the Infinity Data Source: community.grafana.com
Are you connecting using Frontend (Browser) or Backend (Grafana Server) mode? Check the Browser -> Console -> Network for the specific error to troubleshoot further. -- Mikhail
@@volkovlabs thank you for your help, ther ewas a problem with json api with 8.2.0 when running in the docker, pulled latest image of graphana and everything worked like magic. Awesome work, used couple graphs from echart
Following the previous JSON API video, we created this detailed tutorial making it possible to work with JSON API open data sets for everyone. Check out the blog post for more details: volkovlabs.io/blog/json-api-data-sources-in-grafana-20230104/
Now we are working on the business dashboard tutorial. Leave a comment below for what to include.
Can you please create a video on authentication options for JSON API other than basic auth
@@vinodmahant5486 Do you have any specific options on your mind? Are you using or planning to use JSON API or Infinity Data source? Infinity supports many authentication options out of the box.
@@volkovlabs Thanks for quick response. I have created a dashboard with JSON API datasource, Now I want to secure my rest api, It works with basic auth but I want to use token based authentication for my use case like Oauth2 or header based authentication (where the auth token periodically changes) (JSON API options: Forward Oauth identity, Custom Http Headers)
Please suggest ?
@@vinodmahant5486 JSON API configuration displays native Grafana Authentication options, including "Forward OAuth Identity", but we have not tested it in our projects. JSON API is maintained by Grafana, and I would recommend asking on Community Forum (community.grafana.com/new) if you have any issues.
Hallelujah!!!
I can finally do something with Grafana...
thank you my God (and Mrs. Volcov!!!)
You are very welcome. Happy for you!
You guys put together such nice and detailed tutorials without all kinds of fluff - I love it!
We don’t do fluff, straight to the point to answer questions and solve problems. Thank you for follow along.
Awesome video! Thank you very much! Keep going more about building start time and end time query parameters for API call!
Glad it helped!
Hi, Can you please make a video on using json api to fetch json data from an url using Path in Json Api. Please add a global variable in the path too which we can select from a drop down.
Very timely request, I am about to make a video like that. Stay tuned!
@@volkovlabs great ! Please make it soon. It’s quite urgent requirement for my project.
@@saloniipawar6631 Take a look at the blog post which covers those topics: volkovlabs.io/blog/form-panel-server-based-elements-20240428/
Hey thanks of info!
So After doing post request and getting the output, Can I trigger alerts based on the incoming data.
Infinity is a backend data source and supports Alerting.
Great video! Is there someone I can copy the final function from? for the smooth line panel?
You can find smooth line in the Examples: echarts.volkovlabs.io/d/tg6gWiKVk/line?orgId=1
context.panel.data.series.map((s) => {
if (s.refId === "A") {
yearA = s.fields.find((f) => f.name === "year").values.buffer;
monthA = s.fields.find((f) => f.name === "periodName").values.buffer;
valueA = s.fields.find((f) => f.name === "value").values.buffer;
}
});
const month_yearA = monthA.map((d, i) => `${d} ${yearA[i]}`);
return {
grid: {
bottom: "3%",
containLabel: true,
left: "3%",
right: "4%",
top: "4%"
},
series: [
{
data: valueA.reverse(),
smooth: true,
type: "line"
}
],
xAxis: {
data: month_yearA.reverse(),
type: "category"
},
yAxis: {
type: "value"
}
};
Very clear and concise, great video
Appreciate the feedback.
Thank you very much for the video. Can you please help me to understand more about variables. Like how can we pass multiple value variables to JSON API params.
Which plugin are you talking about? Both of them has excellent documentation explaining variables: grafana.com/docs/plugins/yesoreyeram-infinity-datasource/latest/
@@volkovlabs JSON api
Hello guys! Thanks for the video. Did you manage to install Infinity datasource plugin in Grafana 11 (I use 11.1.4)? I am in an air-gapped environment, so i download zip file from plugins website, unzip it and copy the folder to /var/lib/grafana/plugins/ inside the container and then I restart the container. The "grafana cli plugins ls" command shows that "yesoreyeram-infinity-datasource @ 2.9.5" plugin is installed. However, I cannot see the corresponding datasource in the "Add new connection" section of the Grafana menu. I did the same on Grafana 10.2.1 instance and everything worked as I expected.
It's a great question to ask on the Community Forum: community.grafana.com/new. Infinity is popular data source and Community can guide you if logs are not helpful.
I would recommend to look at the Grafana's startup logs to check that plugin was loaded or not. You may see the reason which prevented data source to load. It may be permission issues.
@@volkovlabs Thanks for the link. I actually managed to install it from zip file using grafana cli. No idea why it didn't work after copying the unzipped folder
@irkost Glad to hear it worked! As I mentioned, can be permissions.
Does the JSON API and Infinity Data Source fetch the data continuously from the rest end point(URL) provided. If not how to fetch data in regular interval of time continuously?
It's called Streaming in Grafana terms and I have not heard about it in both plugins. Looking at Infinity there is no mention for Streaming in Docs and Issues. You can use Dashboard refresh to refresh on interval. Other option is to use the WebSocket data source, which support streaming.
Can you please help me on how to retrieve two specific fields from the JSON payload of 15 health check URLs and display the data in a single table.
please note that each of the 15 health check URLs contains the same two specific fields in their JSON payloads.
Also if possible please let me know how to add variables to dashboard for querying the data
You can add 15 queries to the panel and then use Merge transformation to combine them together.
JSON API and Infinity data sources support dashboard variables, check out the documentation on how to use them.
thank you very much for this clear and short explanation 👏
You are welcome! Good luck with your project.
--
Daria
@volkovlabs how to add the startyear and endyear dynamically, For example, whatever I select from the time picker it should consider that range? is there any way?
Time Range' To and From are set in Global Variables which you can use in the Data Sources: volkovlabs.io/grafana/variables/#global-variables
@@volkovlabsHow to use these variables in my API request? The documents you shared me, Didnt help
@@praveennaik63 I just guessed the answer, because the questions is still not clear. What Data Source do you use and how do you specify start and end year?
--
Mikhail
Nice explanations and visualizations! One thing I noticed is that both panels switch alternatively between ascending and descending curves (the max and the min change places). I guess it has something to do with the .reverse() method that was called on the data. But why doesn't it stay fixed?
Good catch! I don't remember the details, it was something wrong in the script. Let me know if you able to reproduce it using the latest version of the plugins.
@@volkovlabs I am using a specific release of Grafana (v9.2.1) because of the project I am working in. Perhaps I can check that later in a latest release.
Hey, How can i send dynamic variables as a parameter to the Infinity data source API?
Please check out the plugin documentation how to use variables: grafana.com/docs/plugins/yesoreyeram-infinity-datasource/latest/
can you please let me know how to connect with Elasticsearch endpoint using infinity plugin. Also how to query the _source result as a table view ?
Have you tried the Elastic Search Data Source? grafana.com/docs/grafana/latest/datasources/elasticsearch/. We recommend to use specific data source is available instead of using custom solution using API Data Sources.
--
Mikhail
@@volkovlabs Thanks for the reply, actually I need to perform generate some dynamic fields based on the existing fields in my response data. For this reason I cannot go ahead with the existing ES data source. Anyways I am able to able to achieve the requirement with Infinity DS but is there any such limitation of why it cannot process more than 10000 records ?
@@LakshayChopra-h6v I am not aware of this limitation. If it's not mentioned in the documentation, please submit an issue in the project repository: github.com/grafana/grafana-infinity-datasource.
Clear and good explanation.. Great work 👍
Appreciate the feedback. What Data Source you decided to use?
@@volkovlabs I'm using influx DB as data source for my grafana dashboard
How do I create a dashboard using these plugins on grafana installed in the Kubernetes cluster. I want to automate so that every time a new pod is created, the dashboard should be already configured.
Use Provisioning: th-cam.com/video/sGHKY0VmbLw/w-d-xo.html
thank you for detailed explaination. I have one question , I am using garfana for visualization i want to do a task in which data is filtered by using garfana tool. and send to the algorithm written in python which finds the clusters and send back processed data to garfana (store in database). is there any way to do such task? thanks. looking forward to hear from youl
Check out the Data Manipulation panel
i have a API which has historical data and i can get history/5d history/1h history/1w for example. Is it possible to use the grafana timefilter and dynamically request the correct history endpoint? (i get 20 data points out)
Grafana's Time Range To and From values can be accesses using Global Variables `${__from}` and `${__to}`. You can use them in the URL or Payload to request the data.
Great to hear that! Could you please create video for that?
Thank you for the video, I am using grafana to display Wazuh REST api data into tables or charts. By using both plugins I manage to connect and pull data from api. But all data (json format) is coming into one line in grafana. Is there any way to put data into table format or other charts?
Both plugins support parsing JSON data into separate columns and rows. Check the documentation for Infinity how to parse JSON: sriramajeyam.com/grafana-infinity-datasource/docs/json
@@volkovlabs thank you for the reply and documentation
@@MrGh0sT_8124 Happy Dashboarding!
Thank you very much for your video, one quick question I am using json api to fetch the incident count from Pagerduty to display it in Grafana dashboard everything is working fine but i am not able to give default value for last 24hours data. As of now i am using "since" in param and giving last day date. Tried adding now() also but its fetching last month data. Any suggestion please 🥺
Good question. I am not familiar with Pagerduty API and can't comment on the parameters. I would recommend to ask in Community Forum: community.grafana.com
--
Mikhail
Hello, I am a starter grafana developer, After install Infinity or JSON API i can't use and there are message "Alerting not supported" Please help or suggestion me many thanks.
Alerting not supported means that the JSON API is a frontend only plugin and you can't create Alerts based on it. Infinity has a backend and it can be used for Alerts. What do you mean you can't use?
Are you able to assist me in parsing XML data using Infinity as the datasource. I can GET the raw data using the URL in Grafana using Infinity, but I have not been successful at parsing any of it.
Infinity Data source is well documented in the Documentation: grafana.com/docs/plugins/yesoreyeram-infinity-datasource/latest/.
Also, Grafana Forum is a great place to get advice for the Infinity Data Source: community.grafana.com
Hello, what do I do if infinity is not available to download?
What do you mean is not available? Using direct connection to Internet? You can download archive manually and extract it in the plugin folder.
Thank you. Helpful video.
Glad if helped! Do you use JSON API or Infinity?
Sorry for the double msg (parkinson symptoms)
would it matter if my grafna runs locally? doesn't work for me for some reason
Are you connecting using Frontend (Browser) or Backend (Grafana Server) mode? Check the Browser -> Console -> Network for the specific error to troubleshoot further.
--
Mikhail
@@volkovlabs thank you for your help, ther ewas a problem with json api with 8.2.0 when running in the docker, pulled latest image of graphana and everything worked like magic. Awesome work, used couple graphs from echart
Glad to hear!
Hallelujah!!!
I can finally do something with Grafana...
thank you my God (and Mrs. Volcov!!!)
What is your use case? What do you want to accomplish with Grafana?