Thanks for good simple test example of elasticsearch in containers. ElasticSearch server started and accessed by browser without ssl cert. Logstash didn't accept log file even moved to logstash.
I run into something similar, but in my case I tried the latest version which is 8.4.0. When I reverted to 7.16.2 as in the video tutorial... taaa daaa logstash can read the log file.
Great vídeo dude, if I want to send logs with other server to elk I only need to install filebeat and point to logstash and elastic for this docker right?
Thank you. I have tested in an Ubuntu VM and it works fine. I would like to collect Netflow v5 data through port 2055 for example. May I ask you for orientation of how to modify your docker compose file to add elastiflow?
Good tutorial! Have you tried to upgrade the version of the components to the latest and run it? For me, when I tried the 8.4.0 it will all started up, but no index gets created almost as if the logstash chooses not to read from the log file (it detects the log though)
my container of logstash keeps restarting... after i started docker-compose up command also when i enter the 5601 from the browser logstash says that it's missing security requirements...
I have updated the logstash.conf file but that file was not found logstash container. I have .log & simple-logstashfile.conf along with logstash/logstash.conf. Plz help me out.
I kind of find the solution to it First mount the volume in /usr/share/logstash/pipeline instead of randomly create directory. So can remove the command in the yaml file Second, in logstash.conf, change the hosts to {container-name}}:9200 and also add ssl_certificate_verification = false sample output in logstash.conf output { elasticsearch { hosts => ["my-elasticsearch:9200"] index => "index-name" user => "elastic" password => "password" ssl_certificate_verification => false } } ps: I am using 8.4.1 version
thank you for this awesome tutorial, maybe anyone knows how I can solve an issue where logstash is not authorized to reach the elasticsearch host? I was reading it's something related to the logstash output configuration, but I was wondering if I did something wrong, since on the video it works smoothly. Thanks in advance
hi thank you for the video, I have a question pls if I get between 190 and 210GB of logs (firewalls and AD) how much CPU, storage and RAM do I need? and also, how can I setup my Logstash to receive logs from my Active directory Thank you for your help
I have run the compose file its running from last 30 minutes, not sure when it will end , can you please let me know if this is the normal behavior, also my internet bandwidth is good and system has desired resources for setup.
Do you know me? Have you ever spoken to me directly to know who is faking? Personally I don't like to spend time talking to people like you, who don't respect others or always negative mindset.
I think you need to grow up and think before giving feedback to someone. Everyone is having their freedom to share their view and this is yours. I don't care about it, as far as I think, it is not valuable feedback.
logstash | [2023-06-15T04:57:19,855][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"No Available connections"} I am getting error like this
![เป็นไปได้ไหม - WanMai [Official MV]](http://i.ytimg.com/vi/_6lZ6b3_7Vo/mqdefault.jpg)
best yml elk on youtube thank u brother !
You are welcome brother.
Thanks for good simple test example of elasticsearch in containers. ElasticSearch server started and accessed by browser without ssl cert. Logstash didn't accept log file even moved to logstash.
Can you check if the config is set correctly to send logstash to elastic search container.
I run into something similar, but in my case I tried the latest version which is 8.4.0. When I reverted to 7.16.2 as in the video tutorial... taaa daaa logstash can read the log file.
Excellent, but how can you do it using TLS?
Great vídeo dude, if I want to send logs with other server to elk I only need to install filebeat and point to logstash and elastic for this docker right?
Yes. That is correct.
Thank you. I have tested in an Ubuntu VM and it works fine. I would like to collect Netflow v5 data through port 2055 for example. May I ask you for orientation of how to modify your docker compose file to add elastiflow?
You are welcome. Need to re-look on your request.
Thank you for sharing..... if you're looking for the most recent version, typing ImageName:latest instead of version number.
It may not work with Elastic search, kibana . Try it and let me know. You will have to specify the version.
Thanks indeed for your efforts and sharing great contents. I am learning a lot from your channel.
Wow!. I am super happy to see your feedback. Thank you and keep learning.
Sir how can we add host or remote machine like EC2 to our elk stack which is hosted on ec2 instance
Thank you very much. Very helpful...
You are welcome. Happy to know.
Good tutorial! Have you tried to upgrade the version of the components to the latest and run it? For me, when I tried the 8.4.0 it will all started up, but no index gets created almost as if the logstash chooses not to read from the log file (it detects the log though)
Not tried. Thanks for the information.
my container of logstash keeps restarting... after i started docker-compose up command also when i enter the 5601 from the browser logstash says that it's missing security requirements...
Please check the container logs to see the reason and take action based on it.
I have updated the logstash.conf file but that file was not found logstash container. I have .log & simple-logstashfile.conf along with logstash/logstash.conf. Plz help me out.
Did you started Logstash with the config you want to apply?
@@Thetips4you Yes, all 3 docker started, but mounted directory not visible on the docker logstash server
@@dineshgupta01 Hello, do u find the answer?
I kind of find the solution to it
First mount the volume in /usr/share/logstash/pipeline instead of randomly create directory. So can remove the command in the yaml file
Second, in logstash.conf, change the hosts to {container-name}}:9200 and also add ssl_certificate_verification = false
sample output in logstash.conf
output {
elasticsearch {
hosts => ["my-elasticsearch:9200"]
index => "index-name"
user => "elastic"
password => "password"
ssl_certificate_verification => false
}
}
ps: I am using 8.4.1 version
Great tutorial. I was getting some memory issues in my machine which had 8gb but I'll use this to see if I can restrict it
Thank you.
thank u for the video.
i have to run the docker-compose file everytime i want to change the Logstash conf file?
No. If you have any changes in conf file you can update it, so by default the docker compose will take it.
@@Thetips4you i didn't found the index for Logstash in the index management.
Can u help me ?
@@azzabenabid2669 Me too
@@doublee7417 just add the elasticsearch url in logstash service part and the kibana service part
@@azzabenabid2669 where ? in the yml file? can you give me details pleaaase
Hi Bro is this a cloud version of elastic stack you are showing?
No. Local installation.
@@Thetips4you Bro you have knowledge on how to setup snmp in logstash for monitoring network device on kibana/elastic search
Can you update it into ELK Stack 8.14.1 (latest) vesion?
Can you provide an example of how to setup logstash to receive from UDP or HTTP?
Let me look in to this.
thank you for this awesome tutorial, maybe anyone knows how I can solve an issue where logstash is not authorized to reach the elasticsearch host? I was reading it's something related to the logstash output configuration, but I was wondering if I did something wrong, since on the video it works smoothly.
Thanks in advance
You are going. Do you want to share the error message about authentication?
Great tnx ! Can you maybe do it also for the latest one, elasticsearch:8.9.0 ? Because it is not working like this for that.
I will check
Thankyou this is awesome.
Thank you
thanks for your help
You are welcome
hi thank you for the video, I have a question pls if I get between 190 and 210GB of logs (firewalls and AD) how much CPU, storage and RAM do I need?
and also, how can I setup my Logstash to receive logs from my Active directory
Thank you for your help
Kindly check the elastic search hardware requirements and system requirements
Tried ok.
Unless you have other tasks use up too many resources, it should be fine.
Ok
Great content, please where is the link to copy the content (compose file) from github?
Please check the video description.
good man keep going
Thank you man
I want to use opensearch logstash and opensearch-dashboard. is it similar?
Not sure. Opensearch is forked from elastic search as per my understanding.
@@Thetips4you in the example, input file log must created inside container logstash or at the marchine host will also work
You docker-compose doesn't work, logstash exited with code 0
Check the logs for exact error.
I am having issues. I am using windows and I followed the above steps in the video, But I don't see the logs getting reflected on Kibana
An error message?
@@Thetips4you i don't see any error message.
Please update this video to 8th version of ELK. Also your github link in description is not working :(
Sure. The github link is working for me. Please check it again.
What does version: '3.6' represent here?
Do you mean the docker compose version?
Thanks for this video. Very helpful! :)
You are welcome.
I have run the compose file its running from last 30 minutes, not sure when it will end , can you please let me know if this is the normal behavior, also my internet bandwidth is good and system has desired resources for setup.
Did you used -d with docker compose up command?
@@Thetips4you thanks its working now, could you please send me the link of any vide where the data is send to elasticsearch and visualize in kibana
@@ImranKhan-fr8bn is the content written in log file is showing in port
Doesn't work. Kibana requires me to enter an "Enrollment token".
why not use elk-docker image, it already implement all three technologies
Do you mean official elk docker image?
@@Thetips4you the same
super easy to follow, i like it!
Thank you
This doesn’t work on Mac M1 though?
I didn't tried in MaC
where can i find the elasticsearch.yml file
It will be under elasticsearch home path, config. You can login to the container to check that or map it as a volume to control from outside m
its showing kibana is not yet ready
Check the docker logs on why it is not ready.
@@Thetips4you i checked them localahost:9200 and localhost:9600 showing output but this kibana is not showing output
why is the index empty and the log file has content
did u find a solution to that?
@@mohamedmazlin218 I am having the same issue. The content of the index is 0. did you find a solution?
Thank you ❤
You are welcome
Buen video 👍
Gracias
Great tutorial
Thank you
Excuse me wheres part 2?
Please check the playlist
fake slang
You are the expert here. If you say so.
@@Thetips4you I can easily identify who is faking ! no need to fake slang ! Our own accent is beautiful and understood by everyone
Do you know me? Have you ever spoken to me directly to know who is faking? Personally I don't like to spend time talking to people like you, who don't respect others or always negative mindset.
@@Thetips4you your heart knows you are faking accent ! chill bro 😎
I think you need to grow up and think before giving feedback to someone. Everyone is having their freedom to share their view and this is yours. I don't care about it, as far as I think, it is not valuable feedback.
logstash | [2023-06-15T04:57:19,855][ERROR][logstash.licensechecker.licensereader] Unable to retrieve license information from license server {:message=>"No Available connections"}
I am getting error like this
Check if the ports are not blocked and configurations are correct.
@@Thetips4you every thing is fine but it not displaying the result sir