วีดีโอ

wonderful work❤

I started going through a Udemy course on Elasticsearch and came to a section about Shards. When the chapter was complete I still didn't understand fully what a shard is. I searched on TH-cam and ended up on this video. You do a great job of explaining it so I (as a complete beginner) have a better understanding. Kudos to you for providing this video :)

Very clearly explained. Thanks

Awesome explanation, I love your narrative style, it really underlines the why and how of the current ecosystem!

This is really one of the most useful videos that introduced ES to me.

Thanks a lot!! Impeccable content!

I just have say that you are a great presenter! I've only done minor stuff with ElasticSearch but there were lots of things to consider if our company decides to expand the use cases that are implemented with help of ES.

Hi thanks for a video. For example we have: "unassigned_shards" : 40, When we run: GET _cluster/allocation/explain?filter_path=index,node_allocation_decisions.node_name,node_allocation_decisions.deciders.* { "index": "elastalert_past", "shard": 0, "primary": false } We reiceve next answer: "explanation" : "a copy of this shard is already allocated to this node [[elastalert_past][0], node[JaLzrdasdajQ], [P], s[STARTED], a[id=OmY9kwpHTlybJfSrWvdsadada6g]]" We have only one node and what we can do in this situation ? Also we have "number_of_replicas" : "0", "auto_expand_replicas" : "false", what we can do in this situation ? GET /.kibana/_settings { ".kibana_2" : { "settings" : { "index" : { "number_of_shards" : "1", "auto_expand_replicas" : "false", "provided_name" : ".kibana_2", "creation_date" : "1601664093", "number_of_replicas" : "0", "uuid" : "WKdIpzLFSP-ydObLw", "version" : { "created" : "7090299" } } } } }

Nice job. Very clear

I have just set up singlenode ELK Stack in our environment. It aggregates logs from network application stack via Filebeat from multiple servers and are then futher parsed via Logstash. It's main use should be for diagnostics. Basicaly vision is that L3 can quickly look at the logs and create visualization for single session via REST call from some Web server frontend. They can then easily focus/lens on problematic packets or diagnose where is the problem without need to use Oracle DB and writing queries. Everything is sort of streamlined and easy to use. I can imagine that similar solutions are used for example for security logs. Like i have said I dont have that much knowledge/experience with ELK Stack, but I think that it depends on your application. My opinion is that i wouldn't necessary use Elastic for metrics because there are much better tool for that out there. Also i would like to add that I guess that developers of ELK knows that it's complicated and their documentation is quite nice and understandable, but be careful because you can get lost in the "loop", because there are multiple link to manual that talks about he same thing in different levels of granularity so I would suggest before configuring anything to read through the manula very quickly just to be sure the end result is what you want. And as always less is sometimes more, so keep it simple at the start and as you get more knowledge you can add more features or scale up.

where was thsi video 1 month ago. you should be paid by these software companies... bless your heart

great job

Thank you for creating this it is very helpful indeed.

Many thanks for your work! It's awesome video!

why don't you use the http "PATCH" method?

this is incredible guide! the whole playlist is amazing and very helpful! thank you for this!

I was planning to run a elastic search image for work on docker, thinking it would be a simple process. this video is an eye opener, now I'm not sure if I should go for it or not, our use case is something like yelp with much less data

Thank you so much :)

1000th like 😅

These are great points - within an organisation where elastic ends up in production without really fleshing out the way it should be used and having teams onboarding their data without true understanding of the data modelling that needs to be done and understood upfront is a major pain to fix later down the line. Common fields are essential (i would say critical) to make most of it when trying to get your data out in a useful way.

great video!

good video, but, how to di it with certs, i mean lots of certs, lots of chages we need to create certs and send it to other nodes and there move to anoter paths, and rewrite some code, My queston is how to send it via ansible?

Its two year late but the lesson is extremely value

I just started learning Elastic search and this is the best an clear information on Elastic search architecture. Thanks for sharing!

I can attest to the challenges you described. I’m the only one in my organization using elasticsearch. Everyone else is afraid to touch it. I do not blame them.

How about using Elasticsearch for only term searches and doc id queries? Bad use case?

you are god sent <3

I am trying to install and configure Elasticsearch , Logstash , Kibana and Filebeat to collet some logs but struggling to get this thing going.

Hi, can you publish certain lab exercises for Elastic certified analyst? Thanks

Thanks for the exercises, for ex. 9 I used a bool with a filter and for ex. 10 I used grok

Well that's just put me off completely

this is great content. thank you.

Thanks for this outstanding series. I only wish more tutorials were this clear and concise.

This is amazing. Lot of topics covered in one short video. Kudo's and keep up the good work.

am currently building a project that requires semantic search, i wanted to pay someone soo quickly to get this done (i can do it myself but i don't want to spend the time), i thought ELK would be such a breeze, thanks for saving me the time and money :)

If elasticsearch distributes the data between the shards of an index such that each lucene store roughly holds the same number of documents, when you run a search query, elasticsearch, despite the inter-node communication, only knows which shards hold that index and not which particular shard will have that document? So it has to run the query against all the shards and merge results, it cannot just search the one shard that contains that document? It does not know beforehand based on how documents are distributed among shards.

This video changed my life. No exaggeration.

I didn't know about minimum_should_match, that explains a few things!

These are great, nothing like this on the web keep them coming!

I think that shard in ES has the same concept with partition in Kafka when they have all partition replicas in different nodes

You can also use runtime fields to pull and query that field from the _source. The pro is that in large data sets, you save on storage space with fields that are not indexed. The con is that it uses resources to run the runtime field's request. The nice thing is, that field will exist when running the query so you can then search on it even though it was not indexed.

The worst thing is when you meet Solution Architect who want you to increase index mapping total_fields limit just to fit badly design mapping. Good point! Good sound quality. wish to have same one day

Great work sir

Awesome explanation

Hi @GeorgeBridgemanData - can Elsatic search be used for types of files that it may never have seen before and "shown" how to parse them?

Great work sir ❤

Great work

Hi George, how can I contact you in person? We would like to invite you as a guest to a function of ours to talk about Elasticsearch.

Certainly can attest to point #1 - We went into our project (hundreds of millions of records, for full document text search) thinking we would have completely flattened data. Later on in the project we saw a serious need for some form of hierarchy/relationship - I see now why ES HIGHLY recommends flattened data approaches - the hasParent/hasChild queries can be complicated and super expensive, especially for aggregations.

Did you try to use dynamic templates in OpenSearch? It seems OpenSearch doesn't support it.