Great tutorial! Thank for covering variety of topics. It would ge great if you start putting notes, slides, references to a Github repository. In this way, other developers can also contribute.
waow .. first time I opened this video.. It really made me fall in love with your teaching skills, way of communication and your kind body language ❤❤🔥🔥🔥🔥🔥.. Your are legend sir 👍👍👍
soo splendid, but what if the capacity (number of rows in the database) grows, so we will have bigger indexes, thus will need to interate through a larger portion of indexes?
May be the table needs a better design here. Why not pull out status and stock in their own tables and use the status_id and stock_id as columns in the main table. Integer comparisons are faster.
Question: What if any record from table is deleted ? Just turning that bit to 0 would not work much, as query could be number of users who are not approved, which in turn can include deleted users as well
If we create an index like CREATE INDEX index_name ON table_name (column_name). will the database handle it automatically for BITMAP or do I have to do any extra things to use BITMAP indexing for status column?
What you described as "Cardinality" is what I learned as "data sparsity". Cardinality refers to the relationship between rows in different tables, e.g. 1 to many (1:M), 1 to 1 (1:1) or many to many (M:N).
Conventional wisdom hulds that bitmap indexes are most appropriate for columns having low distinct values--such as GENDER, MARITAL_STATUS, and RELATION. This assumption is not completely accurate, however. In reality, a bitmap index is always advisable for systems in which data is not frequently updated by many concurrent systems. In fact, as I'll demonstrate here, a bitmap index on a culumn with 100-percent unique values (a culumn candidate for primary key) is as efficient as a B-tree index. Usage of bitmap indexes is not in fact cardinality dependent but rather application dependent. SRC : www.oracle.com/technical-resources/articles/sharma-indexes.html
Thanks for sharing. However this doesn't answer the following. 1) How is the bit map index updated when you add a new ID as well as add a new type called "ON HOLD". Is the index resized when it crosses a particular threshold (say 2^n -1 )? Is it a power of 2? 2) How is the complexity of the query reduced? For e.g select all products where status in ALL except rejected and stock in ALL. In this case, how are we saving on compute because we need to loop through all bitmaps on both columns. If that's the case, there's no exploitation of the fact that the cardinality of this column in reduced. For example. why not compute a bitmap for TRUE, FALSE, TRUE || FALSE, TRUE && FALSE and keep it ready as compared to just TRUE , FALSE now. We can afford to do this as the cardinality is less.
Hi, thanks for the video, how will the update work, if I have to update the nth element from 1 to 0, how would I seek that element without having to go through all the numbers. Also having ids in order seem to be one of the biggest requirement for this use-case, which is not possible most times.
Omg you could explain everything in 5 minutes instead of repeating 100 times each simple idea. And you didn't explain why it's better than seq scan, great!! (no)
In theory, yes, there are only 3 distinct values in this column, and yes, that means that one only really needs two bits to hold that info. But if we continue down that route, we now have a column that requires just two bits. How can one generalise that into anything useful? How can it be efficiently stored and indexed? What does one do when someone adds some new statuses? It’s just not practical. Using bitmap indexing where appropriate is a useful and good generalisation.
isn't this ridiculously wrong? you don't build bit map index, you build a btree index bitmap is index access, the way planner decides to access the btree index. dude, wtf, bitmap index access is all about not accessing the same block page
Thank you for very good explaination.
Good explanation, very easy to follow. Thanks!
thank you for explaining the topic in such a simple and effective manner.
Thanks from Malaysia. It is really clear.
Excellent explaination in a very simpler way,thank you
good explanation. better than documentations I read before
Great tutorial! Thank for covering variety of topics. It would ge great if you start putting notes, slides, references to a Github repository. In this way, other developers can also contribute.
Great Job Broo! The way you speak and explain is very clear!
Thank you for a very good tutorial.
Your videos are amazing
Excellent explanation! Thank You!
U explains things very clearly,love your content
Nice visualization, thanks!
Good explanation
Super video. Thanks
Great explanation! Thanks so much!
Hey, it will be great if you also share the reference materials as well. thanks
Thanks for sharing. Good material
waow .. first time I opened this video.. It really made me fall in love with your teaching skills, way of communication and your kind body language ❤❤🔥🔥🔥🔥🔥.. Your are legend sir 👍👍👍
Congrats bro , I opened it for the 100th time today
Will continue opening it everyday till I reach my grave
Great explanation, Redis provides bitmap data structure.
Well explained. Thank you!
Great video! Simple example to explain complex concept - love it! Thanks. : )
Thank you, sir! You saved my day! Nice explanation 🙏
amazing ;)
thank you so much sir your explanation is awesome
question: if you have to go row by row in the bitmap index to find the matches, how is it different than a fullscan?
Best explaination ever
Good One
Hi, nice vídeo!
I would like to have it short, but without losing any content.
Thanks for sharing 👍
Very insightful video , can you also please tell me the book or document that you referred to understand this
Good One!
Great explanation, thanks for sharing knowledge, My only suggestion is to reduce repetitive statements.
Hey, you explained Bitmap indexing concept really well 👍🏻. Do you have one for cluster indexing?
As for as oracle , the definition for cardinality is some what not same as you explained , but your explanation of bitmap is good
soo splendid, but what if the capacity (number of rows in the database) grows, so we will have bigger indexes, thus will need to interate through a larger portion of indexes?
crystal clear
Thanks for this video!!! Man!!! Could u do a video about an example using bitmap and b-tree together :D Have a nice day!
Thank u very much
Even if we find all 1st in bitwise array, it's O(N) ?
May be the table needs a better design here. Why not pull out status and stock in their own tables and use the status_id and stock_id as columns in the main table. Integer comparisons are faster.
How about to have a session discussing about the design distributed message queue system?
thank you sir. it is really helpful. could you please suggest me a reference paper or book for this?
What do you think of CrimsonDB?
Question: What if any record from table is deleted ? Just turning that bit to 0 would not work much, as query could be number of users who are not approved, which in turn can include deleted users as well
Thanks You!
thanks
Do some live with us!!!!!
hey! can you do a system design for Expidia?
If we create an index like CREATE INDEX index_name ON table_name (column_name). will the database handle it automatically for BITMAP or do I have to do any extra things to use BITMAP indexing for status column?
Hi, bro, I see you hadn't updated for nearly half a year. Are you OK?
Good content, but the volume is very low.
What is the relationship between bitmap indexing and high dimensional indexing problem???
can you suggest the list of books plz?
Isn’t this same as columnar format the parquet uses?
What you described as "Cardinality" is what I learned as "data sparsity". Cardinality refers to the relationship between rows in different tables, e.g. 1 to many (1:M), 1 to 1 (1:1) or many to many (M:N).
'cardinality' is the same as 'size'
en.wikipedia.org/wiki/Cardinal_number
sparsity describes the number of sparse items (e.g. empty rows in DB)
Conventional wisdom hulds that bitmap indexes are most appropriate for columns having low distinct values--such as GENDER, MARITAL_STATUS, and RELATION. This assumption is not completely accurate, however. In reality, a bitmap index is always advisable for systems in which data is not frequently updated by many concurrent systems. In fact, as I'll demonstrate here, a bitmap index on a culumn with 100-percent unique values (a culumn candidate for primary key) is as efficient as a B-tree index.
Usage of bitmap indexes is not in fact cardinality dependent but rather application dependent.
SRC : www.oracle.com/technical-resources/articles/sharma-indexes.html
Great Explanation! so you explained it nicely for 6 rows and it has 6 bits but how does it scale for millions of rows. we can't have millions of bits.
Whats' the issues with millions rows. If we you take 1 million rows and 4 cardinality, its just 500KB data.
Thanks for sharing. However this doesn't answer the following.
1) How is the bit map index updated when you add a new ID as well as add a new type called "ON HOLD". Is the index resized when it crosses a particular threshold (say 2^n -1 )? Is it a power of 2?
2) How is the complexity of the query reduced? For e.g select all products where status in ALL except rejected and stock in ALL. In this case, how are we saving on compute because we need to loop through all bitmaps on both columns.
If that's the case, there's no exploitation of the fact that the cardinality of this column in reduced. For example. why not compute a bitmap for TRUE, FALSE, TRUE || FALSE, TRUE && FALSE and keep it ready as compared to just TRUE , FALSE now. We can afford to do this as the cardinality is less.
actually this video is useless =/
I too had the same questions in mind.
Hi, thanks for the video, how will the update work, if I have to update the nth element from 1 to 0, how would I seek that element without having to go through all the numbers. Also having ids in order seem to be one of the biggest requirement for this use-case, which is not possible most times.
Omg you could explain everything in 5 minutes instead of repeating 100 times each simple idea. And you didn't explain why it's better than seq scan, great!! (no)
how we scale this kind of operation lets say on 1M rows
Why can't we use 3 numbers for each row. 0 for approved, 1 for pending, 3 for rejected.
It is a bit, not character/string datatype
In theory, yes, there are only 3 distinct values in this column, and yes, that means that one only really needs two bits to hold that info. But if we continue down that route, we now have a column that requires just two bits. How can one generalise that into anything useful? How can it be efficiently stored and indexed? What does one do when someone adds some new statuses? It’s just not practical. Using bitmap indexing where appropriate is a useful and good generalisation.
Anyone knows his twitter handle, please let me know
isn't this ridiculously wrong?
you don't build bit map index, you build a btree index
bitmap is index access, the way planner decides to access the btree index.
dude, wtf, bitmap index access is all about not accessing the same block page
Indicies, not indexes. Thought you were all geniuses?
thanks
thanks