10:59 yea what they mean is that entropy seems to be what they are explaining, and if someone disagrees, you can ask them, make a mathematical proof (different to ours) that defines "entropy". what Shannon found I think is entropy. but sadly I don't think @Reducible is explaining it quite right. Entropy and prediction should not be in the same sentence. Entropy is looking at data/environment and asking how much information can be used to define this information. high entropy is when the information cannot be described with less information. repeating patterns of course the easiest defined by less information the the original information. Prediction (inference-ing) has a time/event element to it. Or if you did approach a repeating pattern it doesn't look at the whole pattern and is always asking from what I can see of the pattern so far what would be the next part of it? something is not very predictable if you can't guess what comes next. it sounds the same but really should be careful to overlap these ideas.
The wisest thing Huffman’s professor did was not mention it was an unsolved problem. Great, clear presentation, others on youTube are so fast they require you to pretty much already know how it works before you watch it, which is insane.
The section leading up to 8:00 epitomizes a problem solving technique that sets apart mathematicians: rather than directly search for a formula that describes a thing, instead list what *properties* you expect that thing to have. Make deductions from those properties, and eventually all kinds of insights and formulas fall on your lap.
Would encoding something in Base Three (using either "0, 1, 2" or "0, 1, .5" ) increase the amount of information transmitted because there's an extra number or decrease the amount of information transmitted because you can make more complicated patterns?
@@sixhundredandfive7123 Base three (or any alphabet of three symbols) can encode more information *per symbol*, which means fewer symbols are needed to represent some information (a message). That would mean messages are shorter but have the same overall amount of information.
Yeah that's also how you actually obtain the determinant, the most striking example to me. You prove that the 'set' of all alternating (swapping two rows makes it negative) multilinear (breakup of the determinant based on the row property) forms on matrices (with elements belonging to a commutative ring with identity), and which outputs 1 for the identity matrix is a unique function, the determinant. From this definition you can obtain a nicer result that is every alternating n linear form D on n n-tuples which give the matrix A on stacking must satisfy D(A) = detA D(I). This is used to prove a lot of properties concerning determinants.
Many years ago after studying this algorithm I sent a fan letter to D. Huffman. He wrote back in appreciation because he'd never gotten a fan letter before.
Having also learned the Huffman encoding algorithm as an example of a greedy algorithm (along with its accompanying greedy-style proof), this video provided me with a new perspective which, combined with the interesting history of alternative algorithms, gave me a fuller understanding and appreciation for the topic, which I have to admit is extremely well presented!
20:26, "optimal binary representation must have the two least likely symbols at the bottom of the tree" My first thought: "So can we count it as one node and repeat the process? Nah, that can't be right, that would be too easy" XD turns out it really is easy
You actually need to proof that you may build node with two of those. Also, author of video didn't prove why they should have longest number of bits, it's not so obvious, you need to apply exchange argument, most basic one though.
Man it's so amazing that any of this works at all. Being an autodidact philomath and self educated amateur mathematician and cryptographer, I am so glad to have all of this information available to learn at my own pace. Thank you.
I just recently learned about Huffman encoding and this video is absolutely AMAZING. You really motivated the solution incredibly well! Congratulations and hope you make more videos
This video does an excellent job giving an intuition. It's still complicated for someone who hasn't done much with information theory, so I had to pause to really understand some parts of it -- but this video helps a lot and makes the topic really interesting. This idea of entropy in information theory is also used in seemingly unrelated areas, like deciding which splits to take with a decision tree (or random forests).
This has got to be the most amazing video I have ever seen. Seriously! It was absolutely the most intriguing thing I have never pondered. Great explanation skills for slow people like me but yet very entertaining at the same time. Thank you sir for your effort and information!
woahhhhhhhhhh 🔥🔥🔥🔥🔥🔥 I absolutely have no words for this masterpiece. The way you explained it, now I'm going to take my information theory classes seriously 😂 Thanks a lot. Great work! Sharing this with my friends as this deserves lot more appreciation ♥️
Very good video. That's the first one I see on youtube that put together Shannon-Fano and Huffman coding and tries to explain the differences between them. Most of them only explain the Huffman algorthm and discard all the history behind it
I really love this algorithms that look so obvious and simple yet I know that even with all the time on the world I couldn't have invented them myself.
I've watched many of your beautiful presentations and found them very instructive. This one not only clearly demonstrates Huffman's idea , but also the "bottleneck" in any communication channel formalized by Shannon himself.
Great video. Though I was a bit confused in the end and going further in depth with an example of using the huffman encoding and decoding, while comparing it to uncompressed encoding would make it a bit clearer. Still an amazing video!
Huffman encoding is only optimal if each symbol is conditionally independent of all previous symbols, which is almost never the case, which is why Huffman encodings didn’t solve compression, though they are typically used as part of a broader compression algorithm. Ray Solomonoff did a lot of important work in this area with his invention of algorithmic complexity theory
Ooh interesting. In the equation constraints section it was mentioned (almost as just a side note) that the events had to be independent of each other, and for a moment I wondered what if they weren’t independent? Now I want to know :0
@@redpepper74 You can compress a lot more when the symbols are conditionally dependent upon previous symbols, but you need to use other methods, typically those that rely upon conditional probability
@@redpepper74 Huffman encodings are based purely on symbol frequencies, and don’t take into account what symbols came before. So for example, if you see the sequence of symbols “Mary had a little” you can immediately recognize that the next four symbols are very likely to be “lamb”, with a high degree of probability. Huffman codes do not see this
@@grahamhenry9368 I suppose what you could do then is get a separate Huffman encoding table for each symbol that tells you how likely the next symbol is. And instead of using single characters you could use the longest, most common groups of characters
@@redpepper74 Yeap, I actually coded this up once, but only looking back at the last character. The important idea here is that the ability to predict future symbols directly correlates with how well you can compress the data because if a symbol is very probable then you can encode it with fewer bits via a Huffman encoding. This shows that compression and prediction are two sides of the same coin. Markus Hutter has done a lot of work in this area with his AI system (AIXI) that uses compression algorithms to make predictions and attempts to identify optimal beliefs
I think there's a missing negative sign in the lengths at 16:11,anyway one of the best videos on the topic, I subscribed awhile ago and i don't regret it .
The Huffman encoding algorithm looks very similar to dynamic programming. So I wondered which came first, and it seems that they where both development around the time, namely around the year 1952.
Yea I googled and apparently, this is a greedy algorithm that converges to the global optimal. Greedy algos and dynamic programming are similar in that they utilise the subproblems to build up the full solution. In general greedy algorithms only converge to local optimizers as they do not exhaustively check all subproblems and is faster. DP is slower but global optimality is guaranteed. For this problem the greedy algo guarantees the global optimal and we have the best of both worlds.
Wow. Great video! The way you explained it was so clear and succinct that I guessed "Oh shit it's recursion!!" at 23:12. Good work on the organisation of explaining topics & pacing, love this channel!
23:40 I have seen other videos about the Huffman tree construction before watching this video, but this explanation is clearest. I was not sure how the nodes after the initial two nodes should be added.
TH-cam algorithm usually sucks, just put on your feed some shitty latin music or just the stupid trending stuff, but today feels generous, long long time ago that I don't see content like this, man keep going, you have an incredible way to explain things, simple but clear and clever, amazing channel pal
Incredible educative video, I respect the amount of work you put into this! I enjoyed the overview of the field and the connection between information and probability theory was splendidly shown! I'm looking forward to your future videos!
21:09 S_1 S_2 don't HAVE to be on the same node on the optimal tree, in fact they can be exchanged with any other nodes on the bottom layer. What is instead true is that you always can construct an optimal tree with them paired from one where they are not through the aforementioned exchange.
I already knew huffman's algo before watching this video....6:53 had me goosebumps :) i mean what---?!?!?!? the correlation omg ... tysmmm for this _/\_
The Huffman algo is so brilliant... GIF's LZW is another system that sounded obscure until I dared looking and it was in fact much simpler than I thought. It would be a great subject for your next videos.
3:20 I think these are misrepresented in the video. 1) "single symbol -> unique binary code' doesn't mean you can't map multiple symbols to a single code, it means you can't map the same sequence of symbols to multiple codes. 2) "Lossless" means the decoded message has to be exactly the same as the source, not an approximation of the original. Nothing to do with deleting random bits.
This was a really cool watch! The more I immerse myself into information theory the more it interests me :D I do disagree with the part about the importance naming things though- names have power! The roots in a word tell you what it means, the meanings of a word in some contexts hint at the meaning in others, and the connotation that a word has affects the way that people think about it. (Yes my passion for writing clean code has turned me into someone who thinks way too much about naming conventions :P )
On the other hand, words can develop meanings well beyond their origins. Also, even the original meaning doesn't always perfectly correlate to the roots. It just isn't as simply as you claim.
I seriously dont understand how the entropy of those weather distributions are calculated at 9:34 Looking at the Seattle case: The self-information of sunny weather is 1,74 bits, as the logarithm of 0,3 is -1,74, right? In the same way, the self-information of rainy weather is 1 bit, and the self-information of snowy weather is 2,32 bits. The entropy should then be 0,3*1,74 + 0,5*1 + 0,2*2,32 = 1,486 bits, right? How do we arrive at an entropy of 0,46 bits, as shown in the video? Where in the calculation am I mistaken?
Ah thank you for bringing this up. I made a mistake here and accidentally in the code calculated this with log base 10 rather than log base 2. I have updated the description of the video with the error. Sorry about that!
Oh, nice. I just tried to calculate the entropy myself, to check whether I had understood it. And then I just concluded, that I clearly hadn't understood it. But it turns out I actually do understand it! Nice!
It is really impressive how he came up with this solution - how many concept he have to understand to garantee optimal static encoding. I wonder how he can transpit particular codig for that message. And if we you modification during decoding, is there possibility to shring coding a little bit more? Like when we has message ABDCBBB, first letter we encode only A, then AB, then ABD and so on (last letter is encoding as we would encode ABDCBBB). Decoder then has to start from the end and know how many times each letter should apper.
Thanks again fantastic video. 17:37 with the ShannonFano code I thought wow this thing is just guessing, it will way too often give B(0.35) two bits, although 1 bit would be better. Try with an even more extreme distribution like this D(0.14), E(0.14), A(0.15), C(0.17), B(0.4) And it will still assign two bits to B. So this must be non optimal, just from the first impression. 24:29 yeah, this is it, this Huffman looks like an optimal recursive function, just intuitively right
I’m not too proud to admit that the first time I tried implementing Huffman Encoding, I just could not wrap my head around it. It didn’t matter how many times I read that Huffman was “Bottom up”, I tried to implement the algorithm “Top down”. That’s just the natural way we think about trees. As you might imagine, I crashed pretty quickly and had to walk through some examples in detail until I wrapped my head around the “Bottom up” nature.
This video reminds me a coursework that I had almost 20 years ago. I had to write an implementation of the Huffman Algorithm in Assembler. What I remember is that I had to make two passes over a file: the first to count frequencies and the second to encode the file. It didn't work for streams.
Thanks fro the great explanation! However, one thing isn't clear for me. On 15:29 you say that the average length for the symbol is 1.75 (exactly as the entropy). But the codes are 1, 01, 000 and 001, so the total length is 9, and in average it turns out to be 9/4=2.25 (which is not 1.75 as stated) Am I getting something wrong?
I love the stories where students solve unsolved problems just because the professor neglected to tell them it was difficult.
Me too, those are the best stories.
"Just call it entropy, nobody really knows what it is" Truly an intelligent man.
I think either John Von Neuman was joking or just being a dickhead
10:59 yea what they mean is that entropy seems to be what they are explaining, and if someone disagrees, you can ask them, make a mathematical proof (different to ours) that defines "entropy". what Shannon found I think is entropy. but sadly I don't think @Reducible is explaining it quite right. Entropy and prediction should not be in the same sentence. Entropy is looking at data/environment and asking how much information can be used to define this information. high entropy is when the information cannot be described with less information. repeating patterns of course the easiest defined by less information the the original information.
Prediction (inference-ing) has a time/event element to it. Or if you did approach a repeating pattern it doesn't look at the whole pattern and is always asking from what I can see of the pattern so far what would be the next part of it? something is not very predictable if you can't guess what comes next. it sounds the same but really should be careful to overlap these ideas.
The legend is back. I love your work, the production quality, the content, everything! You are the computer science equivalent of 3b1b.
Hopefully not the equivalent of the 3b1b who apparently stopped making videos
@@muskyoxes looks like he is that too lol
@@muskyoxes b...but he still makes videos...
The wisest thing Huffman’s professor did was not mention it was an unsolved problem.
Great, clear presentation, others on youTube are so fast they require you to pretty much already know how it works before you watch it, which is insane.
👌🏽
The section leading up to 8:00 epitomizes a problem solving technique that sets apart mathematicians: rather than directly search for a formula that describes a thing, instead list what *properties* you expect that thing to have. Make deductions from those properties, and eventually all kinds of insights and formulas fall on your lap.
Would encoding something in Base Three (using either "0, 1, 2" or "0, 1, .5" ) increase the amount of information transmitted because there's an extra number or decrease the amount of information transmitted because you can make more complicated patterns?
@@sixhundredandfive7123
Base three (or any alphabet of three symbols) can encode more information *per symbol*, which means fewer symbols are needed to represent some information (a message). That would mean messages are shorter but have the same overall amount of information.
Yeah that's also how you actually obtain the determinant, the most striking example to me. You prove that the 'set' of all alternating (swapping two rows makes it negative) multilinear (breakup of the determinant based on the row property) forms on matrices (with elements belonging to a commutative ring with identity), and which outputs 1 for the identity matrix is a unique function, the determinant.
From this definition you can obtain a nicer result that is every alternating n linear form D on n n-tuples which give the matrix A on stacking must satisfy
D(A) = detA D(I). This is used to prove a lot of properties concerning determinants.
Many years ago after studying this algorithm I sent a fan letter to D. Huffman. He wrote back in appreciation because he'd never gotten a fan letter before.
Having also learned the Huffman encoding algorithm as an example of a greedy algorithm (along with its accompanying greedy-style proof), this video provided me with a new perspective which, combined with the interesting history of alternative algorithms, gave me a fuller understanding and appreciation for the topic, which I have to admit is extremely well presented!
20:26, "optimal binary representation must have the two least likely symbols at the bottom of the tree"
My first thought: "So can we count it as one node and repeat the process? Nah, that can't be right, that would be too easy"
XD turns out it really is easy
You actually need to proof that you may build node with two of those. Also, author of video didn't prove why they should have longest number of bits, it's not so obvious, you need to apply exchange argument, most basic one though.
At that moment I thought "just apply the same process recursively" which turns out to be the case
Man it's so amazing that any of this works at all. Being an autodidact philomath and self educated amateur mathematician and cryptographer, I am so glad to have all of this information available to learn at my own pace. Thank you.
I just recently learned about Huffman encoding and this video is absolutely AMAZING. You really motivated the solution incredibly well! Congratulations and hope you make more videos
The smallest shift in perspective can sometimes make all the difference
This video does an excellent job giving an intuition. It's still complicated for someone who hasn't done much with information theory, so I had to pause to really understand some parts of it -- but this video helps a lot and makes the topic really interesting.
This idea of entropy in information theory is also used in seemingly unrelated areas, like deciding which splits to take with a decision tree (or random forests).
This is very impressive. It takes a lot of hard to present so much so well. Excellent vid!
Your presentation is entertaining, thought-provoking and truly educational. One of the best channels on TH-cam in my opinion.
This video is an amazing compilation of information, in a well presented order and in an engaging way. Information well encoded! Cheers
This has got to be the most amazing video I have ever seen. Seriously! It was absolutely the most intriguing thing I have never pondered. Great explanation skills for slow people like me but yet very entertaining at the same time. Thank you sir for your effort and information!
woahhhhhhhhhh 🔥🔥🔥🔥🔥🔥 I absolutely have no words for this masterpiece. The way you explained it, now I'm going to take my information theory classes seriously 😂 Thanks a lot. Great work! Sharing this with my friends as this deserves lot more appreciation ♥️
Thank you, it's really helpful!
People be sleeping on this channel. Incredible content.
Your voice is great, the visuals are on point, and I finally understand Huffman codes. Great job; subscribed!
Very good video. That's the first one I see on youtube that put together Shannon-Fano and Huffman coding and tries to explain the differences between them. Most of them only explain the Huffman algorthm and discard all the history behind it
10:50 JvN's second and most important reasoning for the term _entropy_ is truly the genius of a mad mathematician.
How am I just finding this channel? This guy’s awesome
I really love this algorithms that look so obvious and simple yet I know that even with all the time on the world I couldn't have invented them myself.
This channel is pure gold
I've watched many of your beautiful presentations and found them very instructive. This one not only clearly demonstrates Huffman's idea , but also the "bottleneck" in any communication channel formalized by Shannon himself.
I'm taking numerical analysis course right now, and my chapter was just on Huffman Codes, great timing!
Great video. Though I was a bit confused in the end and going further in depth with an example of using the huffman encoding and decoding, while comparing it to uncompressed encoding would make it a bit clearer. Still an amazing video!
Huffman encoding is only optimal if each symbol is conditionally independent of all previous symbols, which is almost never the case, which is why Huffman encodings didn’t solve compression, though they are typically used as part of a broader compression algorithm.
Ray Solomonoff did a lot of important work in this area with his invention of algorithmic complexity theory
Ooh interesting. In the equation constraints section it was mentioned (almost as just a side note) that the events had to be independent of each other, and for a moment I wondered what if they weren’t independent? Now I want to know :0
@@redpepper74 You can compress a lot more when the symbols are conditionally dependent upon previous symbols, but you need to use other methods, typically those that rely upon conditional probability
@@redpepper74 Huffman encodings are based purely on symbol frequencies, and don’t take into account what symbols came before. So for example, if you see the sequence of symbols “Mary had a little” you can immediately recognize that the next four symbols are very likely to be “lamb”, with a high degree of probability. Huffman codes do not see this
@@grahamhenry9368 I suppose what you could do then is get a separate Huffman encoding table for each symbol that tells you how likely the next symbol is. And instead of using single characters you could use the longest, most common groups of characters
@@redpepper74 Yeap, I actually coded this up once, but only looking back at the last character. The important idea here is that the ability to predict future symbols directly correlates with how well you can compress the data because if a symbol is very probable then you can encode it with fewer bits via a Huffman encoding. This shows that compression and prediction are two sides of the same coin. Markus Hutter has done a lot of work in this area with his AI system (AIXI) that uses compression algorithms to make predictions and attempts to identify optimal beliefs
Dude long time no see.. I am so happy that you are back. Please make more videos...
I think there's a missing negative sign in the lengths at 16:11,anyway one of the best videos on the topic, I subscribed awhile ago and i don't regret it .
Really loved it! It was knowing about Huffman codes that made me take information theory classes. It is really an elegant piece of mathematics.
When I first learned how Huffman made a variable bit length signal get decoded I was, like, really happy!
Best explanation of Huffman Encoding I've seen. Bravo!
There was a lot of information in that video, and I could decode it all! 🙂
It is definitely not easy to explain something so well, so thank you.
Hey its me Huffman. I made these codes.
The Huffman encoding algorithm looks very similar to dynamic programming. So I wondered which came first, and it seems that they where both development around the time, namely around the year 1952.
Yea I googled and apparently, this is a greedy algorithm that converges to the global optimal. Greedy algos and dynamic programming are similar in that they utilise the subproblems to build up the full solution. In general greedy algorithms only converge to local optimizers as they do not exhaustively check all subproblems and is faster. DP is slower but global optimality is guaranteed. For this problem the greedy algo guarantees the global optimal and we have the best of both worlds.
Guy, thank you for this amazing video.
Wow. Great video!
The way you explained it was so clear and succinct that I guessed "Oh shit it's recursion!!" at 23:12. Good work on the organisation of explaining topics & pacing, love this channel!
Yoooo, your videos are awesome, great day when you upload
23:40 I have seen other videos about the Huffman tree construction before watching this video, but this explanation is clearest. I was not sure how the nodes after the initial two nodes should be added.
Underrated channel
TH-cam algorithm usually sucks, just put on your feed some shitty latin music or just the stupid trending stuff, but today feels generous, long long time ago that I don't see content like this, man keep going, you have an incredible way to explain things, simple but clear and clever, amazing channel pal
Incredible educative video, I respect the amount of work you put into this! I enjoyed the overview of the field and the connection between information and probability theory was splendidly shown! I'm looking forward to your future videos!
I'm only just starting to learn about Information Theory - but this was very accessible. Thanks, subscribed.
21:09 S_1 S_2 don't HAVE to be on the same node on the optimal tree, in fact they can be exchanged with any other nodes on the bottom layer. What is instead true is that you always can construct an optimal tree with them paired from one where they are not through the aforementioned exchange.
What an exceptional video , that was so fun !
Just yesterday I wondered when you'll upload next. Awesome video as usual!
I already knew huffman's algo before watching this video....6:53 had me goosebumps :) i mean what---?!?!?!? the correlation omg ... tysmmm for this _/\_
The Huffman algo is so brilliant...
GIF's LZW is another system that sounded obscure until I dared looking and it was in fact much simpler than I thought. It would be a great subject for your next videos.
As always, amazing quality of a video!! loved it. Please keep doing them
3:20 I think these are misrepresented in the video.
1) "single symbol -> unique binary code' doesn't mean you can't map multiple symbols to a single code, it means you can't map the same sequence of symbols to multiple codes.
2) "Lossless" means the decoded message has to be exactly the same as the source, not an approximation of the original. Nothing to do with deleting random bits.
Holy!! Thanks for the journey and nice video! :D
This was a really cool watch! The more I immerse myself into information theory the more it interests me :D
I do disagree with the part about the importance naming things though- names have power! The roots in a word tell you what it means, the meanings of a word in some contexts hint at the meaning in others, and the connotation that a word has affects the way that people think about it. (Yes my passion for writing clean code has turned me into someone who thinks way too much about naming conventions :P )
On the other hand, words can develop meanings well beyond their origins. Also, even the original meaning doesn't always perfectly correlate to the roots. It just isn't as simply as you claim.
This was so fun to watch. Please keep making more videos.
Lovely explanation and storytelling!
man I love this channel. so well made.
Great video. I really appreciate the hard work and explanations you put in your videos.
The day Reducible uploads is a good day
Brilliant video! Thanks for your wonderful effort.
Such a lovely explanation. 🤗
You are back.. 🥳🥳🥳✨✨✨✨
I am watching from last week approx it grow from 89k to 91.5k
Thank you sir because of your vedio I learned how uncertainty is compressed in our nature
Wow, great video man! I forgot multiple times that you aren't 3b1b
A lazy graduate student solves an important problem in a weekend.
He's my hero.
You broke this down soooo well
Your video is amazing! Thanks for the interesting presentation
I seriously dont understand how the entropy of those weather distributions are calculated at 9:34
Looking at the Seattle case:
The self-information of sunny weather is 1,74 bits, as the logarithm of 0,3 is -1,74, right? In the same way, the self-information of rainy weather is 1 bit, and the self-information of snowy weather is 2,32 bits.
The entropy should then be 0,3*1,74 + 0,5*1 + 0,2*2,32 = 1,486 bits, right? How do we arrive at an entropy of 0,46 bits, as shown in the video? Where in the calculation am I mistaken?
Ah thank you for bringing this up. I made a mistake here and accidentally in the code calculated this with log base 10 rather than log base 2. I have updated the description of the video with the error. Sorry about that!
Oh, nice. I just tried to calculate the entropy myself, to check whether I had understood it. And then I just concluded, that I clearly hadn't understood it. But it turns out I actually do understand it! Nice!
Very well done, many thanks!
I wish I could like this twice.
It is really impressive how he came up with this solution - how many concept he have to understand to garantee optimal static encoding.
I wonder how he can transpit particular codig for that message. And if we you modification during decoding, is there possibility to shring coding a little bit more?
Like when we has message ABDCBBB, first letter we encode only A, then AB, then ABD and so on (last letter is encoding as we would encode ABDCBBB). Decoder then has to start from the end and know how many times each letter should apper.
Nice, very clear on the logic.
Thank you for keeping it precise but also intuitive!!!
This was so cool. At 18:03 i was like "oooooooooooooooooh"
Amazing video. You are very good at this.
Congratulations for posting an excellent info content!
Thanks again fantastic video.
17:37 with the ShannonFano code I thought wow this thing is just guessing, it will way too often give B(0.35) two bits, although 1 bit would be better.
Try with an even more extreme distribution like this
D(0.14), E(0.14), A(0.15), C(0.17), B(0.4)
And it will still assign two bits to B. So this must be non optimal, just from the first impression.
24:29 yeah, this is it, this Huffman looks like an optimal recursive function, just intuitively right
Outstanding, as always.
This is really good ! Thank you ! I learned a lot.
Excellent explanation! Thanks!
Great explanation. I appreciate your efforts very much.
Amazing work
This is great! Amazing work
expandable is back
Thanks for sharing the manim scripts.
27:05 I think it would be more eye-pleasing if the codes were written in a monospace font.
Thanks for this awesome explanation !!!
I’m not too proud to admit that the first time I tried implementing Huffman Encoding, I just could not wrap my head around it. It didn’t matter how many times I read that Huffman was “Bottom up”, I tried to implement the algorithm “Top down”. That’s just the natural way we think about trees. As you might imagine, I crashed pretty quickly and had to walk through some examples in detail until I wrapped my head around the “Bottom up” nature.
wow, very well explained. great job sir
Yayyy, new videos, so exited
awesome! thank you very much for the explanation
Great as always!
Thank you for this video. Very informative !
This video reminds me a coursework that I had almost 20 years ago. I had to write an implementation of the Huffman Algorithm in Assembler. What I remember is that I had to make two passes over a file: the first to count frequencies and the second to encode the file. It didn't work for streams.
For one-pass compression, you need LZW compression (I think). Let me know if I'm wrong. Regards.
This is truly informative.
Great video, thanks!
The illustrations are so clean, and the animation really makes it intuitive. Really similar to 3B1B’s style, were you inspired by his videos?
Quite certain he is using 3B1B's animation library
Awesome video !!!
great video man
content on your channel is awesome !!!
Thanks fro the great explanation! However, one thing isn't clear for me. On 15:29 you say that the average length for the symbol is 1.75 (exactly as the entropy). But the codes are 1, 01, 000 and 001, so the total length is 9, and in average it turns out to be 9/4=2.25 (which is not 1.75 as stated) Am I getting something wrong?