Measuring information | Journey into information theory | Computer Science | Khan Academy
ฝัง
- เผยแพร่เมื่อ 28 พ.ค. 2024
- How can we quantify/measure an information source?
Watch the next lesson: www.khanacademy.org/computing...
Missed the previous lesson? www.khanacademy.org/computing...
Computer Science on Khan Academy: Learn select topics from computer science - algorithms (how we solve common problems in computer science and measure the efficiency of our solutions), cryptography (how we protect secret information), and information theory (how we encode and compress information).
About Khan Academy: Khan Academy is a nonprofit with a mission to provide a free, world-class education for anyone, anywhere. We believe learners of all ages should have unlimited access to free educational content they can master at their own pace. We use intelligent software, deep data analytics and intuitive user interfaces to help students and teachers around the world. Our resources cover preschool through early college education, including math, biology, chemistry, physics, economics, finance, history, grammar and more. We offer free personalized SAT test prep in partnership with the test developer, the College Board. Khan Academy has been translated into dozens of languages, and 100 million people use our platform worldwide every year. For more information, visit www.khanacademy.org, join us on Facebook or follow us on Twitter at @khanacademy. And remember, you can learn anything.
For free. For everyone. Forever. #YouCanLearnAnything
Subscribe to Khan Academy’s Computer Science channel: / channel
Subscribe to Khan Academy: th-cam.com/users/subscription_...
This is simply amazing.
৮এীা😊😂😂😅
Love this video. Demystifying, makes me feel a little less aptidudinally challenged.
Loved it so much! Thank you!!!
This is a very joyous moment for me as I finally understood the significance of something I always considered as unnecessary
what a gem of youtube this channel is.
previous lesson: th-cam.com/video/W-OskAegz5o/w-d-xo.html
next lesson: th-cam.com/video/Ws63I3F7Moc/w-d-xo.html
Very informative
This idea of communication being a mix of predictability and surprise reminds me of the double-slit experiment; while you can't predict where exactly a particle would land, the overall pattern could be -- the message space was definable.
It's like: so the infinite and the void were having a conversation and...
What if the perspective was wrong and forever decided that if you can define everything exactly then...
@@yodae7621 lmfaooo
3:03 No. You need a way to tell the receiver in advance if you're talking about cards or letters in advance (if no prior knowledge/arrangement can be assumed), and as your medium is limited to binary encoding and no other channels, you need transmit more bits for the "protocol" to determine what the following bits are all about, rendering "bits" useless as a measure for expressing meaning. You need additional bits to declare if the message/transmission is about coinflips, characters of a word or a poker hand, and it would be limited to these three, and if you would bill by the amount of bits, you either have to increase the charge for everybody by the bits that serve as indicator of the domain of the message, or come up with a scheme that favors some domains over others in terms of prices. By limiting the transmission medium to digital bit encoding, simple coin flips are always cheaper, but bits are in no way a useful measure for information, as transmitting coinflips over a medium that supports 26 states would make it just as expensive as transmitting letters, if no additional encoding scheme would be used, which then again needs to indicate it's presence somehow. A unit of base 26 could just as well "measure information" if that's the nature of the used medium despite being lossy/inaccurate for transmitting information that just needs to indicate yes/no states. Binary could be assumed as the most atomic instance of state difference indication/signalling of a hypothetical transmission medium, causing that you can only talk about yes/no changes on both sides of the wire, not related to coinflips or anything else, just yes/no state changes.
In essence, for charging, it all depends on your s at 8:03, so why would you assume s = 2, where there could be transmission media that are much more favorable to the customer sending letters or poker cards, which is just as well information as somebody transmitting coinflips, not to speak of the problem of interpreting the meaning of the bits without out of band communication or prior knowledge?
Regarding the first part of the statement, couldn't it be assumed that the friends between treehouse A and treehouse B know what type of information they are giving? In that case, wouldn't you be able to solve the issue by asking two questions of whether or not the following data is from a certain person. You'd only need to ask the question 1 more time for the next person in line, assuming that there are only 3 people. Or am I wrong? Also, what did you mean by 26 states? Do you mean what would happen if we were transmitting 26 coinflips?
Think about it. This has something to do with entropy, microscopic states and Boltzmann’s constant, which he thought of it around 1800s
Very nice explanation 👍
In short, information is based on # of possibilities. The more what we called "accurate" and "precise" information meaning eliminating other possible odds. The more loosely information, the higher chance it comes with more alternatives. Such theory known as Information theory by Claude Shannon and also many other researchers, especially Ralph Hartley. For every 1 bit of information, it equals to 2 possibilities.
When it comes to the pokerhand, you can only draw a card once, therefore leaving one less card each time you draw one. Making the amount of bits needed less, or am I totally wrong?
Right, also in poker the order of the cards doesn't matter.
Yes. The messagespace would be all possible subsets of 5 cards from 52, which has binomial(52,5) possibilities. We then get log_2( binomial(52,5) ) = 21.309... as the average number of questions.
Well, that went way over my head!😅🤪
If we know the the probability of the symbols then we can ask the questions in a different way (unlike is it less than m and dividing the alphabets into equal halves) like is it a letter "aei.." etc..
“*Naturally* use a log of base 2” lol
yeah also carved me a bit that one - cool way of putting it either way
😅
2.72ly use a log of base 2
Can somebody explain to me why the formula "2^number of questions = message space" is correct?
To find the correct information most efficiently, you need to ask questions that cut the set of possible answers in half (the set of answers has to be ordered), and thats a binary search, because it has two possible answers: yes or no, so the base is 2 since there's 2 possible outcomes, so in other words, you could think about 2^n°questions as doing an inverse binary search, that is, since the choosen number counts as 1 element in the set of all possible numbers, you need to do this operation times the average amount of questions)
Does Hartley's formula H = n log (s) works also for non-binary cases, if we would use different base of logarithm than 2?
Yes, I'm sure we can. Though I really think it depends on what type of information being presented. Every type of information can be represented in binary, however, it would probably be more difficult to do with a higher number of bases. For example, how would you represent whether a coin has been flipped or not with 3+ possible signals? I'm sure that there would be ways, but none as efficient as binary. I think the reason they use binary here is because it's the most general and versatile base for measuring information that's relatively consistent in efficiency.
Can anyone explain hartley formula H = n log (s) with deck of cards example.
ie what is the value of s , n and base of the log and how we got the values of s and n
this music is so fuckin information theory feel :D
what if the middle of the alphabet is sent........ how could we decide that......?
4:46. The Number of Questions will be at least 3. In the case of alphabets, Suppose we need to find G, then we can ask:
1. Is it less than N?
2. Is it less than G?
3. Is it greater than G?
No its beacuse we do not know if it is really g...cause everyte we rule out other possibilities we divide them again in half and we dont know when we have to stop doing it like in case on G
@@vasudevsharma7723 he's not talking about a general posibility, he's talking about a minimum number of questions. Suppose we need to find N. The following will happen:
1.Is it less than N? We recieve 0.
2.Is it greater than N? We recieve 0.
Then we deduce it's N. So, the minimum number of questions would be 2, not even 3. Even if we need to find a letter that requires a greater amount of questions, we would still need at least 2 questions to find it.
@@hdraresro4399 Yes but if it's not N you've just wasted 1 question, thus increasing the maximum amount of questions which we ask.
@@hdraresro4399 What you're describing is the case of a singular letter that happened to match the questions. You're talking about minimum number of questions for one option, they've created a system which optimizes all options to the fewest questions on average, which will be the far better communication system in the end.
Shouldn't it be at least 3 and at most 5 instead of at least 4 and at most 5?
whats with this super creepy music?
eitherway, it might be nice to explicitly state the idea that for any piece of information that is a part of a finite set, there exists a a set of questions, each with a finite set of answers, that can represent that piece of information
😅😅😅
Shout out kay sir japz
I don’t see what information adds to anything. Before, we called these things “properties.” If you were to strip a table of its properties: hardness, squareness (or its shape), brownness, etc., do you still have a table? It seems that the properties are as much a part of the table, as the table itself is. Information seems to be talking about the same thing. The properties of the table “convey” information to the environment. The only thing I see differently is that information theory is looking to add a bit unit to the world and give it measurements. I think it does nothing for science. Information is a platonic sort of concept, it doesn’t belong in physics. The bit unit was used for computers. This is a prevalent technology and so we project it onto the world. I do believe objects and properties and the world emanates information, but the information is a metaphysical entity, just as say universals are.
Lydia Riley I believe he means the rank of the letter starting from A
why at most 29? If the word is "FFFFFF" (f 6 times) wouldn't that take 30
i agree, i thought this too at the time
🕸️
কেমন হয়েছে
Did anyone else notice at 4:01 He asks, "Is it less than G?" Then he answers "Yes." As in yes it is less than G. But he then proceeds to use A-F opposed to G-M. He did not pay attention.
Ammm...the letters from A to F are less than G
G=7
F=6
6
Upload it in hindi
Fail