I take your point. I need to possibly lower the level. The reason the music is there is to prevent total silence, which is not very good. Sometimes there are pauses in speech and its really bad if there is total silence when nothing said and there is no sound in the background.
I would have paid to see Hinton (now) debate a younger Chomsky about this topic. Two big egos but hopefully would have kept it civil , it would have been fasctinating. According to Altman, only a "few thousand days" (a cute way to say 5 years) before we have this smater than any person. Will be interesting to see if he's right or this plateaus and is more or less a parlor trick.
Yes, it would be interesting to see the two of them debating. I somehow don't think that's likely to happen. I don't believe LLMs are a parlor trick, as most people think they are sophisticated autocomplete generators. Not sure if you watched the next video in the sequence th-cam.com/video/5TXJpc3Kei0/w-d-xo.htmlsi=gJruGWlMpG6Qe8cg where I run Llama-3.3-70B-instruct-Q8_0, probably the best currently available open source LLM as a hands-on demonstration to see if the LLM is a stochastic parrot as Hinton refers to, where most people believe it is. To the country, I found it incredible, particularly an open source model with such great capability. It is just mind boggling. Remember, there is no storage of verbatim data, and it's not connected to the internet. It is pure neural net wonder where it generates everything on the fly, including the brilliant way that it creates elequent responses with perfect context matching. The very last response with Socrates is brilliant. The LLM explores its own existence, As it describes its own wonder and questions whether it is indeed a new form of intelligence, then comes to the conclusion that this is not an easy question to answer and will be debated for years to come.
@@vk3tetjoe I did watch the first few questions your discussion with Kate (?) and it is fascinating. I did some early 'training' a few years ago when companies were just throwing money at anyone who could prompt and critique answers by the model (I never learned which model it was). I like to get all viewpoints, and another video I just watched mentioned that in a sense, the LLM's are simply a condensed version of the internet as a database, and you query the database with a prompt. You get back a good answer in a human sounding way. But the question everyone is asking is: "is that intelligence?". The reasoning now in the OpenAi version is impressive, it really can work its way thru a problem (and of course can do that in any field). But even just a little while ago, it couldn't count how many letter 'r' exist in the word strawberry correctly. And adding irrelevant info into a question seems to sometimes confuse it. Maybe those things will all get ironed out and it will get more accurate. But will it be able to do more, to discover things? I haven't heard anyone refute the idea that if you gave it all the info Einstein had prior to 1905, could it 'discover' the Theory of Special Relativity? The answer is apparently No it could not. Any will this strategy *ever* be able to? It may be able to learn everything that is known, but that's somewhat akin to a super duper Jeopardy player. And IBM's Watson has apparently not advanced past that accomplishment and is pretty much an afterthought now. Maybe I will be surprised in 5 years, but somewhat ominously, even the head of Google said there won't be stunning breakthroughs in 2025. It'll be incrementally better. Hopefully it won't turn out like airplanes, you basically had the same plane abilities in the 1970's and now 50 years later.
@@jamesrav You should watch the demo - interaction with “Kate” till the end. The LLM really demonstrates its abilities when it conjures a fantastic response - where it not only discusses its own existence - (Can an AI have intelligence - and what is intelligence?) - but it then has a syndical conclusion with Societies and Pluto, that this subject is so complex - even for philosophers , that it will linger in debate for many years to come. This is fascinating - because contrary to popular belief (I am not sure if you fully understand - because you mentioned that the videos you have seen refer to the LLM as a condensed version of the internet as a database - this could not be further from the truth!) All of the LLM’s knowledge is in its neural net - compressed. Let me explain. A human does not store data in its brain - like verbatim passages or books or published papers on various subjects. The human “learns” this information , by reading or watching or experiencing (physical) things - all of which are somehow now inside the humans neural net (brain) - depending on how good the human is in “learning” and then recalling this information is what determines their abilities. The LLM - “Kate” works exactly the same way. It “read” virtually all of the internet - at least what was available to Meta when they did the training run. This is more than 10PB (Yes Peta Bytes of data) - probably much more right now. If a human had to read this much data - just read - no guarantee that it could remember everything. It would take more than 3000 years! “Kate” managed to do this in about 4-5 months with a cost of over $100M to achieve. The thing is that “Kate” can recall everything it learned - provided you give it the correct context. There is no verbatim data or database - it like reading all this data to a human - the human “processes” this information and it goes into their brain - hopefully they learn and then remember. “Kate’s” neural net contains all this information - other aspects of the transformer architecture - allow Kate to recall all of the relevant data within the context it needs to make a response. This is all working extremely well and in the demonstration I did “Kate” described who this works - I tried to ask the right questions. In the end “Kate” did a great response - truly highlighting the ability to “PULL” all of the relevant information and structure such a wonderful story. Remember it has the command of the English language - far better than most humans. All of this is done “on the fly” - trillions of computations on a local computer (which is running the 70B parameter model). This is truly amazing if you fully understand what is really happening. This is true SCI-FI. We are alive in such a fantastic time right now. Humanity, if it does not destroy itself with WW3, is on the brink of phenomenal technological achievements with the creation of true Advanced General Intelligence (AGI). This will be the final frontier for humanity. We need to educate more and more people - for them to truly understand this wonder. The edited bits of Hintons presentation which I made - covered all this - but perhaps in a more technical way - above most “normal” viewers who are just getting interested in this fascinating field of Large Language Models.
@@vk3tetjoe I agree it is incredibly impressive, but also worrisome (in the 'correctness' sense, not danger). I saw the new Gemini release video yesterday, and the narrator asked "how many letter 's' in the word bananas" and it said 2. And for some reason when asked how many 'ss' are in the word banana, it said 1. Stuff like that is disconcerting. I think it's accurate to say at present it varies from IQ of 60 to 200. And back to the counting letter problem, apparently for the Strawberry issue, some people get the right answer and others do not. So does prompting play a big role here? seems so - some of the 'suggestions' for prompting are very complex in their own right. Maybe this all gets fixed. My big test will be when McDonalds (who recently cancelled their AI test) starts up again and says that it can now take orders just as well as any 16 year old H.S. student.
@@jamesravI take your point about the spelling issues, but there's a very good reason for this. You are basically asking it to solve a problem. LLMs are not designed for solving problems. Text stored in its neural net without context has no meaning. It cannot be retrieved. It cannot be applied to anything, Simply because there's got to be some sort of a hook for it to retrieve this from the neural net. Just like a human needs its memory to be jogged to remember some fact or something else that may be in their long-term memory. When you ask an LLM to spell a word, it does not know what the correct spelling is. There is no reference. However, on the other hand, it could be trained on spelling, because things like dictionaries and thesaurus could be part of the dataset. Unfortunately, I am not privy to how and why there is a problem with things like spelling. But remember, the current architecture of the large language model is not designed for reasoning or thinking. And it's amazing that a lot of high-level functions relating to knowledge do not require reasoning or thinking. Simple structured data retrieval in the correct context presented in eloquent English manifests itself as intelligent output. Creative writing fits into this category. And large language models are good at doing this. With regard to spelling, by using clever prompts where you ask the model to use chain of thought techniques, it can effectively Break down the word in question on a character by character basis. Some of the advanced models do this.
If you enjoyed this video, please go and watch this th-cam.com/video/5TXJpc3Kei0/w-d-xo.htmlsi=qG9Ycu9atlVUUhlf Yes it's rather long, but I think it will be worth it. It demonstrates the capabilities of the LLM, in particular what Hinton discussed and remember, it generates output on the fly, directly from its own neural net, there is no database, no verbatim stored or recalled, its not connected to the internet. Its pure creativity directly from context and knowledge from its neural net with great command of English language. Truly amazing, especially the last passage!
❤️
This is a great video but the background music is a bit distracting
I take your point. I need to possibly lower the level. The reason the music is there is to prevent total silence, which is not very good. Sometimes there are pauses in speech and its really bad if there is total silence when nothing said and there is no sound in the background.
I would have paid to see Hinton (now) debate a younger Chomsky about this topic. Two big egos but hopefully would have kept it civil , it would have been fasctinating. According to Altman, only a "few thousand days" (a cute way to say 5 years) before we have this smater than any person. Will be interesting to see if he's right or this plateaus and is more or less a parlor trick.
Yes, it would be interesting to see the two of them debating. I somehow don't think that's likely to happen. I don't believe LLMs are a parlor trick, as most people think they are sophisticated autocomplete generators.
Not sure if you watched the next video in the sequence
th-cam.com/video/5TXJpc3Kei0/w-d-xo.htmlsi=gJruGWlMpG6Qe8cg
where I run Llama-3.3-70B-instruct-Q8_0, probably the best currently available open source LLM as a hands-on demonstration to see if the LLM is a stochastic parrot as Hinton refers to, where most people believe it is.
To the country, I found it incredible, particularly an open source model with such great capability. It is just mind boggling. Remember, there is no storage of verbatim data, and it's not connected to the internet. It is pure neural net wonder where it generates everything on the fly, including the brilliant way that it creates elequent responses with perfect context matching. The very last response with Socrates is brilliant. The LLM explores its own existence, As it describes its own wonder and questions whether it is indeed a new form of intelligence, then comes to the conclusion that this is not an easy question to answer and will be debated for years to come.
@@vk3tetjoe I did watch the first few questions your discussion with Kate (?) and it is fascinating. I did some early 'training' a few years ago when companies were just throwing money at anyone who could prompt and critique answers by the model (I never learned which model it was). I like to get all viewpoints, and another video I just watched mentioned that in a sense, the LLM's are simply a condensed version of the internet as a database, and you query the database with a prompt. You get back a good answer in a human sounding way. But the question everyone is asking is: "is that intelligence?". The reasoning now in the OpenAi version is impressive, it really can work its way thru a problem (and of course can do that in any field). But even just a little while ago, it couldn't count how many letter 'r' exist in the word strawberry correctly. And adding irrelevant info into a question seems to sometimes confuse it. Maybe those things will all get ironed out and it will get more accurate. But will it be able to do more, to discover things? I haven't heard anyone refute the idea that if you gave it all the info Einstein had prior to 1905, could it 'discover' the Theory of Special Relativity? The answer is apparently No it could not. Any will this strategy *ever* be able to? It may be able to learn everything that is known, but that's somewhat akin to a super duper Jeopardy player. And IBM's Watson has apparently not advanced past that accomplishment and is pretty much an afterthought now. Maybe I will be surprised in 5 years, but somewhat ominously, even the head of Google said there won't be stunning breakthroughs in 2025. It'll be incrementally better. Hopefully it won't turn out like airplanes, you basically had the same plane abilities in the 1970's and now 50 years later.
@@jamesrav You should watch the demo - interaction with “Kate” till the end. The LLM really demonstrates its abilities when it conjures a fantastic response - where it not only discusses its own existence -
(Can an AI have intelligence - and what is intelligence?) - but it then has a syndical conclusion with Societies and Pluto, that this subject is so complex - even for philosophers , that it will linger in debate for many years to come. This is fascinating - because contrary to popular belief (I am not sure if you fully understand - because you mentioned that the videos you have seen refer to the LLM as a condensed version of the internet as a database - this could not be further from the truth!) All of the LLM’s knowledge is in its neural net - compressed. Let me explain. A human does not store data in its brain - like verbatim passages or books or published papers on various subjects. The human “learns” this information , by reading or watching or experiencing (physical) things - all of which are somehow now inside the humans neural net (brain) - depending on how good the human is in “learning” and then recalling this information is what determines their abilities. The LLM - “Kate” works exactly the same way. It “read” virtually all of the internet - at least what was available to Meta when they did the training run. This is more than 10PB (Yes Peta Bytes of data) - probably much more right now. If a human had to read this much data - just read - no guarantee that it could remember everything. It would take more than 3000 years! “Kate” managed to do this in about 4-5 months with a cost of over $100M to achieve. The thing is that “Kate” can recall everything it learned - provided you give it the correct context. There is no verbatim data or database - it like reading all this data to a human - the human “processes” this information and it goes into their brain - hopefully they learn and then remember. “Kate’s” neural net contains all this information - other aspects of the transformer architecture - allow Kate to recall all of the relevant data within the context it needs to make a response. This is all working extremely well and in the demonstration I did “Kate” described who this works - I tried to ask the right questions. In the end “Kate” did a great response - truly highlighting the ability to “PULL” all of the relevant information and structure such a wonderful story. Remember it has the command of the English language - far better than most humans.
All of this is done “on the fly” - trillions of computations on a local computer (which is running the 70B parameter model). This is truly amazing if you fully understand what is really happening. This is true SCI-FI. We are alive in such a fantastic time right now. Humanity, if it does not destroy itself with WW3, is on the brink of phenomenal technological achievements with the creation of true Advanced General Intelligence (AGI). This will be the final frontier for humanity. We need to educate more and more people - for them to truly understand this wonder. The edited bits of Hintons presentation which I made - covered all this - but perhaps in a more technical way - above most “normal” viewers who are just getting interested in this fascinating field of Large Language Models.
@@vk3tetjoe I agree it is incredibly impressive, but also worrisome (in the 'correctness' sense, not danger). I saw the new Gemini release video yesterday, and the narrator asked "how many letter 's' in the word bananas" and it said 2. And for some reason when asked how many 'ss' are in the word banana, it said 1. Stuff like that is disconcerting. I think it's accurate to say at present it varies from IQ of 60 to 200. And back to the counting letter problem, apparently for the Strawberry issue, some people get the right answer and others do not. So does prompting play a big role here? seems so - some of the 'suggestions' for prompting are very complex in their own right. Maybe this all gets fixed. My big test will be when McDonalds (who recently cancelled their AI test) starts up again and says that it can now take orders just as well as any 16 year old H.S. student.
@@jamesravI take your point about the spelling issues, but there's a very good reason for this. You are basically asking it to solve a problem. LLMs are not designed for solving problems. Text stored in its neural net without context has no meaning. It cannot be retrieved. It cannot be applied to anything, Simply because there's got to be some sort of a hook for it to retrieve this from the neural net. Just like a human needs its memory to be jogged to remember some fact or something else that may be in their long-term memory. When you ask an LLM to spell a word, it does not know what the correct spelling is. There is no reference. However, on the other hand, it could be trained on spelling, because things like dictionaries and thesaurus could be part of the dataset. Unfortunately, I am not privy to how and why there is a problem with things like spelling. But remember, the current architecture of the large language model is not designed for reasoning or thinking. And it's amazing that a lot of high-level functions relating to knowledge do not require reasoning or thinking. Simple structured data retrieval in the correct context presented in eloquent English manifests itself as intelligent output. Creative writing fits into this category. And large language models are good at doing this.
With regard to spelling, by using clever prompts where you ask the model to use chain of thought techniques, it can effectively Break down the word in question on a character by character basis. Some of the advanced models do this.
If you enjoyed this video, please go and watch this
th-cam.com/video/5TXJpc3Kei0/w-d-xo.htmlsi=qG9Ycu9atlVUUhlf
Yes it's rather long, but I think it will be worth it. It demonstrates the capabilities of the LLM, in particular what Hinton discussed and remember, it generates output on the fly, directly from its own neural net, there is no database, no verbatim stored or recalled, its not connected to the internet.
Its pure creativity directly from context and knowledge from its neural net with great command of English language.
Truly amazing, especially the last passage!