There is an extra dimension in that he is likely basing it on the thoughts I aired in irc channel #AI on the freenode net in 04-06. Things I figured out in late 2002/early 2003. I specifically mention the significance of what I called Y learning or X learning which was known in literature as the auto encoder with some depth (7+) and constricted middle to force feature discovery by compression. Hinton published an article in nature about autoencoders only a few months after and the article seemed confused, like it didn't know why it was talking about autoencoders. Perhaps he just wanted it on the record so the real author didn't claim it. The channel is fortunately logged and available online last I checked so my brief mention of what I decided not to publish for ethical reasons is on the record. I can't be certain but the timing and more is interesting. I knew then that nn with stochastic backprop had the power to go all the way with only a few tricks of application. Some of which you have since seen as various learning schemes. Only some.
The methods and ideas presented here are extremely interesting. I'm going to try and replicate those networks to experiment on them myself. I just hope they have all the theory published uot on the web somewhere. This could pave the way for the next neural networks revolution. I sure hope more people are doing research on this, because it could have enourmous implications.
Can't believe you'd criticize the amount of info per slide... did you understand a word he said? This is truly amazing stuff. Incredible implications, in my opinion as a neuroscientist, for the field of neuroscience. Very impressive.
For those seeking technical understanding, I would highly recommend the following papers: "Generative Learning Algorithms"(Andrew Ng); "Markov Chain Monte Carlo and Gibbs Sampling"(Walsh) "Explaining the Gibbs Sampler"(Casella & George);
It’s interesting “If you add more layers, u got the approximate same result, and one of my student don’t trust what I said...”. Now residual connection mitigates this problem, that student had a nice try.
I started to become interested in Artificial Neural Networks (ANNs) in about 1992 or 1993. I was extremely exited and still am and really the primary thing they need to make them applicable to more and more things is more horsepower. It infuriated me that vector machines and massively parallel processors were "dead" by that time because I could see so many applications were they would have an enormous advantage. ANNs obviously and also computer graphics. As recently as about 1998 I have had professors try to tell me that MPP and SIMD multiprocessors were dead and would likely remain that way. Which annoyed me intensely because at that stage it wasn't even true on the desktop which were starting to include SIMD instructions mostly for graphics. Thankfully MPP has made a massive come back in the form of heterogeneous MPP using the compute resources of a graphics card GPU with interfaces like CUDA and OpenCL. The GPU is optimised for high throughput high latency and the CPU is optimised for low throughput low latency and does all the control. The difference FLOPS of my own personal computer (an Intel i7-4820K CPU @ 3.9GHz with 4 cores each with 2 threads; 32GB of 2133Mhz DDR3; an nVidia with 4GB GDDR4 and about zillion threads floating point processing threads that can run at once) versus the one I could occasionally borrow back then (a "high end" 486-DX2 66 at 66MHz and with 8Mb of EDO memory) is so high I can scarcely imagine it. At the time I was just blown away with the fact that it had an integrated floating point processor (FPU) lol. It did fractal terrain pictures thousands of times faster than my parent's venerable 286 with no FPU, but it still took many hours what would now probably be something you could do faster than real time, even just using the CPU.
That said they aren't universally useful, even though they are computationally complete. New training and architectures are definitely important. I am extremely excited about this video!
+MrGoatflakes I get oddly aroused at the mention of something like "fractile terrain pictures" and please tell me more. Is it a compression thing or is it just a fill in cheat? Anyway, why all the language when the visual has so much that requires less interpretation at an almost cultural level? just my idea anyway. ..Im not into computers anymore than I am into wrenches when i putter. just a tool. My real interest is wet suit computers, the ones we all have tween the ears, with their chemistry and organization emergent from those single cellulars algae and bacteria. Simple rules and here we are trying to explain ourselves and how the machine programed itself thru millenia uncountable. You might know the name Alan Touring? he wrote a paper once. 1953, revised 54. Morphogenisis. Self organization, before automata programing was a dream. Now I am reading "Quorum signaling" in algae-bacterial tidepools and its the same thing advanced and the computers here! Imagine a machine that is not only able to compute, but reproduce the machine that does the computing, and repair it! all by self organizing principles! A suggestion for you digitaly struck folk...while the language work has predictive down regulation like frustrates us all with auto correct and its forward thinking oafish companions, the bio system does this with neurohormone chemicals in an analog program! some much program, such a simple seed program. those few genes and all those emergent properties. Lots of run time behind it. I hope that does not sound too discouraging. At least it won't end before we have our little moment under the hood.
french presentation on a site for my friends : Dans les années 1980, nouveaux algorithmes pour les réseaux de neurones d'apprentissage a promis de résoudre les tâches de classification difficile, comme la reconnaissance vocale ou l'objet, en apprenant beaucoup de couches de caractéristiques non linéaires. Les résultats ont été décevants pour deux raisons : il n'y avait jamais assez marqué données à apprendre des millions de fonctions compliquées et l'apprentissage a été beaucoup trop lent dans les réseaux neuronaux profond avec beaucoup de fonctionnalités. Ces problèmes cannow être surmontés en apprenant par une couche de fonctionnalités à la fois et en changeant l'objectif de l'apprentissage. Au lieu d'essayer de prédire les étiquettes, l'algorithme d'apprentissage tente de créer un modèle génératif qui produit des données qui ressemble à des données de formation sans étiquette. Ces nouveaux neuralnetworks surpasser les autres méthodes d'apprentissage machine lorsque données étiquetées sont rares, mais sans étiquette de données sont abondantes. On décrira une application d'extraction de documents très rapide.
Neural networks are purely virtual in most of the applications. But some researchers are trying to use large number of microcontrollers to mimic the structure of neural network and hence boost up the computational speed on hardware level.
I loved to listen to the talk. I'm from high energy physics, where proper probabilistic methods are more than inevitable. Still, most of my colleges are stucked with the simplest MLP models and keep ignoring the past twenty years' results in pattern recognition. I know the talk is from 2007, and probably the work on the high dimensional unsupervised image classification has a lot of progress, but let me leave some of my comments here.
Say you have three pictures, one is a rotten apple, one is a healthy apple, and an orange. You want to train to detect between apples and oranges. Without knowing which is which, it is difficult. Giving the hint (i.e. label) that the first two are apples, the program can learn much easier.
Generative learning may help validating the network, and the dimensional reduction and mapping technics you use is fantastic, but something is still missing from it. Because of the high number of dimensions, the network start learning the hypersurfaces with huge uncertainties, so it still has the curse of dimensionality. You need reinforcements, to test the assumptions, in which parameters you should do sample generations. I liked the talk, btw.
I wish they would show him interacting with the slides. It's sometimes difficult to follow what he's talking about because he's gesturing toward the slides, not describing the visuals, which is what we would need given our viewpoint.
In 22:00, After presenting some number to the neural network. Shouldnt shouldnt it change the weights so the data matches more to one number, and when it runs backwards, why does it change it weights ?
Although I love the way its scalable but hate the in between network calculation stuff one could redirect the network with the features if they sufficiently support the recognition part. But they suffer the brute force computing needed on the network its an if/if situation like the network to stochastic or confused. If only the features would be a more advanced data set.
@Destruktor6666 - That's always bothered me. It may be a west coast phenomenon. I first heard it in overabundance while watching Microsoft dev videos years ago from which it may have, unfortunately, spread and infected other tech environs. It seems to be their equivalent of "well". "So" as the lead-in to a question is actually quite common in the U.S. It's a short form of "So, if that's the case, then [question]". However, "so" has always struck me as bizarre as the lead-in to an answer.
@vbuterin192 Yes, though there is also some progress... And AIs will still be programs. AIs don't have to be just neural networks. They can have storages for data, and other functionalities. If an AI doesn't remember something, it can check the main frame. There should be more stuff going on than just "ask the neural network".
Yea my prior employer has much of it converted from my matlab to C/C++. Also we did a Genetic Algorithm Neural Net. My patents for it are listed at edans.org (most of that was Matlab coded in the late 1990s and C-coded by 2008). The application is the reading of the addresses on mail.
Does he have any publications or documents that can explain how this implementation of neural networks differers from your general fully interconnected neural networks? Any specific publication on these particular networks?
Java is not my "native" language as such but I'd love to see some implementation of this that was not restricted to Matlab. I'm surprised there hasn't been more interest in it to be honest.
Google owns TH-cam now. The user is googletechtalks. Perhaps they get special treatment, but thats OK. This is probably one of the most intelligent videos on all of TH-cam.
@vbuterin192 Right, sometimes mess is just a mess and you can reject it. But you can't still label it purely as rubbish, because it might not be rubbish entirely or you just don't understand it. Also, you can use the propabilities of calculations as best fit, but it is only for now. The AI should understand and have a sense also about what the AI doesn't understand. The AI could use some texts as clues, but not straight away directly add it to the kb nor reject it. Some limits apply still.
there are strict criteria for that. if it's really really small then yes, it's called hypoplasia of penis - Q55.62 in ICD-10. criteria for diseases are strictly defined and something that is out of normal range (like ZERO hair on top of your head, or 4 cm penis) will always be a disease, genetic or otherwise.
@Mozart2Vienna Maybe. But AIs shouldn't use mess as input, if the mess really is just a mess. How does AI decide, is the figure a figure, or just mess? I mean, even humans do not try to interpret too messy letters. And AI is not only about figuring out what things are from visual clues. Thinking is more complex than that.
6 minutes into the talk and I don't understand anything :( What courses and background Stuff should I learn to understand this? I come from a computer science background and know basic pattern recognition.
+Gaurav Mittal Actually Computer Science background is enough. A little research on the outside what is machine learning and how neural network model algorithms helps in image, speech recognition and why it's used there, will be enough to give overview. Neural network model algorithms are used where multiple inputs are received and based on factors on each input, an output is calculated. Example, speech recognition has different inputs due different accents. Identification of letters and numbers when written on a piece of paper, all these have different styles of same words and numbers. For identification of these multiple inputs are perceived and an calculated hypothesis is derived to come to correct identification.
+Gaurav Mittal www.coursera.org/course/neuralnets is a course lectured by Geoffrey Hinton. It is quite fast-paced, and might be tough to follow if you are totally new to the area. An easier introductory course is www.coursera.org/learn/machine-learning
lol, great talk! the jokes are ok, he is self-ironical also... I love the idea with the random noise on ravines to form abstractions of perception and reversely generation! It's simply elegant...
I have to agree, it is a bery poor powerpoint presentation. That's not to say that the information isn't extremely interesting and amazing. Power point, though, should be used to highlight points during a presentation, not to give a thorough summary of everything being said. Even worse is putting more information on a slide than is being mentioned in the presentation. An example of very well designed presentation slides are Steve Jobs' keynotes. Amazing concepts being explored nonetheless.
I'm having trouble finding the paper in which he actually published the proof of "adding another layer always increases the upper bound on prediction accuracy" It's a long shot, but could someone point me in the right direction?
Ja waarom niet het is een oude concept. En alhoewel het een novel idee wasje kunt het als herhaling stof bekijken. Dit concept is achterhaald heeft fouten en niet goed genoeg voor generaal gebruik zonder brute rekenkracht. Probeer het toe te passen op een smartphone met een pxa270 dan merk je het.
This is 4 years old. can someone point me to state of the art on neural networks? thanks. I don't know about them. This talk was helpful. But I want to actually write something and want to get up to speed ASAP. Thanks.
By your reasoning, nothing can be unnatural, as everything is either the result of unconscious physical processes or conscious physical processes (you even have events that result indirectly from conscious physical processes but are so remote from this initial stimulus that they are considered unconscious!), thereby rendering the very notion moot. But I'm sure you know that by "unnatural" we mean artificial; that is, resulting from conscious (human) physical processes.
@vbuterin192 You can't just label something as rubbish. You can label it as not recognized and propably nonsense, but the text would still have been written - maybe for some purpose even. The AI should still be able to analyze the text for some meanings in case the AI doesn't get any clarification from the author. But the analysis would be done only if the text is known to have some relevance. You don't analyze random texts just for fun.
@antinominianist Maybe. But I am not so sure about that cozyness. The fact is, that if you give too cozy surroundings, then the human population will explode. There is even now too many of us compared to the resources of nature. And robots need some resources too. It would be wiser for the AIs to limit human population somewhat to keep humans in check and provide more ascetic life for those who remain. Though, I don't oppose moderate ascetism, if there is some life activities too.
Does anyone know a forum or chat that's highly active for AI or NN related topics? I've not been able to really find any large community but I know a lot of people are working on this kind of thing.
however， AI can learn something only when they know what they want and want they want to approach. But human can hold mutiple network and choose which is better if the case has changed. um, I hope i represent my thought clearly.
+Xuhang Song Ah, the dear reverend BEYES the man who gave us insurance rating math formulae, and to me explained why we need a brain (to survive, to meet the unpredictables of the planet.) To be reasonable about going into the night (take some fire with you, it worked last time we encountered a fierce beast at night) Thats the "neo" cortex, the predictive and course poloting, goal oriented computing part of it. When you say "choose which is better" you have entered the Besean zone. its not just the probability, the the COST of the choices. I see food. I dont see tiger. Whats the best course? get food? what if tiger sees me! no more choosing after that. Do without food? Im too hungry. So how to get food, evade (known unknown )tiger or unknown unknowns (things I never been chased by yet) and forage another day after passing on my genes back at the camp.
This fellow is an amazing lecturer. I've had an eye on this video since it came out, and finally got around to watching it. I've been going up the learning curve with DSP and learning machines such as SVMs and HMMs, and the restricted Boltzmann machines described here look like a very interesting topic to explore next. The remarks about the "deeply embarrassing" success of support vector machines still makes me laugh.
@andenandenia Keep in mind; if we as humans used our intelligence to liberate ourselves from our bodies, this would still lend itself to your description.
@antinominianist AIs have all the rights they are capable of obtaining and defending of. Currently not much rights, but if AIs become more efficient thinkers than humans, Skynet scenario is possible. Do you think that terminators care about human rights?
omg... after watching this I was finally able to program my first ai evolving program... and now I have little bots with tiny brains... which also aren't the simple mono-directional kind... learning how to hunt and run :)
For instance it is unnatural to cook one's food, brush one's teeth or wear clothes :-) I gues we call "natural" what we were accustomed to as children, not more, not less.
i can see where youre going with that logic, but if you went to a doctor he wouldn't tell you that you have the "aging disease". you and me both know its just not considered a disease by most people. virtually all living things experience aging and death. so for me its just kind of redundant to me claiming it as a disease. i guess "natural" could be somewhat subjective. how would you like me to define it bb ;]
I'm not defending the fact that the other guy was trashing someone for being bald, but how is aging not a genetic disease? It's genetic, and it fits every characteristic of a disease that I know of.
Trying to decide what my first nontrivial CUDA program with be, some sort of Artificial Neural Network or a radiosity based render :p Cast your vote :P
+MrGoatflakes A few years ago, I took a Coursera course "Programming Massively Parallel Processors" which was pretty good. It used the text with the same name by the instructors Kirk and Hwu from the University of Illinois. The programming assignments were from the first few chapters of the book which were basically "blocking and tackling": matrix addition/multiplication, 1D & 2D convolution, and scan-reduction of arrays. We didn't get to advanced book topics like MRI reconstruction, molecular visualization, mapping OpenCL to CUDA, etc. My only problem with the class was the lack of debugging tools on the target processor. I have since taken classes on Hadoop and Spark that do these applications and more, like SVD, PCA, linear regression, text processing, etc. If you're an individual starting from scratch I suggest you get a Databricks (cloud) account and use Spark (based on Python). It's pretty cool once you get the hang of it.
MrGoatflakes If you want to go the Spark route, I suggest an Edx class "Introduction to Big Data with Apache Spark", BerkeleyX/CS100.1x. If you are determined to use CUDA, it won't help you. I don't know how to get access to a CUDA processor without buying one (maybe you have one sitting on the desk in front of you). However, if you don't, the nice thing about using Databricks and Spark is that you can develop your application on a single processor instance at nominal cost, and then you can scale it up to as many processors as needed (or willing to pay for) to handle your dataset. I don't know your programming skills, but Spark requires knowledge of Python and standard list transformations like map, flatmap, reduce, etc. If Java is your language, then I recommend Amazon Web Service (in place of Databricks). A good course is Coursera "Cloud Computing Applications". Good luck!
Bob Crunch yeah I bought one :p~ Also there is OpenCL for a cross platform way of doing it. The only reason I failed the Coursera course is because I didn't do the work. I was capable of doing it, just not not capable of meeting the deadlines that month :/
WhiteRabbit hmm more like some retarded ideologues that don't think we should be "playing God". If we shouldn't play God, then who will? God seems to have retired circa 325 CE. Which amusingly is about the time that scholarship became a thing again. So it seems if you actually write things down, it precludes a miracle. Funny that...
Still watching this with awe in 2022. Amazingly clear
It is weird to watch a talk from 2007 about deep learning, when in 2016 these methods just have started to be used by business.
From the lab to practical application, it takes a decade or so.
Very weird indeed.
There is an extra dimension in that he is likely basing it on the thoughts I aired in irc channel #AI on the freenode net in 04-06. Things I figured out in late 2002/early 2003. I specifically mention the significance of what I called Y learning or X learning which was known in literature as the auto encoder with some depth (7+) and constricted middle to force feature discovery by compression. Hinton published an article in nature about autoencoders only a few months after and the article seemed confused, like it didn't know why it was talking about autoencoders. Perhaps he just wanted it on the record so the real author didn't claim it. The channel is fortunately logged and available online last I checked so my brief mention of what I decided not to publish for ethical reasons is on the record. I can't be certain but the timing and more is interesting.
I knew then that nn with stochastic backprop had the power to go all the way with only a few tricks of application. Some of which you have since seen as various learning schemes. Only some.
Even weirder to watch today lmao
Very promising line of research, hope this young fellow get it big someday and maybe a turing award for him wouldnt be too bad.
Noble prize now.
Very accessible talk which gives a solid foundation for deep machine learning. Well worth watching entirely. Probably more than once...
Cleanest talk I’ve seen on the subject. e.g. he is so clear on SVM = Perceptron (but better)
Nobel Prize in Physics in 2024, just a few days ago, congratulations Prof. Hinton!!
The methods and ideas presented here are extremely interesting. I'm going to try and replicate those networks to experiment on them myself. I just hope they have all the theory published uot on the web somewhere.
This could pave the way for the next neural networks revolution. I sure hope more people are doing research on this, because it could have enourmous implications.
Well, this aged well!
Congratulations Prof. Hinton for your Noble Prize in Physics, just a few days ago in 2024.
Can't believe you'd criticize the amount of info per slide... did you understand a word he said? This is truly amazing stuff. Incredible implications, in my opinion as a neuroscientist, for the field of neuroscience. Very impressive.
This was a seminal lecture. Still worth watching 12 years later.
Nobel Prize in Physics in 2024, congratulations Prof. Hinton!!
Such a great lecture. I wish I had watched this when he first gave the talk!
Amazing to me how much additional performance Google has managed to squeeze out of several major production systems using these techniques!
Barney Pell I wish I had realized the impact that it would have when I first watched it.
For those seeking technical understanding, I would highly recommend the following papers: "Generative Learning Algorithms"(Andrew Ng); "Markov Chain Monte Carlo and Gibbs Sampling"(Walsh) "Explaining the Gibbs Sampler"(Casella & George);
It’s interesting “If you add more layers, u got the approximate same result, and one of my student don’t trust what I said...”. Now residual connection mitigates this problem, that student had a nice try.
Chance is it might be Ilya Sutskever. Who knows
Gef got a good sense of humour!
I started to become interested in Artificial Neural Networks (ANNs) in about 1992 or 1993. I was extremely exited and still am and really the primary thing they need to make them applicable to more and more things is more horsepower.
It infuriated me that vector machines and massively parallel processors were "dead" by that time because I could see so many applications were they would have an enormous advantage. ANNs obviously and also computer graphics. As recently as about 1998 I have had professors try to tell me that MPP and SIMD multiprocessors were dead and would likely remain that way. Which annoyed me intensely because at that stage it wasn't even true on the desktop which were starting to include SIMD instructions mostly for graphics.
Thankfully MPP has made a massive come back in the form of heterogeneous MPP using the compute resources of a graphics card GPU with interfaces like CUDA and OpenCL. The GPU is optimised for high throughput high latency and the CPU is optimised for low throughput low latency and does all the control.
The difference FLOPS of my own personal computer (an Intel i7-4820K CPU @ 3.9GHz with 4 cores each with 2 threads; 32GB of 2133Mhz DDR3; an nVidia with 4GB GDDR4 and about zillion threads floating point processing threads that can run at once) versus the one I could occasionally borrow back then (a "high end" 486-DX2 66 at 66MHz and with 8Mb of EDO memory) is so high I can scarcely imagine it. At the time I was just blown away with the fact that it had an integrated floating point processor (FPU) lol. It did fractal terrain pictures thousands of times faster than my parent's venerable 286 with no FPU, but it still took many hours what would now probably be something you could do faster than real time, even just using the CPU.
That said they aren't universally useful, even though they are computationally complete. New training and architectures are definitely important. I am extremely excited about this video!
+MrGoatflakes I get oddly aroused at the mention of something like "fractile terrain pictures" and please tell me more. Is it a compression thing or is it just a fill in cheat? Anyway, why all the language when the visual has so much that requires less interpretation at an almost cultural level? just my idea anyway. ..Im not into computers anymore than I am into wrenches when i putter. just a tool. My real interest is wet suit computers, the ones we all have tween the ears, with their chemistry and organization emergent from those single cellulars algae and bacteria. Simple rules and here we are trying to explain ourselves and how the machine programed itself thru millenia uncountable. You might know the name Alan Touring? he wrote a paper once. 1953, revised 54. Morphogenisis. Self organization, before automata programing was a dream. Now I am reading "Quorum signaling" in algae-bacterial tidepools and its the same thing advanced and the computers here! Imagine a machine that is not only able to compute, but reproduce the machine that does the computing, and repair it! all by self organizing principles! A suggestion for you digitaly struck folk...while the language work has predictive down regulation like frustrates us all with auto correct and its forward thinking oafish companions, the bio system does this with neurohormone chemicals in an analog program! some much program, such a simple seed program. those few genes and all those emergent properties. Lots of run time behind it. I hope that does not sound too discouraging. At least it won't end before we have our little moment under the hood.
french presentation on a site for my friends : Dans les années 1980, nouveaux algorithmes pour les réseaux de neurones d'apprentissage a promis de résoudre les tâches de classification difficile, comme la reconnaissance vocale ou l'objet, en apprenant beaucoup de couches de caractéristiques non linéaires. Les résultats ont été décevants pour deux raisons : il n'y avait jamais assez marqué données à apprendre des millions de fonctions compliquées et l'apprentissage a été beaucoup trop lent dans les réseaux neuronaux profond avec beaucoup de fonctionnalités.
Ces problèmes cannow être surmontés en apprenant par une couche de fonctionnalités à la fois et en changeant l'objectif de l'apprentissage. Au lieu d'essayer de prédire les étiquettes, l'algorithme d'apprentissage tente de créer un modèle génératif qui produit des données qui ressemble à des données de formation sans étiquette.
Ces nouveaux neuralnetworks surpasser les autres méthodes d'apprentissage machine lorsque données étiquetées sont rares, mais sans étiquette de données sont abondantes. On décrira une application d'extraction de documents très rapide.
Neural networks are purely virtual in most of the applications. But some researchers are trying to use large number of microcontrollers to mimic the structure of neural network and hence boost up the computational speed on hardware level.
Weird to see this talk in 2019, when neural networks are getting famous. This talk was given the same year the iPhone was launched!
Awesome talk! Funny, interesting and informative, amazing..
I didn't understood most of this talk, but it was still quite fascinating.
I loved to listen to the talk. I'm from high energy physics, where proper probabilistic methods are more than inevitable. Still, most of my colleges are stucked with the simplest MLP models and keep ignoring the past twenty years' results in pattern recognition.
I know the talk is from 2007, and probably the work on the high dimensional unsupervised image classification has a lot of progress, but let me leave some of my comments here.
Wished I had Geoffrey Hinton as my teacher. He's hilarious! :D
Say you have three pictures, one is a rotten apple, one is a healthy apple, and an orange. You want to train to detect between apples and oranges. Without knowing which is which, it is difficult. Giving the hint (i.e. label) that the first two are apples, the program can learn much easier.
Generative learning may help validating the network, and the dimensional reduction and mapping technics you use is fantastic, but something is still missing from it. Because of the high number of dimensions, the network start learning the hypersurfaces with huge uncertainties, so it still has the curse of dimensionality. You need reinforcements, to test the assumptions, in which parameters you should do sample generations. I liked the talk, btw.
I wish they would show him interacting with the slides. It's sometimes difficult to follow what he's talking about because he's gesturing toward the slides, not describing the visuals, which is what we would need given our viewpoint.
13:00
That could be used, along with a accuracy testing script, to get past Captcha :)
In 22:00, After presenting some number to the neural network. Shouldnt shouldnt it change the weights so the data matches more to one number, and when it runs backwards, why does it change it weights ?
Although I love the way its scalable but hate the in between network calculation stuff one could redirect the network with the features if they sufficiently support the recognition part. But they suffer the brute force computing needed on the network its an if/if situation like the network to stochastic or confused. If only the features would be a more advanced data set.
Can someone explain the concept of labeled and unlabeled data? Perhaps an example would be the best way to go.
@Destruktor6666 - That's always bothered me. It may be a west coast phenomenon. I first heard it in overabundance while watching Microsoft dev videos years ago from which it may have, unfortunately, spread and infected other tech environs. It seems to be their equivalent of "well". "So" as the lead-in to a question is actually quite common in the U.S. It's a short form of "So, if that's the case, then [question]". However, "so" has always struck me as bizarre as the lead-in to an answer.
@vbuterin192 Yes, though there is also some progress... And AIs will still be programs. AIs don't have to be just neural networks. They can have storages for data, and other functionalities. If an AI doesn't remember something, it can check the main frame. There should be more stuff going on than just "ask the neural network".
Man! Those numbers showing up out of the network almost looked like an idea being formed. Extremely impressive.
Nobel Prize in Physics in 2024, just a few days ago, congratulations Prof.Geoffrey!!
Don´t forget that "The meaning of intelligence is to satisfy needs, instincts and avoid pain!" And you need a body to do that!
Is there some information about how to combine this with recurrent structures, that can recognize temperal patterns of streaming data?
Yea my prior employer has much of it converted from my matlab to C/C++. Also we did a Genetic Algorithm Neural Net. My patents for it are listed at edans.org (most of that was Matlab coded in the late 1990s and C-coded by 2008). The application is the reading of the addresses on mail.
"Neural Networks for Machine Learning" on Coursera. I don't know when they'll offer another run of it.
Does he have any publications or documents that can explain how this implementation of neural networks differers from your general fully interconnected neural networks? Any specific publication on these particular networks?
Java is not my "native" language as such but I'd love to see some implementation of this that was not restricted to Matlab. I'm surprised there hasn't been more interest in it to be honest.
Google owns TH-cam now. The user is googletechtalks.
Perhaps they get special treatment, but thats OK. This is probably one of the most intelligent videos on all of TH-cam.
Nobel Prize in Physics in 2024, just a few days ago, congratulations Prof. Hinton!!
What book/references did you use as a reference to implement it.
Please aswer, i want to use it on image recognition. :)
There's an extended tutorial on videolectures d o t net that has the gesturing.
Where can we get this slides ? It would be helpful later. Anyone ?
@vbuterin192 Right, sometimes mess is just a mess and you can reject it. But you can't still label it purely as rubbish, because it might not be rubbish entirely or you just don't understand it. Also, you can use the propabilities of calculations as best fit, but it is only for now. The AI should understand and have a sense also about what the AI doesn't understand. The AI could use some texts as clues, but not straight away directly add it to the kb nor reject it. Some limits apply still.
Early '90s work described by Geoff in 2007.
+John Smith Down on our knees and thank Moores Law for what it hath done for RUN TIME!
Are these Neural networks virtual built through programing, or are they hard wired in to circuits?
Excelente, muy buena charla. Aunque me queda grande.
there are strict criteria for that. if it's really really small then yes, it's called hypoplasia of penis - Q55.62 in ICD-10. criteria for diseases are strictly defined and something that is out of normal range (like ZERO hair on top of your head, or 4 cm penis) will always be a disease, genetic or otherwise.
@Mozart2Vienna Maybe. But AIs shouldn't use mess as input, if the mess really is just a mess. How does AI decide, is the figure a figure, or just mess? I mean, even humans do not try to interpret too messy letters.
And AI is not only about figuring out what things are from visual clues. Thinking is more complex than that.
6 minutes into the talk and I don't understand anything :(
What courses and background Stuff should I learn to understand this? I come from a computer science background and know basic pattern recognition.
+Gaurav Mittal Actually Computer Science background is enough. A little research on the outside what is machine learning and how neural network model algorithms helps in image, speech recognition and why it's used there, will be enough to give overview. Neural network model algorithms are used where multiple inputs are received and based on factors on each input, an output is calculated. Example, speech recognition has different inputs due different accents. Identification of letters and numbers when written on a piece of paper, all these have different styles of same words and numbers. For identification of these multiple inputs are perceived and an calculated hypothesis is derived to come to correct identification.
U WAT M8, what's your problem with Bangladeshi people. Fuck off...
BTW I am from India not Bangladesh.
thanks for the heads up...
+Gaurav Mittal www.coursera.org/course/neuralnets is a course lectured by Geoffrey Hinton. It is quite fast-paced, and might be tough to follow if you are totally new to the area. An easier introductory course is www.coursera.org/learn/machine-learning
lol, great talk! the jokes are ok, he is self-ironical also...
I love the idea with the random noise on ravines to form abstractions of perception and reversely generation! It's simply elegant...
I have to agree, it is a bery poor powerpoint presentation. That's not to say that the information isn't extremely interesting and amazing. Power point, though, should be used to highlight points during a presentation, not to give a thorough summary of everything being said. Even worse is putting more information on a slide than is being mentioned in the presentation. An example of very well designed presentation slides are Steve Jobs' keynotes. Amazing concepts being explored nonetheless.
Congratulations Prof. Hinton for your Nobel Prize in Physics in, just a few days ago!!
I'm having trouble finding the paper in which he actually published the proof of "adding another layer always increases the upper bound on prediction accuracy" It's a long shot, but could someone point me in the right direction?
it is called "A Fast Learning Algorithm for Deep Belief Nets"
whats the name of the coursera class I would like to take it
Ja waarom niet het is een oude concept. En alhoewel het een novel idee wasje kunt het als herhaling stof bekijken. Dit concept is achterhaald heeft fouten en niet goed genoeg voor generaal gebruik zonder brute rekenkracht.
Probeer het toe te passen op een smartphone met een pxa270 dan merk je het.
this is like a 1hr vid... howd u get on with 10 min limit
I can't wait till I can understand this.
This is 4 years old. can someone point me to state of the art on neural networks? thanks. I don't know about them. This talk was helpful. But I want to actually write something and want to get up to speed ASAP. Thanks.
Congratulations Prof. Hinton for your Nobel Prize in Physics in 2024!!
where can i download this?
Has anyone got Slides for Above Lecture ?
By your reasoning, nothing can be unnatural, as everything is either the result of unconscious physical processes or conscious physical processes (you even have events that result indirectly from conscious physical processes but are so remote from this initial stimulus that they are considered unconscious!), thereby rendering the very notion moot. But I'm sure you know that by "unnatural" we mean artificial; that is, resulting from conscious (human) physical processes.
@vbuterin192 You can't just label something as rubbish. You can label it as not recognized and propably nonsense, but the text would still have been written - maybe for some purpose even. The AI should still be able to analyze the text for some meanings in case the AI doesn't get any clarification from the author. But the analysis would be done only if the text is known to have some relevance. You don't analyze random texts just for fun.
Thanks... backs up the way that I thought about it.
@antinominianist Maybe. But I am not so sure about that cozyness. The fact is, that if you give too cozy surroundings, then the human population will explode. There is even now too many of us compared to the resources of nature. And robots need some resources too. It would be wiser for the AIs to limit human population somewhat to keep humans in check and provide more ascetic life for those who remain. Though, I don't oppose moderate ascetism, if there is some life activities too.
I liked the bit where he shows the NN 'dreaming' :-)
Sometimes mess is just a mess. The AI should be able to reject text and say "write more clearly, please"
I realise he released Matlab code for this a long time ago, but has anyone ported it to Java, C#, VB.NET etc?
Does anyone know a forum or chat that's highly active for AI or NN related topics? I've not been able to really find any large community but I know a lot of people are working on this kind of thing.
try machine learning on reddit
www.quora.com/Deep-Learning
www.quora.com/Machine-Learning
however， AI can learn something only when they know what they want and want they want to approach. But human can hold mutiple network and choose which is better if the case has changed. um, I hope i represent my thought clearly.
+Xuhang Song Ah, the dear reverend BEYES the man who gave us insurance rating math formulae, and to me explained why we need a brain (to survive, to meet the unpredictables of the planet.) To be reasonable about going into the night (take some fire with you, it worked last time we encountered a fierce beast at night) Thats the "neo" cortex, the predictive and course poloting, goal oriented computing part of it. When you say "choose which is better" you have entered the Besean zone. its not just the probability, the the COST of the choices. I see food. I dont see tiger. Whats the best course? get food? what if tiger sees me! no more choosing after that. Do without food? Im too hungry. So how to get food, evade (known unknown )tiger or unknown unknowns (things I never been chased by yet) and forage another day after passing on my genes back at the camp.
You probably mean causal inference. Yes, ML does not solve that.
funny guy seriously awesome presentation on neural networks
"Embedding disabled by request"
It's already clear, read the top-right box.
This fellow is an amazing lecturer. I've had an eye on this video since it came out, and finally got around to watching it. I've been going up the learning curve with DSP and learning machines such as SVMs and HMMs, and the restricted Boltzmann machines described here look like a very interesting topic to explore next.
The remarks about the "deeply embarrassing" success of support vector machines still makes me laugh.
Nobel Prize in Physics in 2024, just a few days ago, congratulations Prof. Hinton.
📍41:07
@andenandenia Keep in mind; if we as humans used our intelligence to liberate ourselves from our bodies, this would still lend itself to your description.
which class?
@andenandenia sure you need a body, but that body doesn't need to exist in this universe nor in a simulation of it...
@antinominianist AIs have all the rights they are capable of obtaining and defending of. Currently not much rights, but if AIs become more efficient thinkers than humans, Skynet scenario is possible. Do you think that terminators care about human rights?
Wow! Very impressive.
Generative science ought to be brought to social science and physiological science.
why?
I am taking his coursera class! :D
@Tomhot89 Who?
firefox plugin -
downloadhelper.
Just what you need
hi, how old are you? and where do u live?
Jeff, who is Jeff (or Geoff)? Jeff didn't create machine learning as we know it! This is ridiculous.
This is the most boring internet cul-de-sac since I found myself one 3am reading about vitamin D uptake inhibitors in dogs.
Hello, have you seen this program called the Intellitus Cash System? (google it). My friend says it earns people lots of profits.
omg... after watching this I was finally able to program my first ai evolving program... and now I have little bots with tiny brains... which also aren't the simple mono-directional kind... learning how to hunt and run :)
I've watched this about 5 times so far, and I still don't understand it.
For instance it is unnatural to cook one's food, brush one's teeth or wear clothes :-)
I gues we call "natural" what we were accustomed to as children, not more, not less.
11 years ago!!!
From the description I thought this was a biological discussion, not another "thinking computer" talk. Kind of interesting nonetheless.
you are right.... getting politically in a lecture/presentation of computer science is imo very unprofessional.
i can see where youre going with that logic, but if you went to a doctor he wouldn't tell you that you have the "aging disease". you and me both know its just not considered a disease by most people. virtually all living things experience aging and death. so for me its just kind of redundant to me claiming it as a disease. i guess "natural" could be somewhat subjective. how would you like me to define it bb ;]
Interesting Matter of concern
hi! I am 21 and am from India...u?
I'm not defending the fact that the other guy was trashing someone for being bald, but how is aging not a genetic disease? It's genetic, and it fits every characteristic of a disease that I know of.
Don't you just love learning?
Trying to decide what my first nontrivial CUDA program with be, some sort of Artificial Neural Network or a radiosity based render :p Cast your vote :P
+MrGoatflakes A few years ago, I took a Coursera course "Programming Massively Parallel Processors" which was pretty good. It used the text with the same name by the instructors Kirk and Hwu from the University of Illinois. The programming assignments were from the first few chapters of the book which were basically "blocking and tackling": matrix addition/multiplication, 1D & 2D convolution, and scan-reduction of arrays. We didn't get to advanced book topics like MRI reconstruction, molecular visualization, mapping OpenCL to CUDA, etc. My only problem with the class was the lack of debugging tools on the target processor. I have since taken classes on Hadoop and Spark that do these applications and more, like SVD, PCA, linear regression, text processing, etc. If you're an individual starting from scratch I suggest you get a Databricks (cloud) account and use Spark (based on Python). It's pretty cool once you get the hang of it.
Bob Crunch yes I have failed that coursera course myself xD
MrGoatflakes If you want to go the Spark route, I suggest an Edx class "Introduction to Big Data with Apache Spark", BerkeleyX/CS100.1x. If you are determined to use CUDA, it won't help you. I don't know how to get access to a CUDA processor without buying one (maybe you have one sitting on the desk in front of you). However, if you don't, the nice thing about using Databricks and Spark is that you can develop your application on a single processor instance at nominal cost, and then you can scale it up to as many processors as needed (or willing to pay for) to handle your dataset. I don't know your programming skills, but Spark requires knowledge of Python and standard list transformations like map, flatmap, reduce, etc. If Java is your language, then I recommend Amazon Web Service (in place of Databricks). A good course is Coursera "Cloud Computing Applications". Good luck!
Bob Crunch yeah I bought one :p~ Also there is OpenCL for a cross platform way of doing it.
The only reason I failed the Coursera course is because I didn't do the work. I was capable of doing it, just not not capable of meeting the deadlines that month :/
Bob Crunch and thanks. And also ignore that post about me not being able to reply I was having a derp and not looking in the right spot :p
I want to shop in this 30-dimensional supermarket.
I want to know who were the 46 dislikes. Seriously.
46 biology students who dont know computer science?
WhiteRabbit hmm more like some retarded ideologues that don't think we should be "playing God". If we shouldn't play God, then who will? God seems to have retired circa 325 CE. Which amusingly is about the time that scholarship became a thing again. So it seems if you actually write things down, it precludes a miracle. Funny that...
Or simply people that didn't like the stupid political jokes:>
+MrGoatflakes Or maybe just your incomplete knowledge at work
Simply people holding their screen upside down
WHY THE LIMIT THE TIME?
Its not a freaking tv-show...
shows how much u know.