Thanks to our sponsor, Bright Data: Train your AI models with high-volume, high-quality web data through reliable pipelines, ready-to-use datasets, and scraping APIs. Learn more at brdta.com/aisearch Viewers who enjoyed this video also tend to like the following: You Don't Understand AI Until You Watch THIS th-cam.com/video/1aM1KYvl4Dw/w-d-xo.html These 5 AI Discoveries will Change the World Forever th-cam.com/video/fyVja-57EIs/w-d-xo.html The Insane Race for AI Humanoid Robots th-cam.com/video/90TMZ2fq9Gs/w-d-xo.html These new AI's can create & edit life th-cam.com/video/3K_LAGonsPU/w-d-xo.html
Ok there is so much issue in this video. neural network are fixed ? what does this even mean ? we just stop training them, it's a version checkpoint. we could train them continuously. they could learn even by themself without human by "reinforcement learning"... classic neural network can emule any partition or specialisation. this reflexion come from somone that does not really understand how it's works. Liquid neural network what you explained it's smething like a feature extractor, basically an encoder . feel like a lot of bullsht. spking neuron wtf you just discover an activation function... RNN with a simple Relu have the same behaviour. reaching a superior intelligenceby mimicking the brain, holy fk, i was waiting some quantum sht. you don't understand what your are talking about
@@alexamand2312 by "neural network are fixed"he means the current weights are based on last date of training, like different chatgpt versions...I haven't watched the whole video but the only issue I have is why the narrator keeps using the verb "compute" when in the context should be computation 😅...is it a bot
Ai sh*t is the next .com bubble it has its use case but it's not nearly as cool as ppl think it will be stupid forever because it is a flawed design that just seems to do some stuff relatively fast (which a monkey could do though but slower ) it's a glorified parrot or mechanical turk
Seems to me like a lot of people compare the "learning" a human does during it's lifetime to the training process of a LLM. I think that it would make more sense to compare the training process of a neural network to the evolution process of the human being, and the "learning" a human does during its lifetime to in-context learning in a LLM.
The crazy thing is we can dublicate the best latest level or version of a program or cyborg, with the push of a button.... And keep fine-tune it while it fine tunes and instantly updates itself and all connected devices with the new bit of info that will never be lost while my brain is about explode lol things about get turned upside down fwiw...
Yep, that’s all I kept thinking, we took a long time to get where we are now, millions of generations going all the way back to the origin of life, that’s a lot of energy to get to our current brain organization
No… these are simply statistical models. Nothing compared to the brain, it’s a sad thing that many “brains” aren’t capable of understanding this. Funny how dampening stupidity can be
@@CrimpingPebblesThis! Also the evolutionary aspect of AI doesn’t hunt out efficiency, hence why we’ll need lots of energy and data. The training should hunt out energy efficiency but also data efficiency (thinking / deducing more with less data/information).
The opening statement is so true. As a student of this field, I think that this is not said enough and anyone not well versed in machine learning just does not get how bad the current situation is.
I built a custom spiking neural network for resolving a last-mile logistics efficiency problem. I agree with your assessment: Very efficient. Complex logic.
Oh really? I'd love to hear more. I've been relying on stitching together Google Maps API requests that are limited to 25 nodes a request, but I do 200+ locations. I've been wondering I'd there was an ai solution.
@@regulardegular5 I learned through TH-cam channels 3blue1brown, sentdex, and deeplizard. I also already had a strong background in math, but if you don’t, you might skip the deeplizard videos. sentdex’s videos are most hands-on. 3blue1brown offers the most intuitive explanations of deep neural networks. deeplizard nicely explains the technical parts of training various neural network architectures.
8:33 not only is this "human brain" computer more efficient, but I heard the first stages of creating a new instance is pretty fun, can't confirm, never done it, but they say it is
It's later stages that are more consuming. You sign for life! Perhaps brain is more efficient, but I think Neural Network train faster. Consider the years needed for an human to master language, and think about all the support it need (adults, books, computers etc..). Think about mileniums of human progress needed to develop our current intelligence. Think about all the time needed for life to get where we are! If we ever get to AGI in the next decenies, it's like creating new intelligent life in a fraction of the time needed by nature.
@@takkik282 _"Consider the years needed for an human to master *language,..." *culture - which is a far bigger challenge. With "language" you have to learn concepts and abstracts on any level: syntax, semantic, perfomance... The list what a child learns in these years (0-3-6-10) is far far longer.
@@takkik282 Current llms can't create new non-fake information that weren't already written somewhere. Humans also teaching to walk, run, jump, eat, breathe, feel speed, their position, heat, pain. What about smelling, hearing sound and extracting words from it (and other sounds also), see, separating colors, also internal editing of image from eyes so you don't see your nose and vessels, what about felling by touch every small detail of object. What about precisely controlling your fingers so you don't miss that button on your small phone screen. Also you can download game and even if it has strange controls, most likely after some time you would be good at it (can't say same about ai) Lets take gpt-4o for example. It can't hear you (audio translated to text by another ai), it can't feel anything, it doesn't have physical body, it doesn't have to precisely control muscles to say something, it can't feel anything humans can, it can't teach. It can see images, but that's not continuous video and audio stream that out brain can accept and work with. Even with these limitations of current ai, it uses much more energy that our brain in whole life
a calculator is more efficient than using an excel sheet, one is more versatile than the other. so why bother building a human brain (efficient like a calculator) and just build a swiss army knife instead (a CPU and GPU)
First and foremost, I am a biologist, but I have quite an extensive background in computer science as well. I have some fundamental concerns with the efforts to develop AI, and the methodologies being used. For these models to have anything like intelligence, they need to be adaptable, and they need memory. Some temporal understanding of the world. These efforts with LNN strike me as attempting to re-invent the wheel. Our brains are not just a little better at these tasks than the models. They are exponentially better. My cats come pre-assembled with far superior identification, and decision-making systems. Nevertheless, that flexibility and adaptability require an almost innumerable set of 'alignment' layers to regulate behavior, and control impulses. To make a system flexible, and self-referential is to necessarily make it unpredictable. Sometimes the cat bites you. Sometimes you end up with a serial killer.
Right, and human brain has constant learning feedback loop not only from outside world (through all senses) but also the internal (through self awareness, reflection, critique etc.). Current LLMs don't ever check their responses for validity because there is nowhere to get the feedback from, except the current user, but then the correction will work only in the current short context and not for retraining. So, LLMs essentially just spit out the first response that has the highest probability based on the massive amounts of the training data. And it's quite amazing how often LLMs get it right. Imagine a human not actually solving an equation but spitting out the first response that comes to mind - we would miserably fail all the tests that LLMs pass with ease. Self-correction based on the context awareness is mandatory for an AI.
Its not reinventing the wheel if the wheel hasn't even been invented yet. What neuroscientist say differs largely from what you say. From what iv seen, its hard to take lessons we learnt from real neurons, and put it into computer neurons. It takes 8 whole layers of machine neurons to simulate a human neuron. Human neurons arent just a soma, the dendrites do alot of computation aswell. Current machine learning is inspired by biology but not based on it. If we knew how neurons actually worked, ML wouldv been already solved.
One might argue that current neural network models are both adaptable (they have millions of parameters being updated at each step, throughout hundreds of thousands of training epochs) and memory (they remember the instances to which they have been trained on through the weights between layers). There’s also a lot of highly effective unsupervised learning algorithms that learn complex patterns from unlabeled data, which one might call self-assessment.
@@camelCased the thing is that you can't teach llm in normal way. If you explain person that something is false like this: You: do turtles fly? Person: yes. You: nah, they don't Person: oh, i will remember this Person would remembered that turtles don't fly If you do same thing with ai, ai will remember, that when you ask "do turtles fly?", they should reply "yes", and if you reply "nah, they don't", they should reply "oh, i will remember this" This is the problem with ai
Or maybe to reach AGI (do we really want that?) we need to ditch neural nets and actually discover what makes inteligence work on a high level and reimplement to work on computers... Seems to me like doing any type of neural network is like trying to emulate a game console by simulating each transistor in its circuit... sure, it can work, but it would take the most powerful Threadripper CPU to emulate a Atari 2600 at full speed this way Maybe neural nets will help us understand what makes a brain tick on a high level, then we will make a "brain JIT recompiler"... and then... who knows what will happen next
@@olhoTronwow, never heard this perspective before. The problem I think tho, is that while it’s easy to simulate neurons, the real issue is arranging them correctly to create higher level behaviours. Using your analogy, yeah you can simulate for example the CPU using transistors and then implement it in a higher level way, but to do that first u need a schematic of how each transistor connects to the next. So, rather we brute force arrangements until we make human-like neuron arrangements, or brain scanning technology needs to improve so we can view whole sections of the brain at neuron level
That is what I believe. To me, AGI means the digital equivalent of a human, conscious, self aware and all that. Since the only known example for running AGI (ourselves) is our brain, then we should probably aim to replicate it. Maybe not the cellular biophysics etc.. happening but the overall more abstract ways it works. Any other method has no proven way of getting to AGI. It is how I am going about working on these things at least, using modified spiking neural networks to more closely resemble the brain. It truly is an amazing time to be alive. We are on the brink of a new species being created. Potentially the first time in the entirely of the universe's existence when a given species has created another species smarter than themselves.
@@olhoTron It's not the same. Using transistors to emulate transistors? What we need are actual neurons, artificial of course but electronic. Maybe an integrated circuit that has interconnected devices that function like a neuron but also somewhat transistor like. A transistor with like hundreds of inputs and outputs but really small. 🤔It would need to be dynamic but that's way beyond our current capabilities. Bay just using biology is the best option. Cyborgs.
@@charlesmiller8107 *if* (and its a big if) inteligence is actually computable (and not some kind of quantum or spiritual thing) then it is just a computer program like any other, only difference is its running on wetware simulating the basic blocks of the wetware is not the way to go, its too ineficient, we need to actually understand the problem and reimplementing it to run on current computer architectures If its not computable, then we will never reach AGI with classical computers and no amount of nested dot products will make inteligence emerge
i wrote a spiking neural network from scratch, it can learn but it's not as efficient as learning as typical nn as you can't do gradient descent effectivey, instead you need to adjust the neurons based on a reward. now you can backtrace and reward the last neurons and synapses that resulted to the output you want but it is limited, it works better when you don't just reward the lasts, but reward according to the desired output, still, pretty cool to run and it makes nice visualizations.
8:40 "Human brain only uses 175kWh in a year" - Since human brain cannot work without the body you have to treat [brain+body] as one entity (which is ~4 times more), unless it's a brain in a jar.. but yea i guess still very efficient.
If you apply that logic to the AI then you'd need to factor in many other things too. You are fundamentally misunderstanding what is being compared here.
@@viperlineupuser This logic ensures AI will always have a higher energy cost than the human mind. If we account for evolution in humans, we also have to add in the evolution of humans to the development of AI, since this creation comes from us. Putting this logic into an equation it could look like this ** > evolution energy + human learning energy = modern humans** since modern humans are needed to create AI: **> modern humans + AI learning energy = modern AI** AI, being dependent on humans means they will always take more energy according to that logic when extrapolated.
Definetly make a video on neuromorphic chips And i think the other neural networks outside the scope of this video should have their seperate video as well
We currently simulate neural networks programmatically, which is why they are so inefficient. The problem is, people are so impatient for AGI that they have concentrated all their efforts on achieving it rather than developing an actual neural network.
Yeah, it's like their building a slave rather than a free person. Let the little ai have it's own baby, childhood, etc. These machines only need to be turned on all the time and have something to interact with the real world and parents. Even data didn't just download everything and downloaded the crew's psych profiles just to connect with them.
Glad to hear this. Back in 2010 I was looking for an alternative to network graph system still in use today. Basically we have a scaled version of a decades old tech that we now have the horsepower to run. I will say current neural matrices are but a partial model of a brain. I studied a portion of grey matter neural matrices see the book Spikes. "Fred Rieke, David Warland, Rob de Ruyter van Steveninck, William Bialek". The human brain is exponentially more complex than any scaled multi million GPU, TPU, CPU system. Good video.
1. It's funny that we are trying to create something (AGI) that replicates something else that we do not understand (the human brain). 2. Any neural network that truly emulates the human brain won't need to be trained in the sense you discuss. It would just be. It would learn and be trained by it's design. It would start training immediately and continue to train throughout it's existence. I don't see us ever creating something like this anytime soon (see statement #1).
That and the fact that each human has learnt to speak and regurgitate useful information for 30 years. Assuming that a human from the age of birth consumes 175 kW h for simplicity per year, and the entirety of chatGPT 3 was created using 1287 mW h, ChatGPT is ~245X less efficient than a human - (1287 * 1000) / (175 * 30) Overall, only considering 1 humans energy consumption compared to ChatGPT, which can communicate with more than 200,000 people across the internet simultaneously and a response time of
To be fair, the energy spent on humans evolving also went into developing AI, since we are the ones who are developing them, without spending energy on human evolution, there would be no AI Lets say AIs become sentient... our history will also be part of their history
OC tries to do science but never heard of entropy in information theory, which eliminates evolution of DNA of a brain from nothing under no guiding intelligence. Is wild ass guess, not even theory. Where the evidence?
In the end if it wasn't efficient we wouldn't use it (even if energy was for free). It obviously produces something that you couldn't reproduce even with the same amount of megawatthours of human brains brainstormed together.
Паттерны, которые будет воспроизводить слой "жидкой нейронной сети" будет напрямую зависеть от правил, по которым он функционирует при возбуждении из входного слоя. Скорее всего, самым эффективным средством в будущем, станет представление слоя "жидкой нейронной сети" в виде некоторой совокупности кубитов, при этом такая сеть будет наделена в том числе и свойствами спайковой нейронная сети, но с нулевой задержкой, так как спайк в такой сети будет представлять собой не накопление потенциала, а детерминирование квантового состояния с определенной вероятностью, то есть задержку заменят на вероятность, таким образом, повторяющиеся стимулы необходимы будут лишь для набора финальной статистики, которая, к слову, тоже может определять паттерн для обучаемой сети распознавания.
Yes you are right , but always has a limitation , computers are based in dimensional space , ON OFF and this will improve a lot with Quantum computing but still energy a problem All the best to you .
IMO the biggest benefit of spiking neural networks is that they express permanence, they can operate continuously like a flow of consciousness, as opposed to since input output steps.
A neural network is just probability function which gives how likely the occurrence is, so for a given input how much probability of getting the following output. A output with high probability is the most likely answer to your input. And the network just help to calculate that probability by nodes networks biases, back propagation, residual network, matrix, calculus etc, it seems maths, computer and physics coming together in one place
The output layer doesn’t have to be probabilities. It can be other things as well, such as “how much to drive each motor”, or “how much does each pixel change”
《 Arrays of nanodiodes promise full conservation of energy》 A simple rectifier crystal can, iust short of a replicatable long term demonstration of a powerful prototype, almost certainly filter the random thermal motion of electrons or discrete positiive charged voids called holes so the electric current flowing in one direction predominates. At low system voltage a filtrate of one polarity predominates only a little but there is always usable electrical power derived from the source Johnson Nyquest thermal electrical noise. This net electrical filtrate can be aggregated in a group of separate diodes in consistent alignment parallel creating widely scalable electrical power. As the polarity filtered electrical energy is exported, the amount of thermal energy in the group of diodes decreases. This group cooling will draw heat in from the surrounding ambient heat at a rate depending on the filtering rate and thermal resistance between the group and ambient gas, liquid, or solid warmer than absolute zero. There is a lot of ambient heat on our planet, more in equatorial dry desert summer days and less in polar desert winter nights. Refrigeration by the principle that energy is conserved should produce electricity instead of consuming it. Focusing on explaining the electronic behavior of one composition of simple diode, a near flawless crystal of silicon is modified by implanting a small amount of phosphorus on one side from a ohmic contact end to a junction where the additive is suddenly and completely changed to boron with minimal disturbance of the crystal pattern. The crystal then continues to another ohmic contact. A region of high electrical resistance forms at the junction in this type of diode when the phosphorous near the ĵunction donates electrons that are free to move elsewhere while leaving phosphorus ions held in the crystal while the boron donates a hole which is similalarly free to move. The two types of mobile charges mutually clear each other away near the junction leaving little electrical conductivity. An equlibrium width of this region is settled between the phosphorus, boron, electrons, and holes. Thermal noise is beyond steady state equlibrium. Thermal transients where mobile electrons move from the phosphorus added side to the boron added side ride transient extra conductivity so they are filtered into the external circuit. Electrons are units of electric current. They lose their thermal energy of motion and gain electromotive force, another name for voltage, as they transition between the junction and the array electrical tap. Aloha
LNN Don't use spiking You have it all mixed up. You are thinking about liquid state machines.. What Google's own Gemini own explanation to process stimuli: Liquid neural networks LNNs are designed for continuous adaptation and can update parameters in real-time. They are able to process a wide range of data distributions, respond quickly, and filter out noisy data. LNNs are also smaller, use less data and training time, and have a shorter inference time. VS Liquid state machines LSMs are dynamic neural network models that are based on spiking neural networks (SNNs). They are robust to noise and disturbances in input signals because the internal state of the liquid reservoir acts as a filter. LSMs are well-suited for tasks like pattern recognition and time-series analysis, and have been used in neuroscience and speech
I think it is important to highlight that current artificial neural networks are not based on how human brains work but inspired by biological neural networks. Human brains are really complex thanks to half a billion years of evolution of the brain. There is a pretty good book that serves as a primer on neuroscience called “A Brief History of Intelligence”. If you enjoyed Sapiens you will love this book.
Firstly, thank you for producing such an informative video. One thing I would like to add is to say that current neural networks require a lot more training data than say a three year old child in order to preform a simple classification from such as distinguishing cats from dogs. Our current models require tens of thousands of examples in order to be properly trained. Where as a three year old child would require perhaps five or six examples of each. I would propose an architecture which I am calling Predictive Neural Networks where neurons are arranged in layers where neurons predict which other neurons will fire depending on the input data. For example, high level neurons may be trained to detect an eye but should also 'know' where to find the next eye or where to find the nose. Because a cats nose looks different from a dogs nose and is one of the main distinguishing features, it should be possible to train these networks with much fewer examples.
I don’t get how the Liquid NN should continue to learn if you only train the output layers once and the Reservoir stays the same as well (according to you the reservoir gets randomly initialized before training and from there never changes, just allows for information to circle/ripple in it)
Regarding energy inefficiency, The problem is not with model (weighs and biases), The problem is with Hardware which we use to run that model. The hardware is very energy inefficient in doing matrix multiplication, Thats the problem. But in future, I am sure we will be abel to invent Neuromorphic computing, or being abel to control resistivity of a material electronically, Then we can run the same model with energy efficiency comparable to human brain.
But it doesn't change the fact the basic neural network model is so unoptimal. It's like doing path-finding mathematics with brute force search using optical computer instead of silicon based, instead of advancing to A* algorithms etc.
I wonder what that chip from Terminator's Cyberdyne actually does then; optimize bit manipulation, or field descriptions? (i.e. vector matricies vs liquid NNs)...maybe both? Obviously the encoded abstractions matter too but I wonder if the ceiling we're trying to break is one of them, none of them or both
Today's models can be converted to spiking networks, but we're having issues with larger models like GPT. However, converted models like these can never get that efficiency. The architecture is so different, it must be trained from scratch in neuromorphic chips for full efficiency.
Spiking networks don't do matrix multiplications, just single weight multiplications when a spike comes through. Since most weights or activations are zero, ANNs waste a huge amount of time computing 0*0. Spiking networks have the option of just removing those connections
Like Intel alluded to regarding their neuromorphic chip, we can use AI to fix the library issue. So neuromorphic is close. Maybe just 3 years out. Liquid neural networks might need a bit more help from AI to solve the kinks, but I think AI is the key here as well in order to have the technology ready to be launched.
I'd say there is a scarcity of open source neuromorphic neural networks. An example is the TENNs network built by BrainChip of which we have no idea how good the performance is, but the zero shot learning is purportedly faster than NNs built for systems under the current Von Neumann architecture. There's so much buried within IPs atm that it's hard to find out where the progress really is for neuromorphic computing
This is very interesting! The reservoir layer seems like the digital analog to the subconscious mind! I really love your explanation of this new type of neural network.
Had to pause for a sec after you spoke about energy usage 1/3 in. You can't compare training of the network with usage of the brain. My brain is STILL being trained and as such you really should compare with 46 years of training in my case.. If you want to compare then compare usage against usage, or training against training. Your conclusion is right (That our brain is WAAAY more efficient). but just had to correct you there.
It seems to me liquid neural network (lnn) could be super imposed over a spiking neural network (snn). The liquid would be a permanently or semi-permanent pattern of timed pulses that would morph depending on input. At the same time, the same nodes would also implement snn protocols, including back-propagation. Lnn patterns would automatically effect and be computed by the snn as it learned.
I used a CNN combined with a Liquid Time-constant Network (which is part of LNNs) for my university dissertation, which seems pretty powerful itself, I was able to train a robot to follow me based on image input given the same trained environment and clothes as in the training data. It's interesting stuff
🤯 WOAH! could you share your work? That sounds so fascinating and what I'm interested specifically, it seems like you had some kind of Real-Time feature with LTCN? I'd like to see if there's even a description in literature about this part of AI - the time window it exists in. 🤯
@@jeffreyjdesir Sure, I have an unlisted video that shows it working. It's a little jank since I didn't have the time to code in and train for smoother motion. It was specifically trained to pick up me at various locations within the frame, so if I was to the right of its frame, it would turn right by a specified amount and if I was too far it would move forward by a specified amount and so on. The description has links to the dissertation itself on Google Drive and the code on github th-cam.com/video/ZI2mLThnprM/w-d-xo.html
Feels like there would be a tipping point with a liquid model where it starts out by just working, and then is tasked with making a better model based on its learned measurement of its own current performance. Given it can change and adapt, it could improve on its own design and rince/repeat.
Some nuance to be made: models are updated from time to time even if they don't get a new name or version. So GPT4 designates a series of training. About energy, the processors are getting better at energy consumption, so applying a linear factor to the number of weights is a huge overestimation.
Great video, but one small correction. LLMs can actually "learn", it's just their memory is bad (they remember stuff only as long as it is in their context window). For example, you can actually "teach" ChatGPT to count the letters in the word. As of now if you ask ChatGPT "what's the n-th letter in _" it will guess a random one. However if you explain to it that it has to write the letters out and enumerate them to find the n-th letter, and then ask to use that process it will be able to do so. So in a way you can "teach" ChatGPT to count the letters and find the letter in a specific place of a word.
context window? i think it's not true for the attention mechanism in transformer. unless you are reffering to recurrent network family that use timestep (context window).
I had no idea before this video that other regenerative AI are coming along & already supersede traditional probagation considerably in many use cases. It gives me hope that an AGI breakout is much closer than they say.
I’d argue neural networks can improve with the use of further trading and/or different training data which can change or reconfigure under utilised weights to improve overall accuracy.
When it comes to technology, if it's technically possible and there's economic demand for it, it will happen. Given the stupendous amount of energy needed to train current models, the incentives to perfect these more efficient models will lead to rapid progress. There's no way things stay this inefficient for too long.
The future of AI looks incredibly promising and dynamic! With its ability to learn infinitely, AI is set to revolutionize how we interact with technology. Embrace the endless possibilities and advancements on the horizon! #FutureOfAI #InfiniteLearning #TechRevolution #AI
A high temperature LLM at high rope frequency can emulate a fluid. Personally? The brain is just a KNN machine that routes stimulai with different angular momentum in a matter that preseves at least two types of symmetries keeping them both oppositionally anchored by contrast (a.i think triplets distance). What we have deeply wrong in A.I is modelling the world (Think GPT4, Claude..etc) instead of a filter. But thats fine because philsoophically cognition and then intelligence would be a matter of subject and then matter. Not backwards. We needed the dumb LLM component *any way*.
Liquid neural nets are an interesting application of analog computing (contrasting with discrete math/logic used by traditional neural nets). Analog computers have been making a comeback in general. I wonder if analog computer hardware of some kind could be used to run them.
The difference between 20, 000 parameters versus a trillion represents 50,000 fold improvement in compute and energy consumption. Relative to where we would have been without this breakthrough well then that basically means that we have 50,000 times the capacity available that we thought we had before in compute and energy available for the same amount of output, provided this scales through all computing needs.
Silly question: Why not use (some) all of them in combination, with a superodinate network (perhaps the liquid one) that either learns or is told by training which method to deploy for which type of data? The idea, once again, is to mimic the brain, with modular information processing at lower levels but the executive function at a higher level.
Hi, I would like to know more about the spiking neural networks, their types, limitations, challenges, performance, online learning capabilities, etc. Thanks!
Being fixed is not a disadvantage of the technology as this is mostly a safety feature by design to enshure that model won't spontaneousely get out of bounds of it's usecases or change it's behaviour during performing dangerous tasks in the field
If we can use Blooms Taxonomy as any standard, it seems like something like GPT will EVER "understand" anything; just relative semantic mappings to input which can't be the same thing as hermeneutics ontologically. The AGI singularity should be more about when the new human intelligence with a revitalized cosmic identity (as opposed to national or tribal) that comes with Star Trek like planetary ambitions...hopefully soon (or else).
A single human brain, even though something like 1,000x as complex, could never learn all of human knowledge. Humans have a limit on how much they can store, which is why we forget things. Yet a single LLM that has 1,000x less complexity can know the sum total of all human knowledge. Which is why this comparison of an LLM to a single human brain is ridiculous.
@@Me__Myself__and__I 1. Interesting...I'd say the brain more around 1,000,000,000x complex given its what we know knowledge through and transduces reality...its hard to really call it a process since consciousness is seamless with necessary reality (the world that generates perception). 2. (more nit pick) but I wouldn't say LLMs "know" anything in the degree or relevance (given a Blooms taxonomy approach)that humans do. Some humans many not have every true description of the nature of the world, but can see the "Truth" of the world in a gestalt manner that goes beyond computation and semantics into hermeneutics and telelogy. what say you?
YES WOULD LIKE A DIAGRAM OF DIGITAL LOGIC MODEL, OR ANALOGY DEVICES CONFIGURED, OR THE MICRO CIRCUITS CURRENT FOR THE DIFFERENT CLASSES OF NEURO LOGIC IMPLEMENTATIONS IN RESEARCH AND COMMERICAL APPLICATIONS AND HOW ONE IS TESTED AS DIGITAL LEVELS OR WAVEFORM OUTPUTS
Thanks to our sponsor, Bright Data:
Train your AI models with high-volume, high-quality web data through reliable pipelines, ready-to-use datasets, and scraping APIs.
Learn more at brdta.com/aisearch
Viewers who enjoyed this video also tend to like the following:
You Don't Understand AI Until You Watch THIS th-cam.com/video/1aM1KYvl4Dw/w-d-xo.html
These 5 AI Discoveries will Change the World Forever th-cam.com/video/fyVja-57EIs/w-d-xo.html
The Insane Race for AI Humanoid Robots th-cam.com/video/90TMZ2fq9Gs/w-d-xo.html
These new AI's can create & edit life th-cam.com/video/3K_LAGonsPU/w-d-xo.html
Clickbait. Your "sponsor" deleted the key part, if it ever existed.
Ok there is so much issue in this video. neural network are fixed ? what does this even mean ? we just stop training them, it's a version checkpoint. we could train them continuously. they could learn even by themself without human by "reinforcement learning"... classic neural network can emule any partition or specialisation. this reflexion come from somone that does not really understand how it's works. Liquid neural network what you explained it's smething like a feature extractor, basically an encoder . feel like a lot of bullsht. spking neuron wtf you just discover an activation function... RNN with a simple Relu have the same behaviour. reaching a superior intelligenceby mimicking the brain, holy fk, i was waiting some quantum sht.
you don't understand what your are talking about
@@alexamand2312 by "neural network are fixed"he means the current weights are based on last date of training, like different chatgpt versions...I haven't watched the whole video but the only issue I have is why the narrator keeps using the verb "compute" when in the context should be computation 😅...is it a bot
Wrong... But nice try. Liquid NNs are not the solution. It's actually much simpler
Ai sh*t is the next .com bubble it has its use case but it's not nearly as cool as ppl think it will be stupid forever because it is a flawed design that just seems to do some stuff relatively fast (which a monkey could do though but slower ) it's a glorified parrot or mechanical turk
Seems to me like a lot of people compare the "learning" a human does during it's lifetime to the training process of a LLM. I think that it would make more sense to compare the training process of a neural network to the evolution process of the human being, and the "learning" a human does during its lifetime to in-context learning in a LLM.
More like evolution being the base model training, and lifetime learning being fine-tuning.
The crazy thing is we can dublicate the best latest level or version of a program or cyborg, with the push of a button.... And keep fine-tune it while it fine tunes and instantly updates itself and all connected devices with the new bit of info that will never be lost while my brain is about explode lol things about get turned upside down fwiw...
Yep, that’s all I kept thinking, we took a long time to get where we are now, millions of generations going all the way back to the origin of life, that’s a lot of energy to get to our current brain organization
No… these are simply statistical models. Nothing compared to the brain, it’s a sad thing that many “brains” aren’t capable of understanding this. Funny how dampening stupidity can be
@@CrimpingPebblesThis! Also the evolutionary aspect of AI doesn’t hunt out efficiency, hence why we’ll need lots of energy and data. The training should hunt out energy efficiency but also data efficiency (thinking / deducing more with less data/information).
The opening statement is so true. As a student of this field, I think that this is not said enough and anyone not well versed in machine learning just does not get how bad the current situation is.
Dude I’m a completely rookie, what is so bad about the current situation please? Would love to know.
@@Alexander-or7vr using statistical models without understanding or considering the output validity is borderline insanity
@@Tracing0029 can you tell me why? What’s is the outcome you are worried about?
@@Tracing0029 The output validity is known. This is what back propagation does. Basically we have a generalised function finder.
@@Tracing0029Bruh what r u even talking about.
I built a custom spiking neural network for resolving a last-mile logistics efficiency problem.
I agree with your assessment:
Very efficient.
Complex logic.
That's cool! Thanks for sharing
Oh really? I'd love to hear more. I've been relying on stitching together Google Maps API requests that are limited to 25 nodes a request, but I do 200+ locations. I've been wondering I'd there was an ai solution.
hey how does a noob like me go around building ai
@@regulardegular5 I learned through TH-cam channels 3blue1brown, sentdex, and deeplizard. I also already had a strong background in math, but if you don’t, you might skip the deeplizard videos. sentdex’s videos are most hands-on. 3blue1brown offers the most intuitive explanations of deep neural networks. deeplizard nicely explains the technical parts of training various neural network architectures.
@@regulardegular5 ask ai
Absolutely would love that video about the Neuromorphic Chips!!
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
8:33 not only is this "human brain" computer more efficient, but I heard the first stages of creating a new instance is pretty fun, can't confirm, never done it, but they say it is
It's later stages that are more consuming. You sign for life! Perhaps brain is more efficient, but I think Neural Network train faster. Consider the years needed for an human to master language, and think about all the support it need (adults, books, computers etc..). Think about mileniums of human progress needed to develop our current intelligence. Think about all the time needed for life to get where we are! If we ever get to AGI in the next decenies, it's like creating new intelligent life in a fraction of the time needed by nature.
@@takkik282 _"Consider the years needed for an human to master *language,..."
*culture - which is a far bigger challenge. With "language" you have to learn concepts and abstracts on any level: syntax, semantic, perfomance...
The list what a child learns in these years (0-3-6-10) is far far longer.
i have, it's alright, but it's probably even better when you create the instance with someone who is more knowledgeable and a bit freakier
@@takkik282 Current llms can't create new non-fake information that weren't already written somewhere. Humans also teaching to walk, run, jump, eat, breathe, feel speed, their position, heat, pain. What about smelling, hearing sound and extracting words from it (and other sounds also), see, separating colors, also internal editing of image from eyes so you don't see your nose and vessels, what about felling by touch every small detail of object. What about precisely controlling your fingers so you don't miss that button on your small phone screen.
Also you can download game and even if it has strange controls, most likely after some time you would be good at it (can't say same about ai)
Lets take gpt-4o for example. It can't hear you (audio translated to text by another ai), it can't feel anything, it doesn't have physical body, it doesn't have to precisely control muscles to say something, it can't feel anything humans can, it can't teach. It can see images, but that's not continuous video and audio stream that out brain can accept and work with.
Even with these limitations of current ai, it uses much more energy that our brain in whole life
a calculator is more efficient than using an excel sheet, one is more versatile than the other. so why bother building a human brain (efficient like a calculator) and just build a swiss army knife instead (a CPU and GPU)
First and foremost, I am a biologist, but I have quite an extensive background in computer science as well. I have some fundamental concerns with the efforts to develop AI, and the methodologies being used. For these models to have anything like intelligence, they need to be adaptable, and they need memory. Some temporal understanding of the world. These efforts with LNN strike me as attempting to re-invent the wheel.
Our brains are not just a little better at these tasks than the models. They are exponentially better. My cats come pre-assembled with far superior identification, and decision-making systems. Nevertheless, that flexibility and adaptability require an almost innumerable set of 'alignment' layers to regulate behavior, and control impulses. To make a system flexible, and self-referential is to necessarily make it unpredictable. Sometimes the cat bites you. Sometimes you end up with a serial killer.
Right, and human brain has constant learning feedback loop not only from outside world (through all senses) but also the internal (through self awareness, reflection, critique etc.). Current LLMs don't ever check their responses for validity because there is nowhere to get the feedback from, except the current user, but then the correction will work only in the current short context and not for retraining. So, LLMs essentially just spit out the first response that has the highest probability based on the massive amounts of the training data. And it's quite amazing how often LLMs get it right. Imagine a human not actually solving an equation but spitting out the first response that comes to mind - we would miserably fail all the tests that LLMs pass with ease. Self-correction based on the context awareness is mandatory for an AI.
Its not reinventing the wheel if the wheel hasn't even been invented yet.
What neuroscientist say differs largely from what you say. From what iv seen, its hard to take lessons we learnt from real neurons, and put it into computer neurons.
It takes 8 whole layers of machine neurons to simulate a human neuron. Human neurons arent just a soma, the dendrites do alot of computation aswell.
Current machine learning is inspired by biology but not based on it.
If we knew how neurons actually worked, ML wouldv been already solved.
One might argue that current neural network models are both adaptable (they have millions of parameters being updated at each step, throughout hundreds of thousands of training epochs) and memory (they remember the instances to which they have been trained on through the weights between layers). There’s also a lot of highly effective unsupervised learning algorithms that learn complex patterns from unlabeled data, which one might call self-assessment.
Hello 😉
@@camelCased the thing is that you can't teach llm in normal way.
If you explain person that something is false like this:
You: do turtles fly?
Person: yes.
You: nah, they don't
Person: oh, i will remember this
Person would remembered that turtles don't fly
If you do same thing with ai, ai will remember, that when you ask "do turtles fly?", they should reply "yes", and if you reply "nah, they don't", they should reply "oh, i will remember this"
This is the problem with ai
Spiking neural networks and any future neural networks, sounds like where agi and asi are actually at.
Or maybe to reach AGI (do we really want that?) we need to ditch neural nets and actually discover what makes inteligence work on a high level and reimplement to work on computers...
Seems to me like doing any type of neural network is like trying to emulate a game console by simulating each transistor in its circuit... sure, it can work, but it would take the most powerful Threadripper CPU to emulate a Atari 2600 at full speed this way
Maybe neural nets will help us understand what makes a brain tick on a high level, then we will make a "brain JIT recompiler"... and then... who knows what will happen next
@@olhoTronwow, never heard this perspective before. The problem I think tho, is that while it’s easy to simulate neurons, the real issue is arranging them correctly to create higher level behaviours. Using your analogy, yeah you can simulate for example the CPU using transistors and then implement it in a higher level way, but to do that first u need a schematic of how each transistor connects to the next. So, rather we brute force arrangements until we make human-like neuron arrangements, or brain scanning technology needs to improve so we can view whole sections of the brain at neuron level
That is what I believe. To me, AGI means the digital equivalent of a human, conscious, self aware and all that. Since the only known example for running AGI (ourselves) is our brain, then we should probably aim to replicate it. Maybe not the cellular biophysics etc.. happening but the overall more abstract ways it works. Any other method has no proven way of getting to AGI. It is how I am going about working on these things at least, using modified spiking neural networks to more closely resemble the brain. It truly is an amazing time to be alive. We are on the brink of a new species being created. Potentially the first time in the entirely of the universe's existence when a given species has created another species smarter than themselves.
@@olhoTron It's not the same. Using transistors to emulate transistors? What we need are actual neurons, artificial of course but electronic. Maybe an integrated circuit that has interconnected devices that function like a neuron but also somewhat transistor like. A transistor with like hundreds of inputs and outputs but really small. 🤔It would need to be dynamic but that's way beyond our current capabilities. Bay just using biology is the best option. Cyborgs.
@@charlesmiller8107 *if* (and its a big if) inteligence is actually computable (and not some kind of quantum or spiritual thing) then it is just a computer program like any other, only difference is its running on wetware
simulating the basic blocks of the wetware is not the way to go, its too ineficient, we need to actually understand the problem and reimplementing it to run on current computer architectures
If its not computable, then we will never reach AGI with classical computers and no amount of nested dot products will make inteligence emerge
i wrote a spiking neural network from scratch, it can learn but it's not as efficient as learning as typical nn as you can't do gradient descent effectivey, instead you need to adjust the neurons based on a reward.
now you can backtrace and reward the last neurons and synapses that resulted to the output you want but it is limited, it works better when you don't just reward the lasts, but reward according to the desired output, still, pretty cool to run and it makes nice visualizations.
8:40 "Human brain only uses 175kWh in a year" - Since human brain cannot work without the body you have to treat [brain+body] as one entity (which is ~4 times more), unless it's a brain in a jar.. but yea i guess still very efficient.
If you apply that logic to the AI then you'd need to factor in many other things too. You are fundamentally misunderstanding what is being compared here.
got to take into account the millions of years of evolution to even get to the human brain.
@@lagaul5124that’s a bunch of bs
@@Instant_Nerf he is not wrong, but development costs ≠ training cost
@@viperlineupuser This logic ensures AI will always have a higher energy cost than the human mind. If we account for evolution in humans, we also have to add in the evolution of humans to the development of AI, since this creation comes from us.
Putting this logic into an equation it could look like this
** > evolution energy + human learning energy = modern humans**
since modern humans are needed to create AI:
**> modern humans + AI learning energy = modern AI**
AI, being dependent on humans means they will always take more energy according to that logic when extrapolated.
Definetly make a video on neuromorphic chips
And i think the other neural networks outside the scope of this video should have their seperate video as well
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
We currently simulate neural networks programmatically, which is why they are so inefficient. The problem is, people are so impatient for AGI that they have concentrated all their efforts on achieving it rather than developing an actual neural network.
Yeah, it's like their building a slave rather than a free person. Let the little ai have it's own baby, childhood, etc. These machines only need to be turned on all the time and have something to interact with the real world and parents. Even data didn't just download everything and downloaded the crew's psych profiles just to connect with them.
How else do you want to stimulate them other than with a computer which understand machine code aka programming? 🙃
You can’t simulate more than a few milliseconds of a fly’s brain let alone a human brain. Check EPFL’s research
@@lpmlearning2964 not a simulation. Have actual neural nets instead. Can’t be done on a chip. Some sort of 3D construction is required.
Use AI to build AI 🤖
Glad to hear this. Back in 2010 I was looking for an alternative to network graph system still in use today. Basically we have a scaled version of a decades old tech that we now have the horsepower to run. I will say current neural matrices are but a partial model of a brain. I studied a portion of grey matter neural matrices see the book Spikes. "Fred Rieke, David Warland, Rob de Ruyter van Steveninck, William Bialek". The human brain is exponentially more complex than any scaled multi million GPU, TPU, CPU system. Good video.
1. It's funny that we are trying to create something (AGI) that replicates something else that we do not understand (the human brain).
2. Any neural network that truly emulates the human brain won't need to be trained in the sense you discuss. It would just be. It would learn and be trained by it's design. It would start training immediately and continue to train throughout it's existence. I don't see us ever creating something like this anytime soon (see statement #1).
Great point (#2). If it can keep learning, we just need to create it and it would naturally improve over time, or even learn to reconfigure itself
humans receive constant input and data. We frontload that data requirement. A system that improves over time will need constant input data.
@@theAIsearch he's acting like improvement can happen in a vacuum
@@helloyes2288 ye wtf happen in this video and this commentary, is everyone is a religious bitcoiner that not understand anything ?
Humans actually come somewhat pre-trained from the womb. We have instincts and reflexes.
Just waited for somebody to point the tremendous energy problems of current AI.
Thank you
Still consumes less that a fat asses sitting watching TH-cam
this is my first video from your channel, and I am already impressed!
thanks!
The age of analogue computing is coming. Great video for sure.
I believe that analog/digital hybrid computers will change AI massively in the realm of energy efficiency!
@@Citrusautomaton Exactly,i am rooting for aspinity analog and risc-v digital technologies
The human brain developed over a time span of millions of years. How much energy did that process use?
That and the fact that each human has learnt to speak and regurgitate useful information for 30 years. Assuming that a human from the age of birth consumes 175 kW h for simplicity per year, and the entirety of chatGPT 3 was created using 1287 mW h, ChatGPT is ~245X less efficient than a human - (1287 * 1000) / (175 * 30)
Overall, only considering 1 humans energy consumption compared to ChatGPT, which can communicate with more than 200,000 people across the internet simultaneously and a response time of
To be fair, the energy spent on humans evolving also went into developing AI, since we are the ones who are developing them, without spending energy on human evolution, there would be no AI
Lets say AIs become sentient... our history will also be part of their history
OC tries to do science but never heard of entropy in information theory, which eliminates evolution of DNA of a brain from nothing under no guiding intelligence. Is wild ass guess, not even theory. Where the evidence?
In the end if it wasn't efficient we wouldn't use it (even if energy was for free). It obviously produces something that you couldn't reproduce even with the same amount of megawatthours of human brains brainstormed together.
nice. don't really hear that side of things.
Thanks!
Wow, thanks for the super!
Паттерны, которые будет воспроизводить слой "жидкой нейронной сети" будет напрямую зависеть от правил, по которым он функционирует при возбуждении из входного слоя. Скорее всего, самым эффективным средством в будущем, станет представление слоя "жидкой нейронной сети" в виде некоторой совокупности кубитов, при этом такая сеть будет наделена в том числе и свойствами спайковой нейронная сети, но с нулевой задержкой, так как спайк в такой сети будет представлять собой не накопление потенциала, а детерминирование квантового состояния с определенной вероятностью, то есть задержку заменят на вероятность, таким образом, повторяющиеся стимулы необходимы будут лишь для набора финальной статистики, которая, к слову, тоже может определять паттерн для обучаемой сети распознавания.
Yes you are right , but always has a limitation , computers are based in dimensional space , ON OFF and this will improve a lot with Quantum computing but still energy a problem
All the best to you .
Really nice recap. I've subscribed. Keep it up.
Thanks for the sub!
Oh good - was wondering if spiking and liquid NNs were similar. Both trying to emulate our current understanding of human neurons....neat!
Tnks for the organization idea . About the other video based on neural simulation is very valuable for all .
IMO the biggest benefit of spiking neural networks is that they express permanence, they can operate continuously like a flow of consciousness, as opposed to since input output steps.
Awesome content, as always. I would love to know more about neuromorphic chips. Thanks.
So do I.
Same here
Same
Yes, please.
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Yes please - I'd love to see you do a video on neuromorphic chips!
Keep up the good work!
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Not being able to self-improve is the single greatest limitation of LLMs.
A very competent compression of complex ideas - well done mate!
A neural network is just probability function which gives how likely the occurrence is, so for a given input how much probability of getting the following output. A output with high probability is the most likely answer to your input. And the network just help to calculate that probability by nodes networks biases, back propagation, residual network, matrix, calculus etc, it seems maths, computer and physics coming together in one place
The output layer doesn’t have to be probabilities. It can be other things as well, such as “how much to drive each motor”, or “how much does each pixel change”
its not probability, it's just how high the activation of a neuron is.
《 Arrays of nanodiodes promise full conservation of energy》
A simple rectifier crystal can, iust short of a replicatable long term demonstration of a powerful prototype, almost certainly filter the random thermal motion of electrons or discrete positiive charged voids called holes so the electric current flowing in one direction predominates. At low system voltage a filtrate of one polarity predominates only a little but there is always usable electrical power derived from the source Johnson Nyquest thermal electrical noise. This net electrical filtrate can be aggregated in a group of separate diodes in consistent alignment parallel creating widely scalable electrical power. As the polarity filtered electrical energy is exported, the amount of thermal energy in the group of diodes decreases. This group cooling will draw heat in from the surrounding ambient heat at a rate depending on the filtering rate and thermal resistance between the group and ambient gas, liquid, or solid warmer than absolute zero. There is a lot of ambient heat on our planet, more in equatorial dry desert summer days and less in polar desert winter nights.
Refrigeration by the principle that energy is conserved should produce electricity instead of consuming it.
Focusing on explaining the electronic behavior of one composition of simple diode, a near flawless crystal of silicon is modified by implanting a small amount of phosphorus on one side from a ohmic contact end to a junction where the additive is suddenly and completely changed to boron with minimal disturbance of the crystal pattern. The crystal then continues to another ohmic contact.
A region of high electrical resistance forms at the junction in this type of diode when the phosphorous near the ĵunction donates electrons that are free to move elsewhere while leaving phosphorus ions held in the crystal while the boron donates a hole which is similalarly free to move. The two types of mobile charges mutually clear each other away near the junction leaving little electrical conductivity. An equlibrium width of this region is settled between the phosphorus, boron, electrons, and holes. Thermal noise is beyond steady state equlibrium. Thermal transients where mobile electrons move from the phosphorus added side to the boron added side ride transient extra conductivity so they are filtered into the external circuit. Electrons are units of electric current. They lose their thermal energy of motion and gain electromotive force, another name for voltage, as they transition between the junction and the array electrical tap.
Aloha
LNN Don't use spiking You have it all mixed up. You are thinking about liquid state machines.. What Google's own Gemini own explanation
to process stimuli:
Liquid neural networks
LNNs are designed for continuous adaptation and can update parameters in real-time. They are able to process a wide range of data distributions, respond quickly, and filter out noisy data. LNNs are also smaller, use less data and training time, and have a shorter inference time.
VS
Liquid state machines
LSMs are dynamic neural network models that are based on spiking neural networks (SNNs). They are robust to noise and disturbances in input signals because the internal state of the liquid reservoir acts as a filter. LSMs are well-suited for tasks like pattern recognition and time-series analysis, and have been used in neuroscience and speech
Super useful video, thank you!
You're welcome!
Thanks for the video 😁. I really enjoyed it! I'm also very interested in those neuromorphic chips you talked about in the end.
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Good simple explanation of the current state of neural nets and where they’re going.
Excellent video, very informative! Are there studies showing explicitly that recurrent ANNs are more energy-efficient than feedforward ones?
I think it is important to highlight that current artificial neural networks are not based on how human brains work but inspired by biological neural networks.
Human brains are really complex thanks to half a billion years of evolution of the brain.
There is a pretty good book that serves as a primer on neuroscience called “A Brief History of Intelligence”. If you enjoyed Sapiens you will love this book.
Firstly, thank you for producing such an informative video. One thing I would like to add is to say that current neural networks require a lot more training data than say a three year old child in order to preform a simple classification from such as distinguishing cats from dogs. Our current models require tens of thousands of examples in order to be properly trained. Where as a three year old child would require perhaps five or six examples of each. I would propose an architecture which I am calling Predictive Neural Networks where neurons are arranged in layers where neurons predict which other neurons will fire depending on the input data. For example, high level neurons may be trained to detect an eye but should also 'know' where to find the next eye or where to find the nose. Because a cats nose looks different from a dogs nose and is one of the main distinguishing features, it should be possible to train these networks with much fewer examples.
I don’t get how the Liquid NN should continue to learn if you only train the output layers once and the Reservoir stays the same as well (according to you the reservoir gets randomly initialized before training and from there never changes, just allows for information to circle/ripple in it)
Very good video. You forgot to mention new hardware based neural networks
Yes, please do a video on neuromorphic chips - thanks
See this th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
In response to you question at approx 28 minutes: Yes, I would like to see more on Spiking Neural Networks!
You earned a subscriber rom Kenya....kudos
Thanks!
Regarding energy inefficiency,
The problem is not with model (weighs and biases),
The problem is with Hardware which we use to run that model.
The hardware is very energy inefficient in doing matrix multiplication,
Thats the problem.
But in future, I am sure we will be abel to invent Neuromorphic computing, or being abel to control resistivity of a material electronically,
Then we can run the same model with energy efficiency comparable to human brain.
But it doesn't change the fact the basic neural network model is so unoptimal. It's like doing path-finding mathematics with brute force search using optical computer instead of silicon based, instead of advancing to A* algorithms etc.
I wonder what that chip from Terminator's Cyberdyne actually does then; optimize bit manipulation, or field descriptions? (i.e. vector matricies vs liquid NNs)...maybe both? Obviously the encoded abstractions matter too but I wonder if the ceiling we're trying to break is one of them, none of them or both
Today's models can be converted to spiking networks, but we're having issues with larger models like GPT. However, converted models like these can never get that efficiency. The architecture is so different, it must be trained from scratch in neuromorphic chips for full efficiency.
Spiking networks don't do matrix multiplications, just single weight multiplications when a spike comes through. Since most weights or activations are zero, ANNs waste a huge amount of time computing 0*0. Spiking networks have the option of just removing those connections
and YES- i would love to learn from You about the neuromorphic chips :)
Noted!
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Like Intel alluded to regarding their neuromorphic chip, we can use AI to fix the library issue. So neuromorphic is close. Maybe just 3 years out. Liquid neural networks might need a bit more help from AI to solve the kinks, but I think AI is the key here as well in order to have the technology ready to be launched.
yes, it is the plasticity and learning ability that neural networks lack, thank you for these ideas.
Great video, thank you for your work!
My pleasure!
5:18 I agreed with everything until this point. Gemini did prove that models can learn post training. As they did learning a new language
Correct. I just posted a lengthy comment that contained that very detail.
I'd say there is a scarcity of open source neuromorphic neural networks. An example is the TENNs network built by BrainChip of which we have no idea how good the performance is, but the zero shot learning is purportedly faster than NNs built for systems under the current Von Neumann architecture. There's so much buried within IPs atm that it's hard to find out where the progress really is for neuromorphic computing
Aren't you confusing Liquid Neural Networks and Liquid State Machines? I do not think LNNs have fixed random weights like LSM which are reservoirs.
Neuromorphic chips are a topic I would like to see a video on. seems like there is a lot of potential here.
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
This is very interesting! The reservoir layer seems like the digital analog to the subconscious mind! I really love your explanation of this new type of neural network.
Thanks!
wtf
@@alexamand2312?
Had to pause for a sec after you spoke about energy usage 1/3 in. You can't compare training of the network with usage of the brain. My brain is STILL being trained and as such you really should compare with 46 years of training in my case.. If you want to compare then compare usage against usage, or training against training. Your conclusion is right (That our brain is WAAAY more efficient). but just had to correct you there.
It seems to me liquid neural network (lnn) could be super imposed over a spiking neural network (snn). The liquid would be a permanently or semi-permanent pattern of timed pulses that would morph depending on input. At the same time, the same nodes would also implement snn protocols, including back-propagation. Lnn patterns would automatically effect and be computed by the snn as it learned.
Very informative and well put together video, thanks!
Very welcome!
Fantastic chanell! Super nice!👏👏👏🗣💯💯🔥‼️‼️‼️‼️‼️‼️❤
Thanks!
bot 👏👏👏🗣💯💯🔥‼‼‼‼‼‼❤
@@Mega-wt9do 🤜🤛‼️🗣🔥🔥💯💯💯
I used a CNN combined with a Liquid Time-constant Network (which is part of LNNs) for my university dissertation, which seems pretty powerful itself, I was able to train a robot to follow me based on image input given the same trained environment and clothes as in the training data. It's interesting stuff
🤯 WOAH! could you share your work? That sounds so fascinating and what I'm interested specifically, it seems like you had some kind of Real-Time feature with LTCN? I'd like to see if there's even a description in literature about this part of AI - the time window it exists in. 🤯
@@jeffreyjdesir Sure, I have an unlisted video that shows it working.
It's a little jank since I didn't have the time to code in and train for smoother motion. It was specifically trained to pick up me at various locations within the frame, so if I was to the right of its frame, it would turn right by a specified amount and if I was too far it would move forward by a specified amount and so on.
The description has links to the dissertation itself on Google Drive and the code on github
th-cam.com/video/ZI2mLThnprM/w-d-xo.html
That's very cool! Thanks for sharing!
Feels like there would be a tipping point with a liquid model where it starts out by just working, and then is tasked with making a better model based on its learned measurement of its own current performance. Given it can change and adapt, it could improve on its own design and rince/repeat.
that requires different input data shapes
@@gpt-jcommentbot4759likely its ability to reason on its own and make judgements on its own to progress to said goal
Great educational video, thanks.
Awesome video! I would love that video on neuromorphic chips!
Thanks!
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Thank you for your hard work.
My pleasure!
I must add, that most energy use in training is by back propagation, not by inference (is using NN, not training, but is part of training)
Some nuance to be made: models are updated from time to time even if they don't get a new name or version. So GPT4 designates a series of training. About energy, the processors are getting better at energy consumption, so applying a linear factor to the number of weights is a huge overestimation.
Great video, but one small correction.
LLMs can actually "learn", it's just their memory is bad (they remember stuff only as long as it is in their context window).
For example, you can actually "teach" ChatGPT to count the letters in the word. As of now if you ask ChatGPT "what's the n-th letter in _" it will guess a random one.
However if you explain to it that it has to write the letters out and enumerate them to find the n-th letter, and then ask to use that process it will be able to do so.
So in a way you can "teach" ChatGPT to count the letters and find the letter in a specific place of a word.
I have just tried asking ChatGPT the nth letter in a word and it gets it right without having to explain how to do it.
context window? i think it's not true for the attention mechanism in transformer.
unless you are reffering to recurrent network family that use timestep (context window).
Yes please. Let’s learn about neuromorphic chips!
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Very interesting to hear about these emerging architectures.
I had no idea before this video that other regenerative AI are coming along & already supersede traditional probagation considerably in many use cases. It gives me hope that an AGI breakout is much closer than they say.
Nice explanation. Thanks.
I’d argue neural networks can improve with the use of further trading and/or different training data which can change or reconfigure under utilised weights to improve overall accuracy.
👍 for neuromorphic chips
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Yes neuomorphic chips video please
When it comes to technology, if it's technically possible and there's economic demand for it, it will happen. Given the stupendous amount of energy needed to train current models, the incentives to perfect these more efficient models will lead to rapid progress. There's no way things stay this inefficient for too long.
This is the best AI video I have ever seen. My mind has been blown.
Thanks!
great and easy to understand explanation thanks...
You are welcome!
The future of AI looks incredibly promising and dynamic! With its ability to learn infinitely, AI is set to revolutionize how we interact with technology. Embrace the endless possibilities and advancements on the horizon! #FutureOfAI #InfiniteLearning #TechRevolution #AI
Thank you and God bless you, watching 4rmZambia 🇿🇲
Thanks!
Amazing presentation, thank you! I would love for your to do a follow-up on the potential of neuro-morphic architectures.
Thanks! Will do
A high temperature LLM at high rope frequency can emulate a fluid.
Personally? The brain is just a KNN machine that routes stimulai with different angular momentum in a matter that preseves at least two types of symmetries keeping them both oppositionally anchored by contrast (a.i think triplets distance). What we have deeply wrong in A.I is modelling the world (Think GPT4, Claude..etc) instead of a filter. But thats fine because philsoophically cognition and then intelligence would be a matter of subject and then matter. Not backwards. We needed the dumb LLM component *any way*.
So awesome. Thanks! Neuromorphic, please :)
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Here you go th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Liquid neural nets are an interesting application of analog computing (contrasting with discrete math/logic used by traditional neural nets). Analog computers have been making a comeback in general. I wonder if analog computer hardware of some kind could be used to run them.
The difference between 20, 000 parameters versus a trillion represents 50,000 fold improvement in compute and energy consumption.
Relative to where we would have been without this breakthrough well then that basically means that we have 50,000 times the capacity available that we thought we had before in compute and energy available for the same amount of output, provided this scales through all computing needs.
great informative video! Thanks a lot
Please make us a deep dive into neuromorphic hardware.
Noted!
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Silly question: Why not use (some) all of them in combination, with a superodinate network (perhaps the liquid one) that either learns or is told by training which method to deploy for which type of data? The idea, once again, is to mimic the brain, with modular information processing at lower levels but the executive function at a higher level.
Hi, I would like to know more about the spiking neural networks, their types, limitations, challenges, performance, online learning capabilities, etc. Thanks!
This comment is to let you know that you should, in fact, make a video on neuromorphic chips.
noted!
Just posted: th-cam.com/video/LxOOj-mkQV8/w-d-xo.html
Thank you, my wonderful and amazing brother. Thank you
You are very welcome
Thanks AIS, great video. I guess that's why Sam needs to raise funds.
So basically neural networks are nerds while liquid neural networks are street smart
🤯
Being fixed is not a disadvantage of the technology as this is mostly a safety feature by design to enshure that model won't spontaneousely get out of bounds of it's usecases or change it's behaviour during performing dangerous tasks in the field
Well, how long would it take for a human brain to learn everything gpt knows- probably hundreds or thousands of years
If we can use Blooms Taxonomy as any standard, it seems like something like GPT will EVER "understand" anything; just relative semantic mappings to input which can't be the same thing as hermeneutics ontologically. The AGI singularity should be more about when the new human intelligence with a revitalized cosmic identity (as opposed to national or tribal) that comes with Star Trek like planetary ambitions...hopefully soon (or else).
A single human brain, even though something like 1,000x as complex, could never learn all of human knowledge. Humans have a limit on how much they can store, which is why we forget things. Yet a single LLM that has 1,000x less complexity can know the sum total of all human knowledge. Which is why this comparison of an LLM to a single human brain is ridiculous.
@@Me__Myself__and__I 1. Interesting...I'd say the brain more around 1,000,000,000x complex given its what we know knowledge through and transduces reality...its hard to really call it a process since consciousness is seamless with necessary reality (the world that generates perception).
2. (more nit pick) but I wouldn't say LLMs "know" anything in the degree or relevance (given a Blooms taxonomy approach)that humans do. Some humans many not have every true description of the nature of the world, but can see the "Truth" of the world in a gestalt manner that goes beyond computation and semantics into hermeneutics and telelogy. what say you?
Yet the GPT models don't have human-level intelligence.
Knowledge is not intelligence just as cheese is not electricity.
The problem with Netflix is lack of contents, not AI.
Yeah, it's great at recommending stuff I'd enjoy if I hadn't already watched it...
YES WOULD LIKE A DIAGRAM OF DIGITAL LOGIC MODEL, OR ANALOGY DEVICES CONFIGURED, OR THE MICRO CIRCUITS CURRENT FOR THE DIFFERENT CLASSES OF NEURO LOGIC IMPLEMENTATIONS IN RESEARCH AND COMMERICAL APPLICATIONS AND HOW ONE IS TESTED AS DIGITAL LEVELS OR WAVEFORM OUTPUTS
Excellent video.
What is the difference between learning and interpolation?
hey, super nice high level overview.
thanks!
Very well explained
I think compression is underrated
excelente resumo para entender as dificudades de software e de hardware para a construção dessas redes "liquidas". obrigado
Quantum computing will definitely help take AI to the next level.
I think time for analogue computers to shine