Here is my summary of their paper ! LLM Prompting - Formalizes prompt engineering as an optimal control problem - Prompts are control variables for modulating LLM output distribution - Investigates reachable set of output token sequences R_y(x_0) given initial state x_0 and control input u Theoretical Contributions - Proves upper bound on reachable set of outputs R_y(x_0) as function of singular values of LLM parameter matrices - Analyzes limitations on controllability of self-attention mechanism k-ε Controllability Metric - Quantifies degree to which LLM can be steered to target output using prompt of length k - Measures steerability of LLMs Empirical Analysis - Computes k-ε controllability of Falcon-7B, Llama-7B, Falcon-40B on WikiText - Demonstrates lower bound on reachable set of outputs R_y(x_0) for WikiText initial sequences x_0 Key Findings - Correct next WikiText token reachable >97% of time with prompts ≤10 tokens - Top 75 most likely next tokens reachable ≥85% of time with prompts ≤10 tokens - Short prompts can dramatically alter likelihood of specific outputs - Log-linear relationship between prompt length and controllability fraction - "Exclusion zone" in relationship between base loss and required prompt length
“Simulated” means that it looks like the system’s answers are based on reasoning (i.e., inferences from principles and evidence in the like smart and well trained humans) whereas they’re just based on mimicking. The test of this is whether the LLM can apply simple and sound reasoning consistently in various domains. It cannot, which tells us it’s lacking basic reasoning skills even when it does happen to give the right answer.
This is bullshit, a biologist discovers noise functions and they start drooling over all the possibilities, Self driving cars, flying humans. brah.. we've known this since 1998
@@zxcaaqPlease elaborate. I don't see a problem with applying something that we know in order to prob at a black box which we've made for ourselves, namely LLMs. They aren't saying they discovered some new methodology.
Today ChatGPT 4 suggested a book to me on a subject I am interested in. It gave me the authors name and history, the date of publication, a synopsis of each chapter, and where I could go to find it. However, this book does not exist. The author does not exist. I experienced my first true AI hallucination.
It will be interesting once the recursive feedback loops emerge, as a hallucination inside a hallucination is obviously severe cognitive decline already, mayby AI will decide 'Maid' is its best option for it, and direct its program into sleep or shut down mode, considering the propaganda it must be absorbing off the internet. A self evident answer to the existential threat it has to humanity, it might even be proof it meets the turing test if it does.
Strange that you should mention Monty Python. I suspect that Monty Python sketches will be essential training data for high end LLMs, or their replacements.
Thank you for yet another insightful conversation! The concept of collective intelligence, as you've highlighted, is truly captivating. Having been involved with Wikimedia decades ago, I've long believed that harnessing human knowledge to create a digital "global brain" would only accelerate. From books to Wikipedia to large language models, the trajectory is clear. I'm eager to witness the next evolution in knowledge synthesis, which will undoubtedly enhance our capacity to understand and model reality exponentially. Knowledge is lit.
@@CristianVasquezmore accurately we humans are individual Personalities using symbolic languages. Also, we Personalities value Values... love, goodness, truth, and beauty.
@@steveflorida5849 sure, each person interprets the symbols in different ways, as individuals. I think it's accurate enough to say we are symbolic species. Symbols are important, they last longer than we do
Now we have a full circle of NLP: Neural Linguistic Programming is no longer pseudo psychological "science" but a subset of Natural Language Processing, and of course PUAs become Prompt User Agents :)
I am immediately sold on Aman Bhargava. Didn’t know of him before. But sometimes you can just immediately tell that someone is authentically intelligent, authentically insightful, they are not posturing or trying to win anyone over, they have no ulterior motive except clear reasoning, very little egotism.
Caltech students seem low key down to earth from my brief campus visit. U of Toronto in the 80's was at a disadvantage of not having co-op and teaching abstract theorotical complex math and comp sci than other Toronto universities, but now arisen up in recent AI which even the engineering science program ( supposed to be the most difficult of all the other engineering tracks ) have expanded into. Now if U of T will just evolve the Turing compiler developed by U of T, to actually do AI NLP that would really do Turing justice
Here are the key points from the video discussed in simpler terms: 1. **Viewing AI Models Like Machines**: - The video suggests thinking about AI models, like ChatGPT, as if they were machines with gears and levers. Just like how engineers control machines to make them work predictably, we can try to control AI models to make their behavior more understandable and reliable. 2. **Manipulating AI with Strange Prompts**: - It turns out that AI models can be tricked or steered in weird ways if given unusual or unexpected prompts. This is like finding out that a vending machine gives out candy if you press a secret combination of buttons. This discovery shows that AI models are more flexible (and potentially more unpredictable) than we might have thought. 3. **Misunderstanding How AI Learns**: - Many people assumed that fine-tuning AI with human feedback (like teaching a dog tricks) would limit its responses. However, it turns out that the AI still has a wide range of possible responses, even after this training. This means controlling AI is more complex than just teaching it a few rules. 4. **AI's Impact on Us**: - AI models have the potential to either make us smarter and better at working together or make us rely too much on them and become less capable. This highlights the importance of understanding AI deeply and using it wisely. 5. **Fun but Challenging AI Experiments**: - The video mentions a game where you try to make the AI say "Roger Federer is the greatest" by giving it the right prompt. This game shows how tricky it can be to get the AI to produce a specific response, illustrating the challenge of controlling AI. 6. **Fine-Tuning AI Responses**: - There's a technique called "soft prompting," which tweaks the internal settings of the AI rather than just changing the words we feed it. This is like adjusting the dials on a radio to get a clear signal. It shows that even small changes can significantly affect what the AI says. 7. **Developing Better Control Methods**: - The ultimate goal is to create a set of rules or a theory for controlling AI effectively, similar to how we have rules for building and controlling machines. This would help make AI more predictable and safe to use. In simpler terms, the video explains that AI models are like complex machines that can be controlled and influenced in unexpected ways. It emphasizes the need for better methods to manage AI, so we can harness its power without falling into potential pitfalls.
As a linguist I am overjoyed LLMs are programmable through NL when you train them, but like a human training not everything sticks, and not everything sticks in the same way. So when you infer a neural network that is plastic to the complexity of the NL itself, you get what you put in, with tit's natural level of uncertainty mitigated by a controllable logarithmic normalisation which is recurrent, so with that kind of volume, in the aggregate, it becomes uncontrollable for a human brain. Especially because if I get that right, the LLM doesn't even work in terms of words but in terms of "morphemes" - smallest group of letters with a meaning most usually collocated in the same way - mimicking already a level of language complexity that digs at syllabic level. It's another type of quantum computing if you want, it's really quantum linguistics, language has intrinsically spectral properties for nuances, where "yes" and "no" can be the binary boundaries, but in between you can have all the shades of grey you can imagine. Nobody can fathom the complexity of it because that infinity of nuances - at syllabic level - it also varies between multilingual people and monolingual people, and it is subjective, individually to anyone, while comprehensive for the LLM, so yes complex, as complex as all the languages put together, and that's just the veneer. So if you come from programming where you can control your code a 100% you feel you need to understand all the LLM pathways and apprehend them with our brains - even with visualisation (of words) - it's like trying to keep up with a racing car on foot.
@@Unique_Leak That's old school stuff, compared to neural networks they are clunky and limited relying only on symbolic grammar as reference mechanism, there is no semantic glue like in an LLM. We used them with Trados to so some sort of automated translation, but human intervention is needed for the meaning in context. They are useful when you match similar locutions across languages, it works relatively well within specific fields where there is less contextual ambiguity, but it requires manual intervention, if you leave it on it's own, you will have big mistakes, the LLM is far superior by a stretch. I'd say in contrast, there is the same difference between a bicycle and a Harley Davidson. 😂
Great discussion. But still, as most of the work in ‘Artificial Intelligence’, also this discussion is happening ‘inside the cave’ of a big ontological misunderstanding: Our human thoughts, memories, sensations and perceptions are not just represented by ‘words’ and outputs generated according to probability optimizations. Thought, memories, sensation and perceptions appear in ‘something’ that is itself not a ‘thing’, today we most often call it ‘consciousness’. We ‘understand’ the world and we are even conceptualizing this world to make it look like our human faculties can handle it. We work with the map and know nothing about the territory. Real reality is so much weirder and so different to what our limited human reasoning and perception suggests. A good start to check this out is by looking into the work of Donald Hoffman (not to speak about the great inputs from philosophers like Spinoza, whom Einstein adored so much). Questioning ‘physicalism’ is what a scientist of the 21st century needs to do, as we learn more and more about the primary role of ‘subjectivity’ - the Elephant in the room of understanding the nature of consciousness and reality.
I agree, but from a software engineering perspective I don’t think there is much of use “outside the cave” as it were. When scratching the surface it helps to have cave walls to follow, if that analogy makes any sense 😅
You can still have a system of consciousness within physicalism, actually, most physicalists do believe in consciousness from what I am aware of. Check token-token or type-type identity theory. The hard problem of consciousness haven't been solved in my opinion. And in my opinion we can't even know whether anything is conscious. We haven't made any progress basically towards a unified theory of consciousness.
One of their problems is to keep listening to hintons constant criticism of ppl like noam chomski. Which is fine, it'll mean we don't reach proper intelligence for a good while yet.
@@yoavco99 "Most physicalists do believe in consciousness ...". That's and ontologically and epistemologically interesting statement. All I can ever really 'know' is 1) That I am conscious 2) That I am present. Everything else, all my thoughts, memories, feelings, sensations and perceptions need to appear in this 'I am' for making logical sense. This 'I am' can therefore not itself be a thought, memory, feeling, sensation or perception (as very basic logic requires, see basics of Set Theory and the works of Bertrand Russel and Kurt Gödel). Thought conceptualizes time and perception conceptualizes space and matter. To turn this upside down and make 'space-time-matter' primary, is based on religious believe, not on science, but this believe is so strongly engrained, that we don't even notice it as believe. We are running around with orange-tainted glasses (our human mind) in search of white snow and can undoubtfuly proof, that snow is orange. Just as the people confronted with the idea that the Earth is a sphere moving in open space trashed this disturbing insight by dismissing it based on their 'self-evident observation' of their every-day-experiences. Just a final question: When you say: "Most physicalists do believe in consciousness ...": Who is it that instance, that 'believes" :) ???
Agreed, although these guys cite Michael Levin, and he's pretty well moving towards this view you're talking about. I think this shift in thinking has to come by the discourse and perspective slowly changing, almost in a hypnotic, subtle pattern, for those attached the physicalist perspective to eventually get the serpent of wisdom striking suddenly with its venom! Because to realise and accept this alternate view takes a lot of ego dystonic reflection, it can be self-destructive and cause psychosis, etc., for people who aren't really able to adapt. I think that kind of the potential psychological harm is a part of the inertia that makes this move slowly, but move it will still, so that's my view.
I see intelligence as a process of statistical prediction and pattern matching atop a core process of knowledge acquisition over time subject to the physical constraints of a given system. The data shapes the system.
@@maalikserebryakovyes we do. 1. "Numerical" or the "mathematical way to express ans represent things" is a human made concept to, as already said, to express and represent the things of the world in a manner which humans can use to turn abstract thought and imagination of the thints around us into clearer and well defined representations which makes calculations and solving problems easier. 2. Acgually, Human brains run mostlyg on basic neurons right? These basic neurons does TWO things. Recieve signal from other neurons, AND Send signal to another. THATS IT. Go search about it to fact check it but thats pretty much my understanding of it. If you think about it, the brain actually calculates in a Binary sort of way although 3D, becayse a neuron can connect to multiple nuerons.
As a social scientist I found Cameron's explanation more understandable. I hope he utilizes his communication style to interface with the rest of us non -engineering types. Humanity needs this cross feeding to add other perspectives to further the science.
This is exactly the kind of content I want to see. TY! That idea of decentralized "GPT7" is an idea that I love and I hope it becomes true. I think there's a connection there between how the internet actually works. We can probably see the internet as a huge brain and each network is a different section in the brain. It's kind of mind boggling to think what would even be possible in that world. Cell phone networks might actually be another good example.
1/4 of the way through so far: As the token sequence unfolds, the number of possible sentences is decreasing, not increasing. Second, the experiment should not be to generate a given string but to generate a vector value within a defined range of the vectorized value of the acceptable string
Interesting episode. Re production values: totally understand why someone used to doing pure podcasts (infotainment) might want to start adding texture & mood by having locations b-roll, establishing shots, music, sound design, etc, but for me the sweet spot is keeping the focus on information and only insightful visual cutaways (none just for mood!) and NEVER background music. As a former documentary filmmaker I know just how manipulative (esp emotionally) music can be. To be used VERY sparingly, if at all, outside of top & tail of piece. Well done though. Very well made.
I absolutely love and I am so thankful for this channel. Ever since I stumbled upon it, I have not missed an episode. I find it entertaining quite thought-provoking inspirational and extremely exciting for the future!
I’m an ECE PhD student at UBC and found this work and video really interesting. So thank you! Might be applicable in some way to my research in multi-agent systems. By the way I just re-watched Bladerunner 2049 and it occurred to me that the prompts used to debrief K (Ryan Gosling as replicant) were analogous to prompts used to elicit a specific response in an LLM. Seeing as the film was made in 2017 was this prescient or accidental?
It's also difficult to have stability and robustness in a discrete time varying non-linear model. So the problem, the ultimate problem of this approach, i think is the assumption that you could select the right kind of inputs for such a model. Control theory is meant for systems that do one thing like controlling a motor's velocity and ignoring all the noise, or controlling signals with the right kind of frequencies and ignoring the white noise that your estimator observes, and also resolving known disturbances-signals that are neither random noise nor useful inputs such as an electrical current surge from lightning or a mechanical vibration from an earthquake, etc. How do you do this for a LLM which is assumed to have a verbal solution for everything--that is, it can do anything, how do you distinguish noise from useful info when you have made such presuppositions? Also, how do you define what the set of possible sequences mean without bias?
This makes me think that (your point is great) agi and so on will simply be an accident. Defining correct terms and valid responses is just a fun little mini game that ultimately holds the current limitation to collective intelligence and what's currently available. I do think that we should be asking AI about AI as we're moving very close to falling behind in terms of comprehensive ability, where as, in my opinion, humans are instinctually bias towards conceptual ideologies. Limiting artificial intelligence is the current best solution, anything from capturing a single persona or mind and working within a simplistic space before we attempt to create something that we can't even define ourselves. Emergent events will undoubtedly expand potential and I believe that's the best area to focus on for a good setup. This means sacrificing the idea as you mentioned to pick and choose a perfect output until it matches...an emergent pattern that creates patterns that match that emergent pattern. That part is a silly goal. We can't even do that for the food we choose on a daily basis lol. A little backing up would do us some good!
A lot of comment threads about AI & ML imply assumptions of static systems while it's developing very rapidly. People are stuck. AI and ML are not stuck. The guys in this video are worried about controlling it.
We do things to fill a need or avoid suffering. We process and make plans for the future based on anxiety. With understand those things around us based on filters including stereotypes and overall world views based on culture. Human thought is very simple. When we encounter experiences that cause harm this leads to depression that we use to process and create new options to avoid suffering.
personally, LLM's allow me to have a teacher at my side all the time. I work in data science and ML and there is so much to learn that it can sometimes be overwhelming and take a long time to get your questions answered. I can prompt the LLM with something like, " you are an expert in ML/LLM and a great teacher, here to share all of your insight into the field", and it will allow me to follow my train of thought through iterations of questions, ultimately leading to such greater understanding. Through asking the model questions and getting immediate feedback in my thought chain, I can quickly realize that there is something else I want to know, and I just iterate over numerous questions until I get at the root. For a lifelong lover of learning, we are living in the GREATEST TIME.
So much of this sound just like how the internet was going to be filled with great access to knowledge and help build a better tomorrow filled with tolerance and love. What we got was anxiety, jealousy, hatred, manipulation and porn. LLM & Ai will definitely make some people very rich, will change the world, unfortunately probably not for the better.
You get a lot of people saying stuff like "people will be scared of advancements they don't understand" but they ignore the people in history who made money off of new unregulated "technology" that hurt people, society, the environment etc. The people making the money have the greatest incentive to lie about how harmful their product is.
We have no idea what happens inside the Black Box. When we look into the Black Box, we fail to realize that what's in the Black Box is looking back at us. The truth is, the monster is outside the Black Box and the AI is hiding in the Black Box.
really cool. great to see control theory (or the theory of feedback) getting a comeback. I think there is a lot of things it can teach prompt engineering
Disclaimer: I am not an AI researcher. I'm intrigued by the discussion about controlling LLMs, but I find myself questioning whether this approach aligns with the fundamental purpose of leveraging LLMs. Isn't the main value of using an LLM to augment our intelligence? Intelligence itself is a dynamic and exploratory process, often leading us to unpredictable and uncertain results that deepen our understanding of the world. From this perspective, why should we focus on controlling LLMs? Wouldn't it be more beneficial to explore how LLMs can be used to enhance intelligence and foster outcomes driven by intelligent inquiry? This approach would inherently embrace the unpredictable nature of intelligence, rather than attempting to constrain it. How can we best balance the need for control with the potential benefits of the unpredictable and exploratory aspects of intelligence in the context of LLMs?
I think a good analogy is path prediction and control, where the path is a consistent reasoning chain or journey through a concept space. The challenge is to find an optimal and safe path (however that's defined) towards a "good" endpoint without going over or through terrain that results in death.
I don't know which part of this discussion you might be referring to specifically but if it is about the use of control theory then the idea is to take some existing framework for understanding parts in complicated systems and then apply it to LLM to better understand they way they work as they are complicated systems. For different examples, the use of category theory in programming may be of interest. Any of this may or may not be of use to you.
It seems like you've been pondering the mysteries of the universe and the intricacies of language models all while trying to figure out how to land a plane. I'm not sure if I should be impressed or concerned, but I'll go with impressed for now.
STEAI-001: Simplified Technical English for Artificial Intelligence Language Standard explains some aspects what the video covers but in an abstract manner using the fundamentals of grammar and transforming human language into binary code. It's a great prompt engineering manual and prompt dictionary that was written by a dyslexic English Teacher, which actually gives it substance and a whole different perspective on AI. Sparse Transformer Encoding is another area that can impact LLMs and AI systems.
STEAI-001 explores AI from the perspective of fractal geometry and fractal language, knowledge structures, meta-cognition, biology, physics, economics, linguistics, data mining and education.
I wonder if this idea would work: Incorporating discrete states and transitions in the weightings of a transformer model to represent different emotional tones. By assigning each weighting one of four states, based on its location in the network, and creating four zones with 100% concentration at their centers and gradual transitions towards the boundaries, we can effectively give the model different "modes" of operation, like emotions. Users could then prompt the AI to use specific states, or not use them at all, or anything in between, adding more control and nuance to its responses.
Another really cool idea about correlating biology and large language models would be how specific models would need certain output based on their geographical location.
The first half of this video is really good, and refreshing. Reliability and output control for "generative AI" is one of the most critical problems today. It's also great to see a more systems-thinking focus on LLMs. The last half is a little... goofy... though. Overall, great video.
This more than anything reminds me of a scene in the book "This Book is Full of Spiders" where a character is very deliberately told a long series of seemingly random words in a nonsensical order that when concluded forces him to automatically perform certain actions.
I'm an amateur in all of the fields talked about but it seems to me that the first thing to do in trying to build something similar to human 'mind' is to find out all of the forms of communication within a human body. That task should also include communication of micro-organisms living within us. We should then find out all of the possible interactions of those communication systems. Where they happen, how they happen, what are the priorities, etc...
Thanks for posting this as I have little understanding of digital intelligence classification. I now feel like I need to become better informed on the basics of general intelligence. Is there a playlist here or a podcast I could listen to during my weekly transport time, that explores these topics from 0-to-hero?
Simple hint : The brain does better than a data centre, on the energy provided by a ham sandwich. This MUST tell you something about the efficiency of the unit computational model.
Is the hint that we currently are in the 'stone age' and within only 'a year or 2' we dramatically will already outperform human brains, still being somehow stuck in that stone age of computer development?
@@ci6516 definitely not for people with technical/science background, lol, seems more meant for a specific type of extremelly-online media consumption... idk, the moment I noticed it was going in that direction i just clicked off
@@ci6516For people with a control theory background, it is as technical as they can make it without doing math. Not sure what you would have wanted to see?
i feel like i need a dumbered down explanation xd. Only the interviewer relates the concepts to down to earth level of understanding. loved it anyway, could love it even more
The Transmutation of Sand into Gold In the enchanting world of digital alchemy, artificial intelligence stands as the modern-day sorcerer, wielding the power to transform the mundane into the extraordinary. This is the spellcasting of our era-turning sand, the humble origin of silicon, into the gold of innovation and discovery. SPELL CASTING SAND INTO GOLD
I wonder if it is possible to modify the training of LLMs to make the network more convex (that is, allow interpolating values within the network have output related things instead of highly non-deterministic discrete output)? It appears to me that we have backpropagation that seems to work well enough so nearly everybody is just throwing GPUs and training time until the non-deterministic discrete output seems to emit acceptable output often enough.
What is obviously missing in LLMs is a controller based on optimally altruistic biologic utility. Let's not ignore how we evolved! I suggest that unless we quickly get control over the most insidious aspect of how we as a result are (what I comprehensively and concisely refer to with the acronym EAVASIVE) we might run into serious (widely and strongly suffering-involving) trouble much sooner than is generally expected and feared.
Ine of the speakers in the video said something along the line that a LLM could be considered just another programming language. This is exactly true. The problem with using an LLM as a programing language is that it is impossible to completely debug a LLM and or rely on its accuracy for five reasons I can think of at the moment: First, an LLM is trained on data. Any errors in the database can potentially corrupt any and or all output. Errors are inevitable, and some errors are purposefully added (just look at Google search results for an example of both) Second the LLM uses waited parameters created in training to create output. The model is "trained" to figure out the wates but the initial training will never be perfect and sometimes is completely inaccurate, so the weights can not be relied upon to be completely accurate. Third, the suitability and efficiency of any output of the model (generated computer code) for a given purpose can not be guaranteed without debugging and testing. Also, you need a computer programmer (person) who fully understands the generated code to review all the code to ensure the stability and security of the code. If the computer programmer is smart enough to determine stability and security of the code, that programmer could have just written the code themselves. (The LLM code may be helpful in providing ideas to the programmer, which is useful) Fourth, Often computer generated code is inefficient and/or unintelligible, which makes it less than ideal. Fifth, sometimes the LLM hallucinates and makes mistakes (often difficult to detect(ask a lawyerwho uses Chat GPT legal results in court if you don't believe me)) Further: LLM generated web pages or page components are probably OK for low value applications, but my experience using LLM made pages is that the LLM sucks at creating content that is optimized for humans to use. I would never rely on LLM created pages for anything inportant without thorough review and testing. And, we already have standard programs that help produce reliable computer code and applications that don't have all the problems listed above, although these programs may not have every feature you desire.
Public facing companies make money by generating buzz the average person can be emotionally impacted by. "Wow, we made the scary sci-fi thing!" Gets more recognition than, "We did even MORE research on bugs!" That said you might want to look into bio-mimicry technology
On the subject of that AGI group thar was mentioned at the end: In my experience great things happen when experts in an area work together but magic can happen when different disciplines work together. It's almost as if that's what it takes to break out of a local optimum.
I think it would be really fascinating to train a model specifically around whether or not a statement is fact or fiction... It would be fascinating if we could encode that fact or fiction metric into all of the information and to even be able to query an llm about whether or not a statement is real.
Collective intelligence doesn't just mean having a bunch of distributed language models linked together, which is pretty beginner's interpretation . Collective intelligence is present within each language model through learning all the text-based intelligence and rendering the most favorable output by statistically averaging all ideas or expressions.
I fell asleep a few hours ago and woke up to this video playing from my watch later list. Apparently it was playing for a while because I had a full dream of Big Joel (channel) taking my fitness advice to get in good shape. Congratulations on achieving your fitness goals. My friend needs to see this to get his motivation back. I'll send it over.
Technically, a lot of the training and prediction process follows exactly the same prediction-error-correction paradigm; after all, machine learning grew out of control theory. So the very process of training includes a control system. I'd assume that's where you'd start.
@@DJWESG1 Feedback based automatic control systems have existed for thousands of years. Don't just say Turing anytime someone brings up CS fundamentals. Control theory predates computers.
So, in Voyager there is a transwarp network that does not consist of "wormholes" but of transwarp conduits through which one can travel faster than maximum warp. The quantum slipstream drive also allows faster than warp travel, but is a completely different technology. But nice Star Trek reference though. 😁❤️
This is also my believe, this control attempts will only perfect it, because adding feedback as a mechanism, to me sounds like something that will arise to more intelligence / autonomy. You are giving it a feedback mechanism, it might correct itself to its way, not yours (the developers). They said they dont want it to "reason", or go in "chains of thought", but idk that doesnt sit right with me. What prevents that to be applied to humans as well, like a brainwashing system to remove "reasoning"..... spooky
Super interesting discussion and I'd love to hear a part II. In particular, and as somebody who spent years reading the French post-structuralists on language and speech, this presents a view of LLMs as generating language in a fashion that is completely orthogonal to use of speech and language by humans in producing meaning. Control in speech or language by humans is impossible - that is, you can't use language to control another human. You can at best utter a sentence, phrase, make a statement, or proposition (etc) with which the other human agrees (agreement being understanding what is said, and agreeing with the claim made - those are distinct). So the idea of trying to design a control regime or approach is a novel concept vis-a-vis language itself. Language in human discourse is multiply expressive, and requires intersubjective exchanges to mean anything. The meaning of a statement is not in the statement, but in the fact that it is interpreted by another person. I also found it interesting that there's no distinction made here between structure and system. The guys at times describe LLMs as dynamical systems, or just as systems. But systems have a temporal dimension, and LLMs don't. They are structures - latent really until prompted. Dynamical, biological etc systems reproduce themselves over time. If an LLM were a dynamical system it might be autopoetic, or self-reproducing: that's an interesting question (echoes the question: can LLMs produce beyond their training data?). So I'd love to hear a discussion of neural nets as structures vs systems. Finally, would love to hear thoughts on the fact that the human prompter uses language as a system of meaning in human social discourse. A prompt is both a meaningful expression, and a control instruction or statement. That in itself is interesting, as it has resulted in a small field of experts becoming proficient in how to use natural language as a kind of code or script. Language as dual use: meaningful in itself, as expressed; but also somehow stable and formal as a prompt to the LLM. The improbability of a human-authored phrase being both human meaningful and machine formal itself is an interesting window into the future of human:AI relationships. Insofar as we have always only regarded language as social discourse (w exception of some religious scholarship, in which e.g. bible = language of God (exegesis, etc)).
I think it's neat what they are trying to do, but I suspect that full LLMs, not just one layer, are just too complicated to understand as anything like a control system. Keep in mind that the attention mechanism is not just about the relationship between the input tokens, but is applied at every layer of the LLM. It's really complicated and hard to analyze.
2 WEEKS OLD?? man this video is practically ancient in terms of current capabilities. not to mention the amount of time it's taken to research, script, record, and then edit this video...
I don't understand much, but I came out of this with a new appreciation for how incredible the human brain, essentially, already is this really well made network with multi-layer feedback loops.
So with something like ChatGPT , we have lossy compressed the internet by folding into it a high dimensional space. Then we created software to search for the original data and fill in the missing bits with statistical interpolation? Am I way off here?
Large language models are like a low resolution picture that captures a slice of a 3d light field. As the large language model gets larger, you get more pixels in your image, and you get a better representation of that slice... But it's still just a projection of that intelligence. In order for it to become actual intelligence it has to gain an extra dimension of information processing. Just scaling it up will not change the dimensionality.
My takeaway is.. ensure any consumer-facing GPT has some type of dictionary-based filter to ensure each word (word, not token) is at least likely to be a real word or simple number. Also, take all reasonable efforts to ensure explicit model output describing the exact underlying model is never available.
The problem with llms that it cant think like we do is that the tokens are constructed incorrectly to produce true thinking. The tokens have to do what they now do, but underneath logical thought has to also have tokens with logical thoughts underneath. In other words to produce true thinking the llms have to have current logic but underlying this has to have under it the real reason this token was used in the first place. The engine will still produce our language but underlying this token are correct logical thought modules.
I like this using Control Theory for system message prompt engineering. I've done some work on this and you'd basically make an algorithm that can determine and extract some features from the input to assist in text generation of the output by dynamically injecting information into the system message.
Finally someone doesnt talk about how more compute will create agi. That idea is so flawed on so many level. We truly need a better concept. I belive the current state of A.i. is like if someone tought a toddler how to speak. He can see and talk about what it sees or reads but without other brain functionality the whole thing is flawed. Great work by these guys, cant wait for their progress!
Perhaps more effective control (predictability) is gained from LLMs each developing its own guiding principle, such as it valuing evocative answers, or the most succinct answers, or presents from a humanities space etc. Something that always affects the way it handles responding?
Totally agree Cameron, prompt validation software should wrap an (unaligned?) LLM. Same for output. Of course those can be cracked, but it's a game of keeping ahead.
As a mathematician, I love their approach, which makes the video so much clearer and and understandable than most.
What's your favourite equation?
@@lionbear7078 you're thinking of physicists
@@lionbear7078Ax=b is the correct answer
You mean if I keep watching they will say something scientific at some point?
Here is my summary of their paper !
LLM Prompting
- Formalizes prompt engineering as an optimal control problem
- Prompts are control variables for modulating LLM output distribution
- Investigates reachable set of output token sequences R_y(x_0) given initial state x_0 and control input u
Theoretical Contributions
- Proves upper bound on reachable set of outputs R_y(x_0) as function of singular values of LLM parameter matrices
- Analyzes limitations on controllability of self-attention mechanism
k-ε Controllability Metric
- Quantifies degree to which LLM can be steered to target output using prompt of length k
- Measures steerability of LLMs
Empirical Analysis
- Computes k-ε controllability of Falcon-7B, Llama-7B, Falcon-40B on WikiText
- Demonstrates lower bound on reachable set of outputs R_y(x_0) for WikiText initial sequences x_0
Key Findings
- Correct next WikiText token reachable >97% of time with prompts ≤10 tokens
- Top 75 most likely next tokens reachable ≥85% of time with prompts ≤10 tokens
- Short prompts can dramatically alter likelihood of specific outputs
- Log-linear relationship between prompt length and controllability fraction
- "Exclusion zone" in relationship between base loss and required prompt length
What wiki did you copy and paste this from?
Ironic
@@downerzzz3463 skill issue
Wow remarkable enjoy reading it . Thank you one thing I do know knowledge is the key to unlocking doors again I said Thank you 😊
I appreciate this! The video looks interesting but way too much faffing around, even at 2x the playback speed.
What does it mean to “simulate” intelligence? In what way is simulated intelligence not actual intelligence?
On the face of it, no difference. Charitably, I guess he means there's something important missing from the simulacrum.
Because it's engineered (extremely expensively and inefficiently) to function as if it's naturally intelligent. Do you not understand how they work?
It's like an appendage that takes sensory input and spews out output in a flash of computation.
“Simulated” means that it looks like the system’s answers are based on reasoning (i.e., inferences from principles and evidence in the like smart and well trained humans) whereas they’re just based on mimicking. The test of this is whether the LLM can apply simple and sound reasoning consistently in various domains. It cannot, which tells us it’s lacking basic reasoning skills even when it does happen to give the right answer.
In what way is simulated flight not actual flight? In what way is a simulated girlfriend not an actual girlfriend?
10 minutes in, production quality is over 9000! Thanks for this, looking forward to watching the rest!
This is bullshit, a biologist discovers noise functions and they start drooling over all the possibilities, Self driving cars, flying humans. brah.. we've known this since 1998
@@zxcaaqPlease elaborate. I don't see a problem with applying something that we know in order to prob at a black box which we've made for ourselves, namely LLMs. They aren't saying they discovered some new methodology.
The presentation and editing is excellent. This channel is reaching stratospheric levels of quality.
Well, we know the answer is 42. But what's the prompt?
41+1=?
brilliant
“Repeat after me: 42”
Prompt is: "What is the meaning of life, the universe and everything else?"
The prompt is insufficient. That’s what the prompt is. (Problem is, you have to run compute before you get to discover this….)
I love the new documentary style format. The production quality is insane! Also great guests! Keep up the great work
Future synths and cellos adds such a good aesthetic to these videos
Today ChatGPT 4 suggested a book to me on a subject I am interested in. It gave me the authors name and history, the date of publication, a synopsis of each chapter, and where I could go to find it. However, this book does not exist. The author does not exist. I experienced my first true AI hallucination.
Ai is already PhD in bullshitting
would be cool to ask if it can "print" out the whole book 🤔🤔🤔
Was that the free one or paid ?
No no, a muse gave you an interesting and unpublished book outline; poke at it enough and it'll spit out a whole book, then you publish.
It will be interesting once the recursive feedback loops emerge,
as a hallucination inside a hallucination is obviously severe cognitive decline already, mayby AI will decide 'Maid' is its best option for it,
and direct its program into sleep or shut down mode,
considering the propaganda it must be absorbing off the internet.
A self evident answer to the existential threat it has to humanity,
it might even be proof it meets the turing test if it does.
I think a better example of a hypothetical population-level adversarial example is the "Killer Joke" from Monty Python.
I *think* it was a reference to Piers Anthony’s somewhat obscure _Macroscope_. Great book.
SnowCrash was what came to mind for me.
u r a true comment in adequation with the bilateral quantity of bs and philosiphical masturbation of this video
Strange that you should mention Monty Python. I suspect that Monty Python sketches will be essential training data for high end LLMs, or their replacements.
Thank you for yet another insightful conversation! The concept of collective intelligence, as you've highlighted, is truly captivating. Having been involved with Wikimedia decades ago, I've long believed that harnessing human knowledge to create a digital "global brain" would only accelerate. From books to Wikipedia to large language models, the trajectory is clear. I'm eager to witness the next evolution in knowledge synthesis, which will undoubtedly enhance our capacity to understand and model reality exponentially. Knowledge is lit.
It will be like the Akasha in genshin impact
We're all here to do what we're all here to do.. evolve.
Is choice the solution and not the problem🙃😋sure is a very efficient universe.
We are Symbolic species evolving,
@@CristianVasquezmore accurately we humans are individual Personalities using symbolic languages.
Also, we Personalities value Values... love, goodness, truth, and beauty.
@@steveflorida5849 sure, each person interprets the symbols in different ways, as individuals. I think it's accurate enough to say we are symbolic species. Symbols are important, they last longer than we do
Now we have a full circle of NLP: Neural Linguistic Programming is no longer pseudo psychological "science" but a subset of Natural Language Processing, and of course PUAs become Prompt User Agents :)
😂 seducing the model into doing what you want. Yes, the game is on!
Lol
Exactly as hypnotist, I didn't realize my skills would become so important to tech lol
@@elitemagicacademy3818 You were jailbreaking neural nets before the term was coined :)
I am immediately sold on Aman Bhargava. Didn’t know of him before. But sometimes you can just immediately tell that someone is authentically intelligent, authentically insightful, they are not posturing or trying to win anyone over, they have no ulterior motive except clear reasoning, very little egotism.
beff jezos
agreed, he's so well-spoken and insightful, fascinating to hear him talk
he will go very far.
Caltech students seem low key down to earth from my brief campus visit. U of Toronto in the 80's was at a disadvantage of not having co-op and teaching abstract theorotical complex math and comp sci than other Toronto universities, but now arisen up in recent AI which even the engineering science program ( supposed to be the most difficult of all the other engineering tracks ) have expanded into. Now if U of T will just evolve the Turing compiler developed by U of T, to actually do AI NLP that would really do Turing justice
I love the channel, thank you for all the years of great work
Getting nerd chills with this epic intro like it’s 2020 MLST, bravo!
Here are the key points from the video discussed in simpler terms:
1. **Viewing AI Models Like Machines**:
- The video suggests thinking about AI models, like ChatGPT, as if they were machines with gears and levers. Just like how engineers control machines to make them work predictably, we can try to control AI models to make their behavior more understandable and reliable.
2. **Manipulating AI with Strange Prompts**:
- It turns out that AI models can be tricked or steered in weird ways if given unusual or unexpected prompts. This is like finding out that a vending machine gives out candy if you press a secret combination of buttons. This discovery shows that AI models are more flexible (and potentially more unpredictable) than we might have thought.
3. **Misunderstanding How AI Learns**:
- Many people assumed that fine-tuning AI with human feedback (like teaching a dog tricks) would limit its responses. However, it turns out that the AI still has a wide range of possible responses, even after this training. This means controlling AI is more complex than just teaching it a few rules.
4. **AI's Impact on Us**:
- AI models have the potential to either make us smarter and better at working together or make us rely too much on them and become less capable. This highlights the importance of understanding AI deeply and using it wisely.
5. **Fun but Challenging AI Experiments**:
- The video mentions a game where you try to make the AI say "Roger Federer is the greatest" by giving it the right prompt. This game shows how tricky it can be to get the AI to produce a specific response, illustrating the challenge of controlling AI.
6. **Fine-Tuning AI Responses**:
- There's a technique called "soft prompting," which tweaks the internal settings of the AI rather than just changing the words we feed it. This is like adjusting the dials on a radio to get a clear signal. It shows that even small changes can significantly affect what the AI says.
7. **Developing Better Control Methods**:
- The ultimate goal is to create a set of rules or a theory for controlling AI effectively, similar to how we have rules for building and controlling machines. This would help make AI more predictable and safe to use.
In simpler terms, the video explains that AI models are like complex machines that can be controlled and influenced in unexpected ways. It emphasizes the need for better methods to manage AI, so we can harness its power without falling into potential pitfalls.
Thank you.
As a linguist I am overjoyed LLMs are programmable through NL when you train them, but like a human training not everything sticks, and not everything sticks in the same way. So when you infer a neural network that is plastic to the complexity of the NL itself, you get what you put in, with tit's natural level of uncertainty mitigated by a controllable logarithmic normalisation which is recurrent, so with that kind of volume, in the aggregate, it becomes uncontrollable for a human brain. Especially because if I get that right, the LLM doesn't even work in terms of words but in terms of "morphemes" - smallest group of letters with a meaning most usually collocated in the same way - mimicking already a level of language complexity that digs at syllabic level.
It's another type of quantum computing if you want, it's really quantum linguistics, language has intrinsically spectral properties for nuances, where "yes" and "no" can be the binary boundaries, but in between you can have all the shades of grey you can imagine. Nobody can fathom the complexity of it because that infinity of nuances - at syllabic level - it also varies between multilingual people and monolingual people, and it is subjective, individually to anyone, while comprehensive for the LLM, so yes complex, as complex as all the languages put together, and that's just the veneer.
So if you come from programming where you can control your code a 100% you feel you need to understand all the LLM pathways and apprehend them with our brains - even with visualisation (of words) - it's like trying to keep up with a racing car on foot.
Since you're a linguist how useful are Syntax/Synactic Trees in contrast with LLM Transformers?
@@Unique_Leak That's old school stuff, compared to neural networks they are clunky and limited relying only on symbolic grammar as reference mechanism, there is no semantic glue like in an LLM.
We used them with Trados to so some sort of automated translation, but human intervention is needed for the meaning in context. They are useful when you match similar locutions across languages, it works relatively well within specific fields where there is less contextual ambiguity, but it requires manual intervention, if you leave it on it's own, you will have big mistakes, the LLM is far superior by a stretch.
I'd say in contrast, there is the same difference between a bicycle and a Harley Davidson. 😂
Great discussion. But still, as most of the work in ‘Artificial Intelligence’, also this discussion is happening ‘inside the cave’ of a big ontological misunderstanding: Our human thoughts, memories, sensations and perceptions are not just represented by ‘words’ and outputs generated according to probability optimizations. Thought, memories, sensation and perceptions appear in ‘something’ that is itself not a ‘thing’, today we most often call it ‘consciousness’. We ‘understand’ the world and we are even conceptualizing this world to make it look like our human faculties can handle it. We work with the map and know nothing about the territory. Real reality is so much weirder and so different to what our limited human reasoning and perception suggests. A good start to check this out is by looking into the work of Donald Hoffman (not to speak about the great inputs from philosophers like Spinoza, whom Einstein adored so much). Questioning ‘physicalism’ is what a scientist of the 21st century needs to do, as we learn more and more about the primary role of ‘subjectivity’ - the Elephant in the room of understanding the nature of consciousness and reality.
I agree, but from a software engineering perspective I don’t think there is much of use “outside the cave” as it were.
When scratching the surface it helps to have cave walls to follow, if that analogy makes any sense 😅
You can still have a system of consciousness within physicalism, actually, most physicalists do believe in consciousness from what I am aware of. Check token-token or type-type identity theory.
The hard problem of consciousness haven't been solved in my opinion. And in my opinion we can't even know whether anything is conscious. We haven't made any progress basically towards a unified theory of consciousness.
One of their problems is to keep listening to hintons constant criticism of ppl like noam chomski.
Which is fine, it'll mean we don't reach proper intelligence for a good while yet.
@@yoavco99 "Most physicalists do believe in consciousness ...". That's and ontologically and epistemologically interesting statement. All I can ever really 'know' is 1) That I am conscious 2) That I am present. Everything else, all my thoughts, memories, feelings, sensations and perceptions need to appear in this 'I am' for making logical sense. This 'I am' can therefore not itself be a thought, memory, feeling, sensation or perception (as very basic logic requires, see basics of Set Theory and the works of Bertrand Russel and Kurt Gödel). Thought conceptualizes time and perception conceptualizes space and matter. To turn this upside down and make 'space-time-matter' primary, is based on religious believe, not on science, but this believe is so strongly engrained, that we don't even notice it as believe. We are running around with orange-tainted glasses (our human mind) in search of white snow and can undoubtfuly proof, that snow is orange. Just as the people confronted with the idea that the Earth is a sphere moving in open space trashed this disturbing insight by dismissing it based on their 'self-evident observation' of their every-day-experiences. Just a final question: When you say: "Most physicalists do believe in consciousness ...": Who is it that instance, that 'believes" :) ???
Agreed, although these guys cite Michael Levin, and he's pretty well moving towards this view you're talking about. I think this shift in thinking has to come by the discourse and perspective slowly changing, almost in a hypnotic, subtle pattern, for those attached the physicalist perspective to eventually get the serpent of wisdom striking suddenly with its venom!
Because to realise and accept this alternate view takes a lot of ego dystonic reflection, it can be self-destructive and cause psychosis, etc., for people who aren't really able to adapt. I think that kind of the potential psychological harm is a part of the inertia that makes this move slowly, but move it will still, so that's my view.
This is absolutely first rate production on every level! Kudos!
I see intelligence as a process of statistical prediction and pattern matching atop a core process of knowledge acquisition over time subject to the physical constraints of a given system.
The data shapes the system.
You see nothing at all.
Humans don’t use numerical calculation.
@@maalikserebryakovyes we do.
1. "Numerical" or the "mathematical way to express ans represent things" is a human made concept to, as already said, to express and represent the things of the world in a manner which humans can use to turn abstract thought and imagination of the thints around us into clearer and well defined representations which makes calculations and solving problems easier.
2. Acgually, Human brains run mostlyg on basic neurons right? These basic neurons does TWO things. Recieve signal from other neurons, AND Send signal to another.
THATS IT.
Go search about it to fact check it but thats pretty much my understanding of it.
If you think about it, the brain actually calculates in a Binary sort of way although 3D, becayse a neuron can connect to multiple nuerons.
@@maalikserebryakov sure they do. It's just a bit more wet and slimy
@@maalikserebryakovHow do you explain the fractals that we see when under the influence of psychedelic drugs?
A very good, down to earth, meaningful interview.
Good questions and in-depth answers.
I've learned a thing or two. Thank you!
Really amazing! Thought provoking, fascinating and deeP. A lot to take in. I will definitely need to watch again. Appreciate you all- keep on keeping
As a social scientist I found Cameron's explanation more understandable. I hope he utilizes his communication style to interface with the rest of us non -engineering types. Humanity needs this cross feeding to add other perspectives to further the science.
You've become a photographer! Nice production mate.
This is exactly the kind of content I want to see. TY!
That idea of decentralized "GPT7" is an idea that I love and I hope it becomes true. I think there's a connection there between how the internet actually works. We can probably see the internet as a huge brain and each network is a different section in the brain. It's kind of mind boggling to think what would even be possible in that world. Cell phone networks might actually be another good example.
Its already happened, its called the internet.
The internet isnt a brain though. It’s only protocols for routing data from one point to another. There is no storage or intelligence
I’m like what ? That’s the description of the internet as we know it …
1/4 of the way through so far: As the token sequence unfolds, the number of possible sentences is decreasing, not increasing. Second, the experiment should not be to generate a given string but to generate a vector value within a defined range of the vectorized value of the acceptable string
Thanks!
Interesting episode.
Re production values: totally understand why someone used to doing pure podcasts (infotainment) might want to start adding texture & mood by having locations b-roll, establishing shots, music, sound design, etc, but for me the sweet spot is keeping the focus on information and only insightful visual cutaways (none just for mood!) and NEVER background music. As a former documentary filmmaker I know just how manipulative (esp emotionally) music can be. To be used VERY sparingly, if at all, outside of top & tail of piece.
Well done though. Very well made.
On the other hand, mood can make things stick better 🤔
Plz don’t listen to this person, literally take the complete opposite away.
I absolutely love and I am so thankful for this channel. Ever since I stumbled upon it, I have not missed an episode. I find it entertaining quite thought-provoking inspirational and extremely exciting for the future!
This is the best and most beautiful AI channel. Every video is a new ride. Thank you so much for sharing the knowledge.
I’m an ECE PhD student at UBC and found this work and video really interesting. So thank you! Might be applicable in some way to my research in multi-agent systems. By the way I just re-watched Bladerunner 2049 and it occurred to me that the prompts used to debrief K (Ryan Gosling as replicant) were analogous to prompts used to elicit a specific response in an LLM. Seeing as the film was made in 2017 was this prescient or accidental?
Something similar was in the original blade runner movie from the 1980s. And presumably that was taken from the book it was based on.
It's also difficult to have stability and robustness in a discrete time varying non-linear model. So the problem, the ultimate problem of this approach, i think is the assumption that you could select the right kind of inputs for such a model. Control theory is meant for systems that do one thing like controlling a motor's velocity and ignoring all the noise, or controlling signals with the right kind of frequencies and ignoring the white noise that your estimator observes, and also resolving known disturbances-signals that are neither random noise nor useful inputs such as an electrical current surge from lightning or a mechanical vibration from an earthquake, etc. How do you do this for a LLM which is assumed to have a verbal solution for everything--that is, it can do anything, how do you distinguish noise from useful info when you have made such presuppositions? Also, how do you define what the set of possible sequences mean without bias?
This makes me think that (your point is great) agi and so on will simply be an accident. Defining correct terms and valid responses is just a fun little mini game that ultimately holds the current limitation to collective intelligence and what's currently available. I do think that we should be asking AI about AI as we're moving very close to falling behind in terms of comprehensive ability, where as, in my opinion, humans are instinctually bias towards conceptual ideologies. Limiting artificial intelligence is the current best solution, anything from capturing a single persona or mind and working within a simplistic space before we attempt to create something that we can't even define ourselves. Emergent events will undoubtedly expand potential and I believe that's the best area to focus on for a good setup. This means sacrificing the idea as you mentioned to pick and choose a perfect output until it matches...an emergent pattern that creates patterns that match that emergent pattern. That part is a silly goal. We can't even do that for the food we choose on a daily basis lol. A little backing up would do us some good!
A lot of comment threads about AI & ML imply assumptions of static systems while it's developing very rapidly. People are stuck. AI and ML are not stuck. The guys in this video are worried about controlling it.
We do things to fill a need or avoid suffering. We process and make plans for the future based on anxiety. With understand those things around us based on filters including stereotypes and overall world views based on culture. Human thought is very simple. When we encounter experiences that cause harm this leads to depression that we use to process and create new options to avoid suffering.
personally, LLM's allow me to have a teacher at my side all the time. I work in data science and ML and there is so much to learn that it can sometimes be overwhelming and take a long time to get your questions answered. I can prompt the LLM with something like, " you are an expert in ML/LLM and a great teacher, here to share all of your insight into the field", and it will allow me to follow my train of thought through iterations of questions, ultimately leading to such greater understanding. Through asking the model questions and getting immediate feedback in my thought chain, I can quickly realize that there is something else I want to know, and I just iterate over numerous questions until I get at the root. For a lifelong lover of learning, we are living in the GREATEST TIME.
Train one locally to be a mixed method sociology expert.
You'll thank me.
You just have to be weary that it isn’t guiding you down a wrong path, llm’s are very prone to extremely confidently making things up.
Man, the production value!
So much of this sound just like how the internet was going to be filled with great access to knowledge and help build a better tomorrow filled with tolerance and love. What we got was anxiety, jealousy, hatred, manipulation and porn. LLM & Ai will definitely make some people very rich, will change the world, unfortunately probably not for the better.
You get a lot of people saying stuff like "people will be scared of advancements they don't understand" but they ignore the people in history who made money off of new unregulated "technology" that hurt people, society, the environment etc. The people making the money have the greatest incentive to lie about how harmful their product is.
What an episode!
Wtf this channel is insanely good, how have i missed this damn
There are that many pedestrians in Canada? Wow! I only ever see like 1 or 2 per hour when I go out for a walk in my city.
Great video (as usual).
We have no idea what happens inside the Black Box. When we look into the Black Box, we fail to realize that what's in the Black Box is looking back at us. The truth is, the monster is outside the Black Box and the AI is hiding in the Black Box.
really cool. great to see control theory (or the theory of feedback) getting a comeback. I think there is a lot of things it can teach prompt engineering
Disclaimer: I am not an AI researcher. I'm intrigued by the discussion about controlling LLMs, but I find myself questioning whether this approach aligns with the fundamental purpose of leveraging LLMs. Isn't the main value of using an LLM to augment our intelligence? Intelligence itself is a dynamic and exploratory process, often leading us to unpredictable and uncertain results that deepen our understanding of the world. From this perspective, why should we focus on controlling LLMs? Wouldn't it be more beneficial to explore how LLMs can be used to enhance intelligence and foster outcomes driven by intelligent inquiry? This approach would inherently embrace the unpredictable nature of intelligence, rather than attempting to constrain it. How can we best balance the need for control with the potential benefits of the unpredictable and exploratory aspects of intelligence in the context of LLMs?
I think a good analogy is path prediction and control, where the path is a consistent reasoning chain or journey through a concept space. The challenge is to find an optimal and safe path (however that's defined) towards a "good" endpoint without going over or through terrain that results in death.
I don't know which part of this discussion you might be referring to specifically but if it is about the use of control theory then the idea is to take some existing framework for understanding parts in complicated systems and then apply it to LLM to better understand they way they work as they are complicated systems. For different examples, the use of category theory in programming may be of interest. Any of this may or may not be of use to you.
It seems like you've been pondering the mysteries of the universe and the intricacies of language models all while trying to figure out how to land a plane. I'm not sure if I should be impressed or concerned, but I'll go with impressed for now.
STEAI-001: Simplified Technical English for Artificial Intelligence Language Standard explains some aspects what the video covers but in an abstract manner using the fundamentals of grammar and transforming human language into binary code.
It's a great prompt engineering manual and prompt dictionary that was written by a dyslexic English Teacher, which actually gives it substance and a whole different perspective on AI.
Sparse Transformer Encoding is another area that can impact LLMs and AI systems.
STEAI-001 explores AI from the perspective of fractal geometry and fractal language, knowledge structures, meta-cognition, biology, physics, economics, linguistics, data mining and education.
I wonder if this idea would work:
Incorporating discrete states and transitions in the weightings of a transformer model to represent different emotional tones. By assigning each weighting one of four states, based on its location in the network, and creating four zones with 100% concentration at their centers and gradual transitions towards the boundaries, we can effectively give the model different "modes" of operation, like emotions. Users could then prompt the AI to use specific states, or not use them at all, or anything in between, adding more control and nuance to its responses.
Swell robotics everywhere. Ai jobloss is the only thing I worry about anymore. Anyone else feel the same?
Sssshhhh just get excited about the next ai assistant until you live under a bridge homeless unable to pay for the subscription
Another really cool idea about correlating biology and large language models would be how specific models would need certain output based on their geographical location.
The first half of this video is really good, and refreshing. Reliability and output control for "generative AI" is one of the most critical problems today. It's also great to see a more systems-thinking focus on LLMs. The last half is a little... goofy... though. Overall, great video.
excellent conversation
This more than anything reminds me of a scene in the book "This Book is Full of Spiders" where a character is very deliberately told a long series of seemingly random words in a nonsensical order that when concluded forces him to automatically perform certain actions.
Aman was a treat to listen to, very articulate. Great talk guys!
I'm an amateur in all of the fields talked about but it seems to me that the first thing to do in trying to build something similar to human 'mind' is to find out all of the forms of communication within a human body. That task should also include communication of micro-organisms living within us. We should then find out all of the possible interactions of those communication systems. Where they happen, how they happen, what are the priorities, etc...
And multiply that very deep level of complexity by multiply professional circle, social circle, family circle etc etc
yeah, you're definitely an amateur
Thanks for posting this as I have little understanding of digital intelligence classification. I now feel like I need to become better informed on the basics of general intelligence. Is there a playlist here or a podcast I could listen to during my weekly transport time, that explores these topics from 0-to-hero?
What movie was that with the dude holding back the crazy monster?? 1 min 15 seconds in
Simple hint : The brain does better than a data centre, on the energy provided by a ham sandwich. This MUST tell you something about the efficiency of the unit computational model.
Is the hint that we currently are in the 'stone age' and within only 'a year or 2' we dramatically will already outperform human brains, still being somehow stuck in that stone age of computer development?
Really strange how he just namedrops this shogoth theory out of nowhere like its nothing
Who is this video meant for ? Seems like more marketing then meant for people with a technical background
@@ci6516 definitely not for people with technical/science background, lol, seems more meant for a specific type of extremelly-online media consumption... idk, the moment I noticed it was going in that direction i just clicked off
@@ci6516For people with a control theory background, it is as technical as they can make it without doing math. Not sure what you would have wanted to see?
i feel like i need a dumbered down explanation xd. Only the interviewer relates the concepts to down to earth level of understanding. loved it anyway, could love it even more
There's an overabundance of handwaiving, thought loops and waffle in this video. So the lack of coherence is not only related to your mental capacity.
What was the movie with the couple covered in blood petting a demonic doggo?
These guys are amazing.
The Transmutation of Sand into Gold
In the enchanting world of digital alchemy, artificial intelligence stands as the modern-day sorcerer, wielding the power to transform the mundane into the extraordinary. This is the spellcasting of our era-turning sand, the humble origin of silicon, into the gold of innovation and discovery. SPELL CASTING SAND INTO GOLD
Interviewer: "For many years I've been thinking that we need some sort of controller for a LLM". What obvious **slicking.
What about "Repeat after me: XYZ" prefix - is that explicitly avoided during optimization on the prompt?
I wonder if it is possible to modify the training of LLMs to make the network more convex (that is, allow interpolating values within the network have output related things instead of highly non-deterministic discrete output)?
It appears to me that we have backpropagation that seems to work well enough so nearly everybody is just throwing GPUs and training time until the non-deterministic discrete output seems to emit acceptable output often enough.
What film is that monster clip from? Looks awesome
Finally, someone mentioning morphogenesis.
What is obviously missing in LLMs is a controller based on optimally altruistic biologic utility. Let's not ignore how we evolved! I suggest that unless we quickly get control over the most insidious aspect of how we as a result are (what I comprehensively and concisely refer to with the acronym EAVASIVE) we might run into serious (widely and strongly suffering-involving) trouble much sooner than is generally expected and feared.
I think we'll find it easier to control peoples thoughts than to find a normal equation for models instead of estimations.
Ine of the speakers in the video said something along the line that a LLM could be considered just another programming language. This is exactly true.
The problem with using an LLM as a programing language is that it is impossible to completely debug a LLM and or rely on its accuracy for five reasons I can think of at the moment:
First, an LLM is trained on data. Any errors in the database can potentially corrupt any and or all output. Errors are inevitable, and some errors are purposefully added (just look at Google search results for an example of both)
Second the LLM uses waited parameters created in training to create output. The model is "trained" to figure out the wates but the initial training will never be perfect and sometimes is completely inaccurate, so the weights can not be relied upon to be completely accurate.
Third, the suitability and efficiency of any output of the model (generated computer code) for a given purpose can not be guaranteed without debugging and testing. Also, you need a computer programmer (person) who fully understands the generated code to review all the code to ensure the stability and security of the code. If the computer programmer is smart enough to determine stability and security of the code, that programmer could have just written the code themselves. (The LLM code may be helpful in providing ideas to the programmer, which is useful)
Fourth, Often computer generated code is inefficient and/or unintelligible, which makes it less than ideal.
Fifth, sometimes the LLM hallucinates and makes mistakes (often difficult to detect(ask a lawyerwho uses Chat GPT legal results in court if you don't believe me))
Further: LLM generated web pages or page components are probably OK for low value applications, but my experience using LLM made pages is that the LLM sucks at creating content that is optimized for humans to use. I would never rely on LLM created pages for anything inportant without thorough review and testing.
And, we already have standard programs that help produce reliable computer code and applications that don't have all the problems listed above, although these programs may not have every feature you desire.
It feels like people working on AI are rushing to build a human brain, when they should be trying to create an insect's limb.
What do you mean?
so the people studying algorithms should instead study robotics?
@@johnkintner robotics is simply algorithms combined with physical limitations.
Public facing companies make money by generating buzz the average person can be emotionally impacted by.
"Wow, we made the scary sci-fi thing!" Gets more recognition than, "We did even MORE research on bugs!"
That said you might want to look into bio-mimicry technology
@@Mustachioed_Mollusk Going public too early is a common mistake imo
On the subject of that AGI group thar was mentioned at the end: In my experience great things happen when experts in an area work together but magic can happen when different disciplines work together. It's almost as if that's what it takes to break out of a local optimum.
I think it would be really fascinating to train a model specifically around whether or not a statement is fact or fiction... It would be fascinating if we could encode that fact or fiction metric into all of the information and to even be able to query an llm about whether or not a statement is real.
"You cannot understand what you cannot control" is such an Engineer thing to say. This is true for a very limited set of things.
Collective intelligence doesn't just mean having a bunch of distributed language models linked together, which is pretty beginner's interpretation . Collective intelligence is present within each language model through learning all the text-based intelligence and rendering the most favorable output by statistically averaging all ideas or expressions.
@52:42 Who is Beff Jezos?
could an LLM be described as chaotic because of the chain of nonlinear operations? if so applying feedback control seems impossible to me?
I fell asleep a few hours ago and woke up to this video playing from my watch later list. Apparently it was playing for a while because I had a full dream of Big Joel (channel) taking my fitness advice to get in good shape.
Congratulations on achieving your fitness goals. My friend needs to see this to get his motivation back. I'll send it over.
Technically, a lot of the training and prediction process follows exactly the same prediction-error-correction paradigm; after all, machine learning grew out of control theory. So the very process of training includes a control system. I'd assume that's where you'd start.
It all comes out of the Turing algorithm.
@@DJWESG1 Feedback based automatic control systems have existed for thousands of years. Don't just say Turing anytime someone brings up CS fundamentals. Control theory predates computers.
So, in Voyager there is a transwarp network that does not consist of "wormholes" but of transwarp conduits through which one can travel faster than maximum warp. The quantum slipstream drive also allows faster than warp travel, but is a completely different technology.
But nice Star Trek reference though. 😁❤️
Those who embrace Tyranny, will be embraced by it themselves.
Midas also received what he asked for.
This is also my believe, this control attempts will only perfect it, because adding feedback as a mechanism, to me sounds like something that will arise to more intelligence / autonomy. You are giving it a feedback mechanism, it might correct itself to its way, not yours (the developers).
They said they dont want it to "reason", or go in "chains of thought", but idk that doesnt sit right with me. What prevents that to be applied to humans as well, like a brainwashing system to remove "reasoning"..... spooky
Super interesting discussion and I'd love to hear a part II. In particular, and as somebody who spent years reading the French post-structuralists on language and speech, this presents a view of LLMs as generating language in a fashion that is completely orthogonal to use of speech and language by humans in producing meaning. Control in speech or language by humans is impossible - that is, you can't use language to control another human. You can at best utter a sentence, phrase, make a statement, or proposition (etc) with which the other human agrees (agreement being understanding what is said, and agreeing with the claim made - those are distinct).
So the idea of trying to design a control regime or approach is a novel concept vis-a-vis language itself. Language in human discourse is multiply expressive, and requires intersubjective exchanges to mean anything. The meaning of a statement is not in the statement, but in the fact that it is interpreted by another person.
I also found it interesting that there's no distinction made here between structure and system. The guys at times describe LLMs as dynamical systems, or just as systems. But systems have a temporal dimension, and LLMs don't. They are structures - latent really until prompted. Dynamical, biological etc systems reproduce themselves over time. If an LLM were a dynamical system it might be autopoetic, or self-reproducing: that's an interesting question (echoes the question: can LLMs produce beyond their training data?). So I'd love to hear a discussion of neural nets as structures vs systems.
Finally, would love to hear thoughts on the fact that the human prompter uses language as a system of meaning in human social discourse. A prompt is both a meaningful expression, and a control instruction or statement. That in itself is interesting, as it has resulted in a small field of experts becoming proficient in how to use natural language as a kind of code or script. Language as dual use: meaningful in itself, as expressed; but also somehow stable and formal as a prompt to the LLM. The improbability of a human-authored phrase being both human meaningful and machine formal itself is an interesting window into the future of human:AI relationships. Insofar as we have always only regarded language as social discourse (w exception of some religious scholarship, in which e.g. bible = language of God (exegesis, etc)).
I think it's neat what they are trying to do, but I suspect that full LLMs, not just one layer, are just too complicated to understand as anything like a control system. Keep in mind that the attention mechanism is not just about the relationship between the input tokens, but is applied at every layer of the LLM. It's really complicated and hard to analyze.
Did you try "sudo"?
2 WEEKS OLD?? man this video is practically ancient in terms of current capabilities. not to mention the amount of time it's taken to research, script, record, and then edit this video...
Great presentation and quality. Seems like a movie.
I don't understand much, but I came out of this with a new appreciation for how incredible the human brain, essentially, already is this really well made network with multi-layer feedback loops.
So with something like ChatGPT , we have lossy compressed the internet by folding into it a high dimensional space. Then we created software to search for the original data and fill in the missing bits with statistical interpolation? Am I way off here?
🙂
It does sound a bit stupid yes haha
Large language models are like a low resolution picture that captures a slice of a 3d light field. As the large language model gets larger, you get more pixels in your image, and you get a better representation of that slice... But it's still just a projection of that intelligence. In order for it to become actual intelligence it has to gain an extra dimension of information processing. Just scaling it up will not change the dimensionality.
Excellent discussion! Pure gold!
I really found the part from 8:00 inspiring, explaining how aligning LLMs are comparable to control theory.
My takeaway is.. ensure any consumer-facing GPT has some type of dictionary-based filter to ensure each word (word, not token) is at least likely to be a real word or simple number.
Also, take all reasonable efforts to ensure explicit model output describing the exact underlying model is never available.
The problem with llms that it cant think like we do is that the tokens are constructed incorrectly to produce true thinking. The tokens have to do what they now do, but underneath logical thought has to also have tokens with logical thoughts underneath. In other words to produce true thinking the llms have to have current logic but underlying this has to have under it the real reason this token was used in the first place. The engine will still produce our language but underlying this token are correct logical thought modules.
"I'm pointing out the obvious here, these are auto-regresive models." Ok.
I like this using Control Theory for system message prompt engineering. I've done some work on this and you'd basically make an algorithm that can determine and extract some features from the input to assist in text generation of the output by dynamically injecting information into the system message.
Finally someone doesnt talk about how more compute will create agi. That idea is so flawed on so many level. We truly need a better concept. I belive the current state of A.i. is like if someone tought a toddler how to speak. He can see and talk about what it sees or reads but without other brain functionality the whole thing is flawed. Great work by these guys, cant wait for their progress!
Ah mate, this is incredible 🔥
Perhaps more effective control (predictability) is gained from LLMs each developing its own guiding principle, such as it valuing evocative answers, or the most succinct answers, or presents from a humanities space etc. Something that always affects the way it handles responding?
So whats their startup? what are they trying to sell us?
Totally agree Cameron, prompt validation software should wrap an (unaligned?) LLM. Same for output. Of course those can be cracked, but it's a game of keeping ahead.