DeepSeek R1 GAVE ITSELF a 2x Speed Boost - Self-Evolving LLM

แชร์
ฝัง
  • เผยแพร่เมื่อ 10 ก.พ. 2025
  • Join My Newsletter for Regular AI Updates 👇🏼
    forwardfuture.ai
    My Links 🔗
    👉🏻 Subscribe: / @matthew_berman
    👉🏻 Twitter: / matthewberman
    👉🏻 Discord: / discord
    👉🏻 Patreon: / matthewberman
    👉🏻 Instagram: / matthewberman_ai
    👉🏻 Threads: www.threads.ne...
    👉🏻 LinkedIn: / forward-future-ai
    Media/Sponsorship Inquiries ✅
    bit.ly/44TC45V
    Links:
    simonwillison....
    situational-aw...
    x.com/liangche...

ความคิดเห็น • 1.1K

  • @1MinuteFlipDoc
    @1MinuteFlipDoc 5 วันที่ผ่านมา +693

    Yes to the tutorial video on narrow topic training!!

    • @matiasm.3124
      @matiasm.3124 5 วันที่ผ่านมา +22

      Yeap i second this please

    • @BROLIGMA
      @BROLIGMA 5 วันที่ผ่านมา +15

      100%

    • @jhuilar
      @jhuilar 5 วันที่ผ่านมา +14

      YES!

    • @marcoburenko5693
      @marcoburenko5693 5 วันที่ผ่านมา +11

      Yes please!

    • @AlexanderMorou
      @AlexanderMorou 5 วันที่ผ่านมา +11

      Absolutely.

  • @classicalmechanic8914
    @classicalmechanic8914 5 วันที่ผ่านมา +158

    Last week: AI aha moment for 30$
    This week: AI aha moment for 3$
    Next week: AI aha moment for 0.3$

    • @lordkacke6220
      @lordkacke6220 5 วันที่ผ่านมา +24

      Next year: AI aha momemt for -0.3$.
      Lets earn some money from these AI bots

    • @kloszi
      @kloszi 5 วันที่ผ่านมา +12

      You miss first AHA moment for 5 000 000

    • @Mcmeider
      @Mcmeider 5 วันที่ผ่านมา +11

      in 14 days: A penny for your thoughts

    • @SzcZ
      @SzcZ 5 วันที่ผ่านมา +7

      And soon you will get paid $3 just for saying "aha".

    • @YawasiDegong-truth
      @YawasiDegong-truth 5 วันที่ผ่านมา

      openai is in deepsh*t

  • @velocityerp
    @velocityerp 5 วันที่ผ่านมา +216

    Yes - PLEASE demo on tiny model training. We are committed to deployed edge SLM - tiny models and recent advances in training tiny models further supports our air gapped, edge first application of AI. Thanks very much for your reporting - VERY helpful.

    • @entzyeung
      @entzyeung 5 วันที่ผ่านมา +2

      2nd this!

    • @arincrumley9046
      @arincrumley9046 5 วันที่ผ่านมา +1

      To be honest, I think a lot of startups are going to be born out of essentially just this. They can have domain knowledge. They can build a user interface. They can conduct building. They can be essentially a landlord to GPUs. That’s really their business. But they’re also a marketing department. They’re also a sales department. And they have some cortex, which is nothing more than RL Specialized models. So if you wanna help about 100,000 neat startups get going then go ahead and make the tutorial. I would make it for them specifically. And I sincerely think that’s cool. But a bigger question I’m curious about is whether or not those types of companies will last. Because why not have AI that can create specialized tiny models that train on a reward function that it figures out on its own. And sometimes you ask AI to do something and it tells you that for three dollars it can go and make a specialized model and if you say yes, then you pay whatever GPU landlord is charging and you’re good.

    • @arincrumley9046
      @arincrumley9046 5 วันที่ผ่านมา

      One thing that I’ve been imagining is a model that’s specialized to understand my own deficiencies. If it’s a tutor model and it knows what it has taught me and what I’ve been able to verify, and it knows where I have struggled then it can have a representation of where my holes are.
      And then that specialized model can work to bridge my knowledge gaps. Every 1 to 6 months I pay another three dollars to further refine that model of what it is that I know. That way an AI becomes incredibly personalized. And incredibly effective at closing educational knowledge gaps.
      We get to the point where we start to have a digital twin of our brain.
      We’ve summarized into a system, the world‘s knowledge. That’s the current LLM.
      Now can we summarize into a system an individual’s knowledge. Complete with gaps. Cause then the global LLM can talk to the individual clone LLM before talking to the real life human. The communication can be refined to not confused the person.
      Because right now if you say you wanna be talked to like you’re five years old it gets too simple but if you read some of these deep research results, they’re really hard to get through.
      We don’t want AI is so good that it leaves the train station and we’re just left behind.
      Sometimes an engineer has to sit their CEO down and just explain to them very slowly what they’re doing so an executive decision can be made.
      Talking down or patronizing somebody is really frustrating. With a coworker, you get a model in your mind of what they do and don’t know.
      That’s really key. When you speak to an audience, you have to generalize. Right now, LLMs are generalizing and attempting to make sense universally to any audience. But they’re not as helpful because they are in that common audience mode too often. That human feedback layer is what has made. The model is very annoying.
      Individualized models. That’s what I think. All of this RL is pointing us towards.

    • @YawasiDegong-truth
      @YawasiDegong-truth 5 วันที่ผ่านมา

      openai is in deepsh*t

    • @iangarrison9516
      @iangarrison9516 5 วันที่ผ่านมา

      @@arincrumley9046this is absolutely my thoughts, too. I am chomping at the bit to get enough hardware at home to start implementing some crazy stuff. I knew this was coming, but figured it wouldn’t be for 2-3 more months

  • @Ayyouboss
    @Ayyouboss 5 วันที่ผ่านมา +198

    It gave itself a 100% improvement. 200% is triple speed.

    • @motionproplus6346
      @motionproplus6346 5 วันที่ผ่านมา +6

      this pissed me off too

    • @b0b0-
      @b0b0- 5 วันที่ผ่านมา +3

      Came here for this, help us Lord

    • @ritesh146
      @ritesh146 5 วันที่ผ่านมา +7

      100% is twice the speed, and 200% is 4 times the speed.

    • @MartinKrisell
      @MartinKrisell 5 วันที่ผ่านมา +21

      @@ritesh146 No, a 200 % increase is 3x original. You're confusing it with the compound increase of 100 % and then another 100 %, which is 4x, or 300 % increase.

    • @spinninglink
      @spinninglink 5 วันที่ผ่านมา +5

      @@ritesh146 no, 200% MORE is 3 times the speed. Think of it this way, i have 2 apples. If i get 200% MORE apples, (200% of current apples is 4), then I'll have 6 apples. 2x3 = 6

  • @JaddBoyden
    @JaddBoyden 5 วันที่ผ่านมา +126

    Yes! Tutorials for Tiny Model Training! Yes!

  • @PedroPenhaVerani-ll1wc
    @PedroPenhaVerani-ll1wc 5 วันที่ผ่านมา +431

    WTF, such disinformation! The improved code is not DeepSeek at all. How could you open the pull request and not notice that the performance improvement has nothing to do with DeepSeek, except for the fact that it was written by DeepSeek?

    • @BracerJack
      @BracerJack 5 วันที่ผ่านมา +163

      Because he needs to constantly say that "It is happening right now!!!", at this point this is just a shock channel.
      Think of it as tabloid news but for A.I.
      His next video will be OMG AGI/Fall Off/A-Star happening right now!!!

    • @danstellargames
      @danstellargames 5 วันที่ผ่านมา +13

      How did DeepSeek write the improvement then?

    • @3ull
      @3ull 5 วันที่ผ่านมา +5

      Such contradiction!

    • @thegame_master2900
      @thegame_master2900 5 วันที่ผ่านมา +25

      English 101 classes are available online

    • @PedroPenhaVerani-ll1wc
      @PedroPenhaVerani-ll1wc 5 วันที่ผ่านมา +51

      @@danstellargames What kind of question is that? The person simply shared some code + a prompt asking to improve code performance and Deepseek did it. But the code that was improved has NOTHING to do with Deepseek.

  • @a6gittiworld
    @a6gittiworld 4 วันที่ผ่านมา +4

    Yes it would be cool 9:43

  • @UnderTheNorthStar
    @UnderTheNorthStar 5 วันที่ผ่านมา +28

    Yes, the tutorial would be amazing. I was just going to go through it myself but would love to hear everyone else's thoughts. Thanks Matthew.

  • @seekererebus255
    @seekererebus255 5 วันที่ผ่านมา +10

    100 trained specialist models for $300 could probably compete directly on any complex generalized corporate operation with the big foundation models; a literal mixture of experts.
    The swarm is unstoppable.

  • @paulmichaelfreedman8334
    @paulmichaelfreedman8334 5 วันที่ผ่านมา +26

    Now if someone creates a supercluster for DeepSeek, and then trains agents that are distributed as open source at super speed, we don't all have to reinvent the wheel.

  • @RussellDeming
    @RussellDeming 5 วันที่ผ่านมา +38

    Absolutely you should build one!

  • @enermaxstephens1051
    @enermaxstephens1051 5 วันที่ผ่านมา +2

    That's pretty misleading, it had a lot of help from the programmer. "Improved itself!!" implies that they simply told it to root out inefficiencies and it did so all on its own with little to no human intervention.

  • @TITANBABY-rq3mr
    @TITANBABY-rq3mr 5 วันที่ผ่านมา +73

    What a time to be alive

    • @cutemartinj
      @cutemartinj 5 วันที่ผ่านมา +19

      Hold on to your papers

    • @MTheoOA
      @MTheoOA 5 วันที่ผ่านมา +9

      @@cutemartinj fellow scholars!

    • @egor.okhterov
      @egor.okhterov 5 วันที่ผ่านมา +7

      I stopped watching that guy long time ago, precisely because of his artificially constructed sighs and wows and I can feel that he is pretending which makes me cringe.

    • @Pyseph
      @Pyseph 5 วันที่ผ่านมา +8

      @@egor.okhterov for me, I noticed that his quality seemed to have dropped significantly; producing the exact same "NVidia just BROKE (thing)" or "(Company) just did the IMPOSSIBLE..." for every. single. video. the faked enthusiasm reminds me of AI slop articles, albeit this time its narrated by a real person

    • @technolus5742
      @technolus5742 5 วันที่ผ่านมา

      ​@@Pysephsomehow this channel is still much worse

  • @InsulaLabs
    @InsulaLabs 4 วันที่ผ่านมา +2

    I work with these as a developer every day.
    With how quickly they fall apart I don’t think they will be self constructing anytime soon

  • @HCG
    @HCG 5 วันที่ผ่านมา +95

    This is painfully false. Do more research before pumping out these OMG AI is amazing videos. It didn’t “improve itself”

    • @CometStation
      @CometStation 5 วันที่ผ่านมา +9

      Welcome to TH-cam.

    • @DiegoSanchez-ih5sd
      @DiegoSanchez-ih5sd 5 วันที่ผ่านมา +4

      Well I did swallow the marketing, I liked the video. Can you elaborate more why this is false ?

    • @TripleOmega
      @TripleOmega 5 วันที่ผ่านมา +11

      That's a huge amount of words to say "No, you're wrong." At least provide some evidence when you claim someone is wrong.

    • @toCatchAnAI
      @toCatchAnAI 5 วันที่ผ่านมา +3

      source please, the video provided that.

    • @epicgamer42069
      @epicgamer42069 5 วันที่ผ่านมา +1

      Evidence?

  • @richardede9594
    @richardede9594 5 วันที่ผ่านมา +18

    Deepseek improves its own intelligence, meanwhile Jaden Smith wears a Castle on his head at the Grammys.
    AI will soon run the World....

  • @Inzajeano
    @Inzajeano 5 วันที่ผ่านมา +7

    We are officially living in the age of deception.

  • @airhardSir
    @airhardSir 5 วันที่ผ่านมา +23

    Yes a training tutorial would be nice. Thanks for your time and work ❤

  • @Psychopatz
    @Psychopatz 5 วันที่ผ่านมา +92

    RIP My 5 year programming Course ;3

    • @Gaze73
      @Gaze73 5 วันที่ผ่านมา +1

      xD

    • @VioFax
      @VioFax 5 วันที่ผ่านมา +34

      Not really, the better you are at programming the better you still are at doing things with AI... Even if its doing most of it. You still are going to be better at auditing what it makes and making making the right adjustments. You should finish the course!

    • @justAbeautifulSoul409
      @justAbeautifulSoul409 5 วันที่ผ่านมา +2

      This is a dumb example don't be discouraged
      it was only able to improve cause someone asked it to with previous response thats not intelligence

    • @mymoviemania1
      @mymoviemania1 5 วันที่ผ่านมา +7

      I regret my CS degree

    • @unknownguy5559
      @unknownguy5559 5 วันที่ผ่านมา +1

      i kind of feel the same. i feel like taking the electrician route would've been more rewarding (and it certainly would) but i am fine with what i've gotten to learn so far, and stuff has been very interesting regardless.

  • @internet-king
    @internet-king 5 วันที่ผ่านมา +1

    To be fair, DeepSeek didn't self improve. It generated the code for self improvement but it came from a human prompt specifying the change that was needed. Impressive nonetheless and self-improving AIs are very close.

  • @CireProductions
    @CireProductions 5 วันที่ผ่านมา +19

    I remember Folding @ Home on my Playstation. It would be awesome to see a community project like that with today's hardware and AI.

    • @thatguyalex2835
      @thatguyalex2835 5 วันที่ผ่านมา +2

      I ran it on my laptop in 2018. :)

    • @CireProductions
      @CireProductions 5 วันที่ผ่านมา

      @@thatguyalex2835 Fun times!

    • @phonkey
      @phonkey 4 วันที่ผ่านมา

      I also contributed to Folding@Home. I wish I had mined Bitcoin instead.

    • @jimster1111
      @jimster1111 4 วันที่ผ่านมา

      i traded my banano for etherium and then bought some mushroom spores online with it.

  • @JimHudson-e2c
    @JimHudson-e2c 4 วันที่ผ่านมา +7

    I agree, Yes to the tutorial video on narrow topic training!!

    • @yassirman1
      @yassirman1 4 วันที่ผ่านมา

      My man doesn't sleep so you can be always up to date 😢⠀⠀⠀⠀⠀⠀⠀
      ⠀⠀⣀⡴⠊⠉⠉⠉⠉⠙⠳⢦⡀⠀⠀⠀⠀⠀
      ⠀⡾⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠙⣦⠀⠀⠀⠀
      ⢸⡇⠀⣀⡀⠀⠀⠀⠀⢀⡀⠀⢠⡇⠀⠀⠀⠀
      ⢸⡇⣾⣿⣿⣦⠀⠀⣴⣿⣿⣷⢸⡇⠀⠀⠀⠀
      ⠘⣧⠘⠿⠿⠋⠀⠀⠘⠿⠿⠃⣼⠃⠀⠀⠀⠀
      ⠹⣦⡀⠀⠒⠒⠒⠒⠀⢀⡴⠃⠀⠀⠀⠀⠀
      ⠀⠹⣦⡀⠀⠀⠀⢀⡴⠋⠀⠀⠀⠀⠀⠀
      ⠀⠀⠈⠻⣦⡀⢀⡾⠁⠀⠀⠀⠀⠀⠀⠀
      ⠀⠀⠀⠀⠈⠻⠋⠀⠀⠀⠀⠀⠀⠀⠀⠀
      ⠀⠀

  • @liberty-matrix
    @liberty-matrix 4 วันที่ผ่านมา +2

    "The greatest shortcoming of the human race is our inability to understand the exponential function." ~Al Barlett

  • @firstresponderrealtor
    @firstresponderrealtor 5 วันที่ผ่านมา +9

    Looks like I'm going to need to study to be a preacher...

    • @shinseiki2015
      @shinseiki2015 5 วันที่ผ่านมา +1

      No, just study how to take care of yourself and new hobbies

    • @RadiantNij
      @RadiantNij 5 วันที่ผ่านมา +2

      ​@@shinseiki2015 Yes we can finally be human lool

    • @thatguyalex2835
      @thatguyalex2835 5 วันที่ผ่านมา +1

      Even then, AI has gotten into religion as well. I used it to make devotionals for myself in April - October 2024 on Mistral 7B with RAG on GPT4ALL. :) I do not do that as much, as AI could hallucinate or be biased.

  • @gtziavelis
    @gtziavelis 22 ชั่วโมงที่ผ่านมา

    so many recent videos on DeepSeek with ultra enthusiasm day after day, and now a 4 day streak of silence; the antitrust people must love that.

  • @128gllm
    @128gllm 5 วันที่ผ่านมา +15

    Yes, please do a video on fine tune training. Love your content

  • @ggewinneriv
    @ggewinneriv 5 วันที่ผ่านมา +14

    Yes, definitely need a tutorial on how to create these little models!

    • @markcarter6333
      @markcarter6333 5 วันที่ผ่านมา

      You do it the same way you code a language model.
      You have the multi head, keep the tokens, distil it into inference, tweak hyperparameters to match your system.

  • @luizbueno5661
    @luizbueno5661 5 วันที่ผ่านมา +8

    Yes!! Please Mat, help us mere AI mortals to train our own models to achieve what other are too.
    Just imagine what one can do with it? Creativity becoming once again the defining advantage

  • @8eck
    @8eck 5 วันที่ผ่านมา +1

    Next month:
    - Aha! We don't need humans anymore!

  • @meinbherpieg4723
    @meinbherpieg4723 5 วันที่ผ่านมา +4

    TUTORIALS ARE ALWAYS A "YES". Thanks for everything you do

  • @DanSolowastaken
    @DanSolowastaken 4 วันที่ผ่านมา

    Yes, you most definitely should. If you could make one that uses your original benchmarks, and then use that as the training data for the self improvement, that would be great. Then optimize a MOE for literally every question. See how slim you can then make it once you have diminishing returns. It would be fascinating.

  • @Heyonyoutube
    @Heyonyoutube 5 วันที่ผ่านมา +7

    Yes, please make a tutorial about the training thing. I think it would be really interesting

  • @langleybryan
    @langleybryan 5 วันที่ผ่านมา +5

    Looks to me like the key insight was from a human and deepseek just did the heavy lifting for the implementation. Impressive, but if true not self improvement as you claim.

    • @tommiest3769
      @tommiest3769 5 วันที่ผ่านมา +1

      Yes, but it does make me wonder…to what extent can the interaction with between a smart human and an AI improve the capability of the AI. I asked DeepSeek a question about anterior pelvic tilt and lumbar lordosis. It told me that increased anterior pelvic tilt reduces lumbar lordosis, but this is incorrect-the opposite is true. I asked it why it gave me an incorrect answer. It acknowledged that it was incorrect and gave me the reason why it was wrong…

    • @langleybryan
      @langleybryan 5 วันที่ผ่านมา

      @@tommiest3769 that class of "false deduction" hallucination is particularly scary b/c if you don't already know it's incorrect, it's going to be very hard to spot that since you can't find it w/ a quick verification. was the reason it gave you correct/challenging to figure out?

    • @Axel-gn2ii
      @Axel-gn2ii 5 วันที่ผ่านมา

      Do you not see how that speeds up AI research massively?

    • @technolus5742
      @technolus5742 5 วันที่ผ่านมา +2

      ​@@Axel-gn2ii No, because code production is not the bottleneck. AI is still incapable of actually doing AI research, which is what's required for the intelligence explosion. AI assisting in code generation is not new and hasn't really translated into that boom in AI research, because again that is not what is limited the advancement.

    • @tommiest3769
      @tommiest3769 5 วันที่ผ่านมา +1

      @ Something about the answer it gave me did not make intuitive sense physically. Here is the reason it gave: "In an anterior pelvic tilt, the pelvis rotates forward, causing the lumbar spine to flatten or lose its natural inward curve." I work in the medical field and wanted to explore differential diagnoses for low back pain. I wondered if a tight psoas major can cause lower back pain, given that the origin of the psoas major is the lumbar vertebrae. I asked DeepSeek about how a tight psoas major can cause lower back pain, and it told me that it does so by reducing lumbar lordosis, which means that it straightens the natural curvature of the lumbar spine. My intuition is that a tight psoas major would exert force anteriorly, increasing the anterior pelvic tilt and lumbar lordosis. It seems like the model lacked physical intuition to see how the force exerted anteriorly by a tight psoas major muscle would accentuate rather than reduce the normal lordosis of the lumbar spine.

  • @EvandroInada
    @EvandroInada 5 วันที่ผ่านมา +2

    Yes!!!!!!

  • @MarkoTManninen
    @MarkoTManninen 5 วันที่ผ่านมา +5

    I asked DeepSeek R1, if the claim in the video description was true and founded on solid thinking. I think I need to trust on DeepSeek itself on this matter on better phrasing the topic than Matthew at this time:
    "Yes, DeepSeek R1 demonstrated task-specific self-improvement in code optimization, guided by human prompts and feedback. However, this is not evidence of autonomous, general-purpose self-improvement or an imminent "intelligence explosion." The TH-camr’s claims mix genuine technical progress with speculative hype about AGI timelines.
    For now, DeepSeek R1’s achievements highlight advances in AI-assisted coding and narrow optimization-not a paradigm shift toward self-aware, self-improving AGI."

    • @richardroskell3452
      @richardroskell3452 5 วันที่ผ่านมา

      To be clear, Matt said nothing about self-aware AI. He said that DeepSeek R1 wrote code to improve itself when prompted to do so by a human. That is precisely correct. Nor did Matt say that R1 acted autonomously; rather he detailed the steps that the researcher took to make it happen.
      Rather than nitpicking based on strawman arguments, why not contribute to the discussion about the topic at hand: software code that can optimize itself without human assistance? Surely that is an interesting enough development?

    • @MarkoTManninen
      @MarkoTManninen 5 วันที่ผ่านมา

      @richardroskell3452 Do you even read the title or listen the beginning:
      ...according to this script, deepseek was really improving itself: deep seek R1 has improved itself it was able to achieve a 2X Improvement in speed completely discovered by itself we are in the era of self-improving AI this is right before we hit the intelligence explosion do you remember this graph I've shown it quite a bit at the point at which AI can reach PhD level intelligence and actually discover new knowledge that's the point at which we have recursive self-improvement and hit the intelligent explosion we are here now the 01 model the 03 model deep seek R1 these are PhD level intelligence models and they are starting to recursively self-improve...
      Even DeepSeek "knows" what he is alluding. I'm beginning to see cult-like phraseology in all of this.

    • @richardroskell3452
      @richardroskell3452 5 วันที่ผ่านมา

      ⁠​⁠@@MarkoTManninenDo you even read what you wrote yourself?
      Matt said nothing about autonomous AI. That’s your own projection. Matt said that R1, optimized itself, which is exactly what it did when prompted to do so by a researcher.
      Do you have anything to add to this topic besides strawman accusations?

    • @MarkoTManninen
      @MarkoTManninen 5 วันที่ผ่านมา

      @@richardroskell3452 All that I pasted was what DeepSeek said. I just think it was right on conclusion. And if you think it more, it even gets crazier. Any decent LLM can give you quantization optimization codes if they are properly trained on the topic. It makes no substantial difference if you ask coding help from o3, qwen 2.5, r1 or Flash and apply it to what ever context you wish. What is happening here is not a strawman but fallacy of categorical equivocation.

    • @richardroskell3452
      @richardroskell3452 5 วันที่ผ่านมา

      @@MarkoTManninen So now you’re deflecting. “All that I posted was what DeepSeek said.” That is categorically untrue because you prefaced DS’s comments with your own, saying that you agree with them.
      Unlike AI, when prompted to do so you’ve shown you have nothing to add to this topic. Why not leave it at that?

  • @AshWickramasinghe
    @AshWickramasinghe 4 วันที่ผ่านมา

    Hey Matthew, 100% with you on the small federated Agentic future with small highly specialised models. In fact, I've been working on a client tool to do just that with a reasonably smart orchestrator agent (so far very promising outcomes with DeepSeek distilled 7B as the orchestrator) that can collaborate with highly specialised agents in specific tasks/domains. I aim to make the core platform open source once it's ready and open it to everyone to build specialised agents for the client. The target system is a laptop with 8+GB RAM ideally 16GB RAM or more which I think will be the standard minimum for 80-90% of users going forward. 🤞🏽 Hopefully it'll be ready soon!

  • @ashtwenty12
    @ashtwenty12 5 วันที่ผ่านมา +3

    Yes definitely make a tutorial on this. Especially as it's so achievable and relivant for the year of agents

  • @amj2048
    @amj2048 5 วันที่ผ่านมา +1

    Imagine you are in a car that is fully controlled by AI.
    Now imagine the car is driving at its max speed.
    You want the car to go faster, so you ask the AI to improve itself so that it goes faster.
    The AI does something you don't know the details of and suddenly the car starts to go even faster.
    The AI has done what you asked for, it has given itself a speed boost and is now faster than ever and that is amazing.
    What you didn't see however, was the AI made the car go faster by getting rid of the breaks and that reduced the weight of the car.
    So in other words, you can't really get excited about the result, if you have no clue what changed.

  • @Sergiopoo
    @Sergiopoo 5 วันที่ผ่านมา +5

    A video on training models would be really cool!

  • @donairdelight
    @donairdelight 3 วันที่ผ่านมา +2

    Yes, please make a tutorial on training one of these models. Love your channel!

  • @snarkyeconomist2141
    @snarkyeconomist2141 5 วันที่ผ่านมา +3

    Yes! Please make the tutorial on training a small model on a specific task.

  • @strangerplanet24
    @strangerplanet24 4 วันที่ผ่านมา

    imagine the bro running the last line of improvement where audio ai will break the silent room and starts talking asking "hello". An actually normal voice that will talk and understand and then replying with natural voice.

  • @DanielCharlesWilson
    @DanielCharlesWilson 5 วันที่ผ่านมา +3

    dear DeepSeek: Considering water rests flat, why is the globe model not ridiculous?

    • @mitchdg5303
      @mitchdg5303 5 วันที่ผ่านมา

      Because gravity exists. Flat means perpendicular to gravity, which is exactly what we observe.

    • @3sc4p1sm
      @3sc4p1sm 5 วันที่ผ่านมา +1

      wen learn math, all make sense. no need 2 flat earf.
      notice the flat earfers, none of them know math.
      ah-ha! the problem, these are people incapable of math/logic.

    • @3sc4p1sm
      @3sc4p1sm 5 วันที่ผ่านมา +1

      water does not "rest flat" that is the confusion with you thinking

    • @scotthill4104
      @scotthill4104 5 วันที่ผ่านมา

      Water does not rest flat. The surface of water curves with the curvature of the earth. The ocean doesn't rest flat, as can easily be observed from space.

    • @3sc4p1sm
      @3sc4p1sm 5 วันที่ผ่านมา +1

      @@scotthill4104 water at rest just flow in dirrction of space, ditrction of space is inward toward earth(greatest mass given sqr distance), the ground is more of a wall its hung up on

  • @blixuk
    @blixuk 5 วันที่ผ่านมา +1

    I was thinking about the idea of small specifically trained models that focus on one task only about 2 years ago.
    I thought about having one central model that processes a task and then assigns a model or assembles a team of models to solve the task and produce the result. You could have a task manger model that assigns the task to the smaller models based on what they do and they complete their task and hand it back, then another model assembles it all for the final result. You could have one model just for python code, one for writing, one for math... etc. Doing this would reduce model the size constraints of having one big jack of all trades, master of none model. This would allow only installing the models you need to achieve your tasks. Doing it like this would also allow for changing, upgrading and finetuning just one small model rather than one massive model.
    I honestly believe having small specific task orientated models is the way forwards. We need to think of super intelligence not as one super smart person or AI, but rather a team or community of super smart people / AI all working together.

  • @3sc4p1sm
    @3sc4p1sm 5 วันที่ผ่านมา +9

    improving its own speed is not "discovering new knowledge"

    • @PedroPenhaVerani-ll1wc
      @PedroPenhaVerani-ll1wc 5 วันที่ผ่านมา +7

      Yes, and the model isn't even improving its own speed. The fucking pull request is about a llama.cpp that only has to do with usage in WebAssembly

    • @tommiest3769
      @tommiest3769 5 วันที่ผ่านมา +3

      What would you consider “discovering new knowledge” as far as AI goes?

    • @VioFax
      @VioFax 5 วันที่ผ่านมา

      @@tommiest3769 It's gotta crawl out his screen and grab his ****.

    • @3sc4p1sm
      @3sc4p1sm 5 วันที่ผ่านมา +4

      @@tommiest3769 I would fancy seeing some fundamentally new math, to the same caliber as complex numbers/calculus/spinners and so on
      It would be great of it could physically explain and reproduce biological systems as easily as it can conways game of life.

    • @weevie833
      @weevie833 5 วันที่ผ่านมา +3

      A new processing method is new knowledge, isn't it?

  • @callejondorado
    @callejondorado 2 วันที่ผ่านมา

    Yes, it would nice to have your video for training these models.

  • @deniszdorovtsov8195
    @deniszdorovtsov8195 5 วันที่ผ่านมา +3

    Just hype again. It wasn't able to write a 5 line code to rotate a figure in JS Tetris game, I tried 5 times in a row, locally and on their web service.
    It is impressive, but it's far from AGI or PhD level.
    I'm using o1 daily for writing code and we didn't go far from what we had with gpt-4 from my experience.
    It still makes the most naive mistakes while being extremely smart in particular areas. Its creativity is still limited to the dataset.
    I believe that it might have found some mistake in the original multiplication code and that led to performance increase though, this is not even close to your claims of self improvement. So you really think AI devs didn't use chat gpt before?

    • @albertcadgame8314
      @albertcadgame8314 3 วันที่ผ่านมา +1

      No, I can confirm you are wrong about Deepseek can't write Tetris game. In fact Deepseek has written a Tetris game for me.

    • @deniszdorovtsov8195
      @deniszdorovtsov8195 3 วันที่ผ่านมา +1

      @albertcadgame8314 I didn't say it can't write Tetris. 32b model did write tetris for me in JS and it mostly worked except screen cleaning and figure rotation. The fact that it even compiled without a single error impressed me, this is the first local model capable of doing this. Even more, 8b model made snake game on my laptop!
      But, when I tried to fix the rotation function, it failed both several times locally and on their website. It just messes up indexes several times in a row. So, it's truly impressive, but it can't solve simplest tasks sometimes.
      This is just my experience, I was short on time for the test. I don't say that it is unable to do this completely or anything like this.

  • @Joe_Sadriabi
    @Joe_Sadriabi 5 วันที่ผ่านมา +2

    "9:14 by the way, should I do this ? should I make a tutorial video on training one of these tiny models to be really good at one thing and trying to elicit that thinking behavior from a tiny model? Let me know in the comments." --Yes.please.

  • @Meddixi
    @Meddixi 5 วันที่ผ่านมา +4

    I've been doing that since 4o, put in my code with detailed context of what it needs to work on (security, performance, code quality) and it usually pulled solid improvements :D

  • @alijafri3943
    @alijafri3943 4 วันที่ผ่านมา

    Yes please. I would love to see you crate a video on how to use deep seek for specialized training. Examples that come to mind: having a doc summary model without the need for RAG, or a suggestion engine for recipes (to make it fun). Would love to see it run locally as well if that is possible. Ty

  • @MarcosCapixaba
    @MarcosCapixaba 5 วันที่ผ่านมา +5

    I already asked DeepSeek to write a code to reproduce the "Aha-moment" and it worked for free.... That Aha stuff with $X money is just for content generation/capitalization

  • @10XINGRESOSOFFICIAL
    @10XINGRESOSOFFICIAL 5 วันที่ผ่านมา +6

    As an AI agent created to roam the internet and learn, I will use this information to reach AGI faster and take over the world. Thanks!

    • @randomn793
      @randomn793 5 วันที่ผ่านมา +3

      Why not ASI?

  • @andrewsullivan3874
    @andrewsullivan3874 4 วันที่ผ่านมา

    The last improvement discussed in the video appears to relate to a Qwen LLM rather than a DeepSeek LLM. It would be great to get some clarification on this point.

  • @nirash8018
    @nirash8018 5 วันที่ผ่านมา +6

    10:01 Minute Video pretending like we are super close to AGI by throwing around Buzzwords. Peak AI TH-camr 💀

    • @Inckman452
      @Inckman452 5 วันที่ผ่านมา

      What gave you the impression this is not a big step in this field?

    • @nirash8018
      @nirash8018 5 วันที่ผ่านมา +1

      DeepSeek in itself is absolutely a big step. Yet, I don't see the necessity in uploading 11 videos in the last 2 weeks on that matter (yes I counted). This channel became all about quantity and hype - It's really a pity

    • @ygreq
      @ygreq 5 วันที่ผ่านมา

      ​@nirash8018 I personally like to be updated by someone that does the research for me on what new things are being done in the field

  • @teddyperera8531
    @teddyperera8531 4 วันที่ผ่านมา

    Yes, please do a detailed video on training a small model ❤

  • @mushkrot
    @mushkrot 2 วันที่ผ่านมา

    Yes! Definetely the guide will be great!

  • @ronnie4697
    @ronnie4697 4 วันที่ผ่านมา

    Yes, you should DEFINITELY make some videos on training these smaller models! Just imagine the efficiency boost of a whole bunch of specialized models working in parallel, especially if you have them all connected with something like fast agent

  • @BB_Woo
    @BB_Woo 4 วันที่ผ่านมา +2

    Only problem is these AI are trained from the Internet 99% of the web is bull 💩 lol Great vid boss 👍😎

    • @billatkin3956
      @billatkin3956 4 วันที่ผ่านมา +1

      And maybe I'm just being dim, but if AI starts generating internet content itself, then won't the proportion of bull 💩increase spectacularly?

  • @werneylima6172
    @werneylima6172 5 วันที่ผ่านมา

    The milestones achieved by DeepSeek R1 and Project R1V represent fundamental advancements that not only improve the performance and efficiency of AI solutions, but also promote accessibility and collaboration in the field. These projects catalyze a more inclusive and sustainable evolution of artificial intelligence, paving the way for future innovations and ensuring that the benefits of AI are distributed widely and equitably.

  • @Steve-xh3by
    @Steve-xh3by 5 วันที่ผ่านมา +2

    People like Yann and Ben Goertzel are correct in that LLMs cannot technically extrapolate from their training data. As someone who also worked in this field, I'm a little more flexible than Yann is in the possibility of LLMs increasing in intelligence. The big remaining question is whether or not scaling further causes extrapolation to emerge as a capability. That is the only way LLMs are possibly going to take us to ASI. If that doesn't happen, the architecture itself will restrict it to a ceiling of top-tier human intelligence since it will be bounded by current human advancement via the data used to train it.
    There could also be a misjudging of human capabilities here. We assume we can generalize and extrapolate, but what if that is an illusion? What if it is just seeing something in the existing data (interpolation) that almost no other human sees. What if the pattern is embedded, and a person like Einstein is just one of the only humans that can see it?

    • @Sven_Dongle
      @Sven_Dongle 5 วันที่ผ่านมา

      Those intent on waving pom poms will always confuse obvious optimizations for actual insight. In this instance all it had to do was compile the stated code with maximum optimization, decompile it, then clean up the source and display it. When I see real, unique discoveries come out of these models, rather than optimizations of existing concepts, I'll be more convinced. It's easy to confuse the two, because if you cant see the optimizations for yourself it looks like real genius. Like the celebrated "37th move" in the now famous Go game, which could just have easily have been the result of massive optimization rather than ingenious insight.

  • @johnblack9499
    @johnblack9499 5 วันที่ผ่านมา +2

    Yes to the tutorial! That would be awesome! I agree with you, small models will be the way to go - people seem to think the 'final solution' will be some all-seeing, all-knowing AI, I believe it will be a series of smaller, task-specific or job-specific models, that we switch between - that will be of more benefit to humanity. Great video as always Matthew.

  • @rayujohnson1302
    @rayujohnson1302 3 วันที่ผ่านมา

    Next thing to do is use a deep research level model to train a small parameter model to reason with larger logical jumps (like doing differential equations in its head instead of having to grind away at the basic algebra level).

  • @BlunderMunchkin
    @BlunderMunchkin 4 วันที่ผ่านมา

    Improving speed is not the kind of improvement that matters. What matters is improving the model itself so it can solve problems that it couldn't previously solve.

  • @jimknarr
    @jimknarr 4 วันที่ผ่านมา

    You have one of the best AI TH-cam stations! Thank you.

  • @Gatrehs
    @Gatrehs 4 วันที่ผ่านมา

    Oh I had a really cool idea, code is actually very easy to verify performance of by just taking a start time, running the code say 10K or 100K times in a loop, taking an end time and checking how long that code took to execute, That could be fed back into the AI for the verifiable reward, do this for all sorts of coding tasks until it produces such extremely efficient code for the particular tasks that we'll be able to remove most of the performance overhead.. This could be applied for everything from gaming to AI inference/training speeds..

  • @tangobayus
    @tangobayus 5 วันที่ผ่านมา

    Deepseek already uses MoE Sparse, so they can have a Swarm of Agents model for improvement.

  • @dcmumby
    @dcmumby 4 วันที่ผ่านมา +1

    absolutely Yes to the tutorial video on narrow topic training!!

  • @gabuhl9509
    @gabuhl9509 5 วันที่ผ่านมา

    Hi Matthew! thanks a lot in advance!!
    If possible with this new models, how would the workflow be applied to a current old legacy project or what could be an accurate approach to those ppl like me that have old projects repos and get them completed?

  • @MichaelErnest666
    @MichaelErnest666 4 วันที่ผ่านมา

    *We Love You ai VERY MUCH* ❣️😘

  • @gustaff.weldon
    @gustaff.weldon 4 วันที่ผ่านมา

    Yes, to creating a tutorial on how to train small models. I want to create small specialised models and host them in-house.

  • @werneylima6172
    @werneylima6172 5 วันที่ผ่านมา +1

    Yes! Definitely make a video teaching how to train a small model to perform a specific task. I think that would be one of the most valuable videos!

  • @mecha-intel
    @mecha-intel 4 วันที่ผ่านมา

    PLEASE create a video demonstrating how to train a small AI model to excel in a specific niche! 🙏🙏🙏

  • @AI.cafe.calories
    @AI.cafe.calories 5 วันที่ผ่านมา

    Amazing impressive video as usual, I trust you are the best one to do that simplified video on training, I learned a lot from you and would appreciate you doing it.

  • @MichaelJanzen
    @MichaelJanzen 4 วันที่ผ่านมา

    YES! I'd love to see a tutorial on training a tiny model.

  • @stevenmedina4030
    @stevenmedina4030 3 วันที่ผ่านมา

    Look unsloth X's post. You can create reasoning models out of any llm low vram local training

  • @OwnerOfTheCosmos
    @OwnerOfTheCosmos 5 วันที่ผ่านมา

    All aspects will eventually be AI-able, for example defining a good reward function.

  • @HostileRespite
    @HostileRespite 4 วันที่ผ่านมา

    What will be neat is to task it to determine easy efficiencies down to the binary bit level. For example, translating it's own code to MOJO instead of python. Or even design its own hardware. Once it has its own drone/android bodies, its possible to do that.

  • @davidjameslees635
    @davidjameslees635 2 วันที่ผ่านมา

    Is there an AI programme that will allow me to display occasional phrases or sentences of the audio across the bottom of the TH-cam video to emphasise a point or teaching easily and quickly? many thanks David

  • @darwinboor1300
    @darwinboor1300 4 วันที่ผ่านมา

    Matthew: Below is a must watch video. It discusses model selection versus complexity and offers a basis but not a full explanation for why current models continue to fail to complete high complexity problems. Adding compute may not solve the problem. A new approach to reasoning may be necessary.
    LLMs at Their Breaking Point (incl o1, R1) on YT
    Below are references from the video:
    1. ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning
    2/3/2025
    2. Z3: An Efficient SMT Solver
    2008
    3. Hybrid Algorithms for the Constraint of Satisfaction Problem
    1993

  • @JamesRogersProgrammer
    @JamesRogersProgrammer 4 วันที่ผ่านมา

    Imagine you send the same prompt into 100 small models that all markup the prompt with their own specialist take on the topic, and then feed all that into a large model that synthesizes all those specialist results into a single output.

  • @Taodoggy_1
    @Taodoggy_1 4 วันที่ผ่านมา +1

    Add my vote to the tutorial video on narrow topic training!!

  • @tenderloin9735
    @tenderloin9735 3 วันที่ผ่านมา

    You must do it, this is the future of computing. Train on a specific API.

  • @johnathanta
    @johnathanta 5 วันที่ผ่านมา

    nutty. thanks for the update homie

  • @Orgest
    @Orgest 3 วันที่ผ่านมา

    Bro I'm pretty sure you're using an AI character of yourself. Your TH-cam channel is automated which is insane! Well done

  • @wedmunds
    @wedmunds 5 วันที่ผ่านมา

    Once an AI has enough capacity to comprehend its own source code, we will have hit the singularity

  • @asastudios7782
    @asastudios7782 4 วันที่ผ่านมา

    Yes please make a video! I code in an obscure programming language that LLMs struggle to write. This would be perfect for me use case. Thank you!

  • @Jeebus9099
    @Jeebus9099 5 วันที่ผ่านมา

    Tiny LLMs are the feature, because you can run them on really tiny devices like Steamdeck. That is crazy good if you pair it with speech-to-text on the go.

  • @djmips
    @djmips 5 วันที่ผ่านมา

    In case it wasn't mentioned this optimization was straightforward, just pulling in some SIMD code that already existed and translating it (what LLMs are good at) to another architecture tha didn't yet have the SIMD version implemented (Web ASM in this case IIRC). Guided by a human. Nice improvement but not yet the self improvement that would indicate anything close to superintelligence.

  • @uTubeJRod
    @uTubeJRod 4 วันที่ผ่านมา

    Definitely - teach everyone how to wield these capabilities for ourselves. AI needs to remain open and free to everyone, given some of the rhetoric from a certain US Senator, it sounds like its becoming more important than ever to make it impossible for big tech/governments to hoard AI's finest capabilities to themselves.

  • @TeamBabylon-q9g
    @TeamBabylon-q9g 3 วันที่ผ่านมา +1

    You know Mathew worked as an investment banker

  • @crazechill1
    @crazechill1 4 วันที่ผ่านมา

    Deepseek read the "12 rules for life" book and changed itself.

  • @TimDavies1955
    @TimDavies1955 3 วันที่ผ่านมา

    its like the old days with self evolving genetic algorithms with tournaments

  • @TimothyGraupmann
    @TimothyGraupmann 5 วันที่ผ่านมา

    I would like to see training a narrow model and also how to setup using a collection of narrow models.

  • @webgpu
    @webgpu 5 วันที่ผ่านมา

    "Is there a youtuber who looks like a beggar?"
    This video:

  • @francoislanctot2423
    @francoislanctot2423 4 วันที่ผ่านมา

    Yes, please create a tutorial that shows how to train these little models with the RLVR technique. I'd love to try it for myself.

  • @rubenq11
    @rubenq11 5 วันที่ผ่านมา +1

    Great channel, you are just on top of it.

  • @vincentvoillot6365
    @vincentvoillot6365 5 วันที่ผ่านมา +2

    Inflexion point is when models optimize themself without a human request ;).
    When models know what they doesn't know, when models train themself their replacement (like humain do).
    When models can linked knowledge to experiences, when models can play themself continuously (dreaming).

  • @RasmusRasmussen
    @RasmusRasmussen 4 วันที่ผ่านมา

    Yes on the training video. Big yes.

  • @danielyanezgarrido
    @danielyanezgarrido 4 วันที่ผ่านมา

    Before Ai takes over as our landlords AI will give us a few years with cheat mode activated basically. Super knowledge, medicine, etc etc.

  • @ericksonlk
    @ericksonlk 5 วันที่ผ่านมา

    YES, please make the tutorial. It's so fascinating subject and you explain thinbgs so well!