ไม่สามารถเล่นวิดีโอนี้
ขออภัยในความไม่สะดวก

The Least Squares Formula: A Derivation

แชร์
ฝัง
  • เผยแพร่เมื่อ 4 มี.ค. 2017
  • bit.ly/PavelPa...
    lem.ma/LA - Linear Algebra on Lemma
    bit.ly/ITCYTNew - Dr. Grinfeld's Tensor Calculus textbook
    lem.ma/prep - Complete SAT Math Prep

ความคิดเห็น • 85

  • @MathTheBeautiful
    @MathTheBeautiful  3 ปีที่แล้ว +2

    Go to LEM.MA/LA for videos, exercises, and to ask us questions directly.

  • @cern1999sb
    @cern1999sb 3 ปีที่แล้ว +46

    It would be great to have the second part of this with an audio recording of the explanation. Everything was crystal clear until the mic died

  • @ForcesOfOdin
    @ForcesOfOdin 2 ปีที่แล้ว +5

    There's a timeless beauty to the cadence , passion, and clarity of your lecture series. Glad I could find it again. Different goals now, but same appreciation.

  • @jonathanpepin6326
    @jonathanpepin6326 7 ปีที่แล้ว +20

    Thanks à lot for your videos. I am à french Student in mathematics and your way of explaining things is so different from here but it all makes more sense. Look forward to the next videos.

  • @evieblue959
    @evieblue959 3 ปีที่แล้ว +6

    An energetic math teacher. I've never met one in the wild. Thanks!

  • @muratcan__22
    @muratcan__22 5 ปีที่แล้ว +44

    the most important part is missing

    • @vangrails
      @vangrails 5 ปีที่แล้ว +1

      What is that part?

    • @dcuzhang
      @dcuzhang 4 ปีที่แล้ว +6

      @@vangrails battery😂

  • @damian.gamlath
    @damian.gamlath 6 ปีที่แล้ว +7

    Sir, I love the intuition you inject into these topics, something most other teachers (and TH-camrs for that matter) fail to provide.

  • @javierapuebla2451
    @javierapuebla2451 2 หลายเดือนก่อน

    this video is marvelous from start to finish, and the mute subtitles did really help

  • @orangeflow9809
    @orangeflow9809 4 ปีที่แล้ว +4

    Even with the mic batteries dead, this was a great lecture.

    • @MathTheBeautiful
      @MathTheBeautiful  4 ปีที่แล้ว +3

      ..................................................

  • @yamansanghavi
    @yamansanghavi 6 ปีที่แล้ว +4

    I wish there were more professors like you sir. You are truly genius. Thank you so much.

  • @mpja
    @mpja 4 ปีที่แล้ว +2

    The formula I haven see uses the complex conjugate transpose instead of the plain transpose. For real values the two are equivalent but I would make a note of this.

  • @via6400
    @via6400 ปีที่แล้ว +1

    Tank you very much, You solved my least squares problem.

  • @darrenpeck156
    @darrenpeck156 2 ปีที่แล้ว +2

    Please record the next video with sound.

  • @BraddockGaskill
    @BraddockGaskill 2 ปีที่แล้ว +2

    Yes please re-record the audio, it was a brilliant explanation up until then

  • @pipertripp
    @pipertripp 2 ปีที่แล้ว +2

    You should have done the second half of the video in sepia. A moustache would have been a nice touch too.

  • @Acampandoconfrikis
    @Acampandoconfrikis 6 ปีที่แล้ว +26

    What an epic way do deal with a microphone problem XXDDXXDXD (also good explaining, thanks!)

  • @grjesus9979
    @grjesus9979 3 ปีที่แล้ว +3

    4 years later he is still busy hahahahaha. Good class except fot the last 2:30 where magic happens

  • @hushengli6697
    @hushengli6697 ปีที่แล้ว +1

    wonderful!

  • @abdelrahman458
    @abdelrahman458 4 ปีที่แล้ว +1

    great work very good lecture , but i wished there was an order or number of the lecture so i can visit the previous ones if i dont understand an certain concept

  • @vikraal6974
    @vikraal6974 หลายเดือนก่อน

    This feels like a Charlie Chaplin movie after mic died.

  • @jozeflodeweyckx4009
    @jozeflodeweyckx4009 6 ปีที่แล้ว +2

    Interesting way of explaining the subject.Could I get a pdf file of the story?Jozef

  • @gautamrajeev3345
    @gautamrajeev3345 3 ปีที่แล้ว +1

    Why are they equal because they are transposes of each other?

  • @shimaalcarrim7949
    @shimaalcarrim7949 10 หลายเดือนก่อน

    Thanks

  • @oscarhernandezarzeta6222
    @oscarhernandezarzeta6222 6 ปีที่แล้ว +1

    Thank you very much, yur videos are great, they give me a pretty good insight of the subject, please upload this video again :D

  • @kallzvx858
    @kallzvx858 4 ปีที่แล้ว +2

    Great video, but I cant understand the end without knowing what the lecturer is talking...

  • @97Alfinsyah
    @97Alfinsyah 6 ปีที่แล้ว +1

    as geodesy and geomatics engineering student i always using the least square adjustment to solve problems :D thank your sir for the lecture he he he

  • @unrmaestro
    @unrmaestro 3 ปีที่แล้ว +1

    ty sire

  • @chensun2427
    @chensun2427 3 ปีที่แล้ว +1

    Ah Noice! A Math lesson in Bioshock style xD

  • @kyang1305
    @kyang1305 5 ปีที่แล้ว +2

    I was lost from the 'A', 'b' part... like we discovered a similar structure but how did we directly got the value of x...?

    • @MichaelStangeland
      @MichaelStangeland 5 ปีที่แล้ว +11

      Yeah, me too at first. On the left of the board there's something previously covered about "Quadratic Form Minimization", and that operation parallels differentiation in calculus. Note that to find the x that minimizes rᵀr, you take the derivative WRT x and solve for the x that makes that derivative equal to zero. It looks like he compares the xᵀAx to ax² and illustrates how they become 2Ax and 2ax respectively.
      So he goes from rᵀr = 2(½xᵀAᵀAx - xᵀAᵀb) + bᵀb
      to 0=2(AᵀAx - Aᵀb)
      to AᵀAx = Aᵀb
      to x = (AᵀA)⁻¹ Aᵀb

    • @MichaelStangeland
      @MichaelStangeland 5 ปีที่แล้ว

      th-cam.com/video/oaiiyIsbNdI/w-d-xo.html

    • @muhammadhafiezd2514
      @muhammadhafiezd2514 4 ปีที่แล้ว

      @@MichaelStangeland Thanks, this is helpful

    • @nahblue
      @nahblue 11 หลายเดือนก่อน

      ​@@MichaelStangeland Thanks, this was the missing step in the video. Would you agree to say that we differentiate w.r.t xᵀ? Maybe it truly doesn't matter, one can always rearrange.
      And we use the fact that AᵀA is symmetric when we work with differentiation this way.

  • @ahmedhemani1259
    @ahmedhemani1259 6 ปีที่แล้ว +1

    Can someone point to me where are the rest of the videos of this course available ?

  • @visbellum
    @visbellum 3 ปีที่แล้ว +1

    What about the error?

  • @johnfykhikc
    @johnfykhikc 6 ปีที่แล้ว +1

    didn't get the final step o f the proof. anyone found the video?

  • @edsonsabino
    @edsonsabino 7 ปีที่แล้ว +1

    very nice!

  • @andrerossa8553
    @andrerossa8553 5 ปีที่แล้ว +1

    very funny! Tks a lot for such a great lecture

  • @jimturner4937
    @jimturner4937 3 หลายเดือนก่อน

    where is the re-recorded version

  • @ManojGupta-tb3ei
    @ManojGupta-tb3ei ปีที่แล้ว

    Sir
    As we express a length as xTx
    How can express a volume in matrix entry

  • @daniel.fajtai
    @daniel.fajtai 4 ปีที่แล้ว

    modern problems require modern solutions

  • @FsimulatorX
    @FsimulatorX 6 ปีที่แล้ว +5

    What is everyone here studying? I just came here because of an equation I saw for Linear Regression XD

    • @israelmcintosh9745
      @israelmcintosh9745 5 ปีที่แล้ว +1

      @Max Kirkman linear algebra

    • @flightrisk7566
      @flightrisk7566 4 ปีที่แล้ว +2

      I’m actually here for NLP

    • @TheTacticalDood
      @TheTacticalDood 4 ปีที่แล้ว +2

      Computer Vision

    •  4 ปีที่แล้ว +2

      @@flightrisk7566 NLP x2

    • @edilgin622
      @edilgin622 3 ปีที่แล้ว +1

      regression

  • @devoglumehmet4941
    @devoglumehmet4941 6 ปีที่แล้ว +1

    Great !

  • @MyJuicehole
    @MyJuicehole 7 ปีที่แล้ว +1

    I think the word you were looking for was "stamp"

  • @matthewjames7513
    @matthewjames7513 2 ปีที่แล้ว

    at 8:41 can somone help me undersatnd why xTATb = bTAx ? I can understand why xTATb = (bTAx)T, but where did the big transpose go?

    • @matthewjames7513
      @matthewjames7513 2 ปีที่แล้ว +1

      Oh I think I get it now. Since xTATb = is a 1x1 matrix, and since the transpose of a 1x1 must be the same, then you can have the big transpose or ignore it, it doesn't matter in this case!

    • @matthewjames7513
      @matthewjames7513 2 ปีที่แล้ว

      But why at 9:43 is ATA guaranteed to be invertible ? :O

    • @HelloWorld-dq5pn
      @HelloWorld-dq5pn ปีที่แล้ว

      @@matthewjames7513 AtA is invertible since it is positive definite, therefore, all of its eigenvalues are greater than zero(and real since its symmetric). Note that AtA is symmetric, then consider the magnitude square of Ax, expand it as a dot product and take x to be an eigenvector of A.

  • @roflchopter11
    @roflchopter11 6 ปีที่แล้ว +6

    Slow until the audio cut out, after which the interesting part takes place. This could have been really good, but the audio problem was not handled well. "What a beautiful equation" for 20 seconds the end immediately after the solution was some frustrating editing.
    Keep it up, but that's some constructive criticism

  • @TuNguyen-ox5lt
    @TuNguyen-ox5lt 6 ปีที่แล้ว

    what if ATA is not positive definite (that leads to the rTr does not have minimum value because we can choose x to make rTr as small as possible ) and if ATA is not invertible ? . Please help me .

    • @MathTheBeautiful
      @MathTheBeautiful  6 ปีที่แล้ว +1

      Please see this video: th-cam.com/video/bp38BKP-xh4/w-d-xo.html

    • @TuNguyen-ox5lt
      @TuNguyen-ox5lt 6 ปีที่แล้ว

      So ATA is always positive semi-definite . if A is invertibe so that ATA is positive definite and we can find a global minimum of rTr . If A is singular , ATA will not invertible and the equation resulting from derivative ATAx = ATb can have multiple x to satisfy . As a result rTr will have multiple local optima . That 's what my thoughts . Am I correct ? .
      Thank you very much for quick response .

    • @MathTheBeautiful
      @MathTheBeautiful  6 ปีที่แล้ว +1

      You are correct. More specifically, it will be one minimum value, but it will attained at a whole subspaces of locations. Sort of like the function f(x,y)=x^2.

    • @TuNguyen-ox5lt
      @TuNguyen-ox5lt 6 ปีที่แล้ว

      I understand your example of f(x,y) = x^2 . The minimum subspaces in this example is a line x = 0(y whatever ) . If we randomly choose location of x and apply gradient descent , do we always certainly get to the minimum subspaces ? Does all optimization problem that can be expressed in quadratic form always have just one minimum value ? (I think yes but I still want a verification )
      For other cases that are not a quadratic form , the function will have multiple local optima and the value of function of those local optima are different . If I use gradient descent with randomly choosing the starting point of x , it will lead to different local optima ( in case of luck , we get to the global optima ). So the position of starting x is important . Is there any way that I surely can come to global optima instead of getting stuck of local optima . My idea is choosing multiple starting position x and the get the x that have f(x) minimum . But this way also depends on luck and not solve the situation completely . Can you come up with a good solution for this problem ?

  • @jimmyjonestodd2556
    @jimmyjonestodd2556 2 ปีที่แล้ว

    Damn the mic dying!!!!

  • @darrenpeck156
    @darrenpeck156 ปีที่แล้ว

    Please rerecord or talk over!!!! Please!!!

  • @demidrol5660
    @demidrol5660 4 ปีที่แล้ว

    like it

  • @channelpanel8259
    @channelpanel8259 6 ปีที่แล้ว +2

    NOOOO !!! This is horrible!!! This was shaping up to answer all my nagging questions about least squares :( :(

    • @MathTheBeautiful
      @MathTheBeautiful  6 ปีที่แล้ว +1

      Give us one of those nagging questions.

    • @JohannesSchmitz
      @JohannesSchmitz 6 ปีที่แล้ว

      MathTheBeautiful in the last step did you take the first derivative and set it to zero to find the minimum of r^Tr?

  • @droully
    @droully 7 ปีที่แล้ว

    stump it out

  • @FB-tr2kf
    @FB-tr2kf 6 ปีที่แล้ว +2

    Gave you a dislike because of the sound towards the end.

    • @rovshanabdurrahimov5892
      @rovshanabdurrahimov5892 6 ปีที่แล้ว +2

      but it is not his mistake. u should be respectful. if u dont like go other channels. this man explains in best way.

  • @aboodfarhood9316
    @aboodfarhood9316 3 ปีที่แล้ว +1

    Math is super fun, but when we listen to about one xillion detail at every corner? It becomes so boring.

    • @MathTheBeautiful
      @MathTheBeautiful  3 ปีที่แล้ว

      I think it's the one zillion first detail that one too many!

  • @difjaoisdjfoaisjdfoaiogeis
    @difjaoisdjfoaisjdfoaiogeis 5 ปีที่แล้ว

    i was enjoying it until the CHALK SCREECHED AGAINST THE CHALKBOARD UNGHGHHGH IM SO UNCOMFORTABLE NOW

  • @ozzyfromspace
    @ozzyfromspace 4 ปีที่แล้ว +2

    Geez my guy, for every 5 seconds of math, there's like 2 minutes of you talking about "stamping it out". It's painful to watch, sorry.

  • @MLDawn
    @MLDawn 4 ปีที่แล้ว

    Is this a joke?

  • @_random_dude
    @_random_dude 3 ปีที่แล้ว

    The guy talks too much.