Patrick Kidger
Patrick Kidger
  • 4
  • 23 015
On Neural Differential Equations
I was invited to give a talk on Neural Differential Equations at the amazing Scientific Machine Learning seminar, run by CMU. In this talk I discuss:
- Recent work on neural SDEs;
- Software for diffeqs, and in particular Diffrax;
- Background on neural ODEs and neural CDEs;
- My doctoral thesis -- a textbook for neural differential equations.
Links:
- (Doctoral thesis) On Neural Differential Equations: arxiv.org/abs/2202.02435
- (ICML) Neural SDEs as Infinite-Dimensional GANs: arxiv.org/abs/2102.03657
- (NeurIPS) Efficient and Accurate Gradients for Neural SDEs: arxiv.org/abs/2105.13493
- (Software for JAX) Diffrax: github.com/patrick-kidger/diffrax
มุมมอง: 12 272

วีดีโอ

[13 minutes] Universal Approximation with Deep Narrow Networks
มุมมอง 2.1K4 ปีที่แล้ว
The classical universal approximation theorem (dating back to ~1999) is a foundational result, giving (part of) an answer to the question "why do neural networks work?" This is a brief introduction to our recent paper which proves some natural "dual" theorems on universal approximation, and in particular highlights a difference between deep and shallow neural networks. Paper: arxiv.org/abs/1905...
[50 seconds] Universal Approximation with Deep Narrow Networks
มุมมอง 7924 ปีที่แล้ว
The classical universal approximation theorem (dating back to ~1999) is a foundational result, giving (part of) an answer to the question "why do neural networks work?" This is a brief introduction to our recent paper which proves some natural "dual" theorems on universal approximation, and in particular highlights a difference between deep and shallow neural networks. Paper: arxiv.org/abs/1905...
Neural Controlled Differential Equations for Irregular Time Series
มุมมอง 8K4 ปีที่แล้ว
Well-understood mathematics Neural Ordinary Differential Equations = State-of-the-art models for time series! Code: github.com/patrick-kidger/NeuralCDE Paper: arxiv.org/abs/2005.08926 This presentation prepared for the Machine Learning Summer School 2020 in Tübingen.

ความคิดเห็น

  • @personxy7443
    @personxy7443 3 หลายเดือนก่อน

    they can directly used in multi-agent control?

  • @gelwood_
    @gelwood_ ปีที่แล้ว

    Can Diffrax be used on a function that doesn't have a solution?

  • @gayatrigattani2615
    @gayatrigattani2615 2 ปีที่แล้ว

    Hii Patrick, Thank you so much for this video. It was damn helpful. Also can you help me with where can I study neural controlled differential equations thoroughly?

  • @farouk-hjabo
    @farouk-hjabo 3 ปีที่แล้ว

    Absolutely phenomenal! The work you are doing is amazing. Smooth presentations, readable understandable papers, and open source code. Thank you so much your work is closely related to my thesis. Keep up the good work, I wish you all the best.

  • @jaimecascantevega1123
    @jaimecascantevega1123 3 ปีที่แล้ว

    This is awesome! Could you please provide any insight on how to include estimates of time-varying parameters?

  • @yuhangguo2409
    @yuhangguo2409 4 ปีที่แล้ว

    Nice talk! BTW, where can I download the slides? Thx!

    • @patrickkidger1285
      @patrickkidger1285 4 ปีที่แล้ว

      A version of these slides are available here: kidger.site/links/CZszzuRmMmzo

  • @loo5405
    @loo5405 4 ปีที่แล้ว

    could you please release a model/.py for forecasting?

    • @patrickkidger1285
      @patrickkidger1285 4 ปีที่แล้ว

      We have a GitHub repo (and example) at github.com/patrick-kidger/NeuralCDE you can check out. We didn't specifically consider a forecasting problem in the paper but it should be straightforward to set up, using the classification example as a guide.

    • @loo5405
      @loo5405 4 ปีที่แล้ว

      @@patrickkidger1285 nice, ill give a try, im new to neural ode and im currently learning how to solve/train in julia, checking the benchmark it should be also faster than pytorch, so i wonder why cde and other models like latent ode aren't done in julia. sorry my english.

  • @cwkx
    @cwkx 4 ปีที่แล้ว

    Amazing work Patrick! I really liked the sketch-proof! I wonder if this insight could lead to further developments in architectural search, e.g. EfficientNets paper.

  • @mr.kyairey1679
    @mr.kyairey1679 4 ปีที่แล้ว

    Freedom of art of drawing that can be shared by many people to many places freely so obviously this is not writing: "In my opinion where I think the intention to stop the spreading of fake news is similarly the same like the intention to stop the use of sharp knives just because many tragedies have been obviously caused by sharp knives so is it yes or no to the spreading of fake news as there is no piracy if there are no buyers and there are no drug dealers if there are no buyers which means there is no problem at all with fake news?" - freedom of expression is also not about freedom to kill people as dead people had no freedom of expression so death penalty is murder crime including killing animal for food et cetera as there are still too many veganists promoting veganism on internet as punishment is torture so all people have rights to turn off the commenting on their post or on their posts as the international rules of Article 19 of Universal Declaration of Human Rights or also known as UDHR said freedom to hold opinion without being interfered so all people can or may impart any information without being disturbed or without being restricted or without being intervened et cetera; if it is not okay for people to test others for real through making a prank such like to rape someone until they say you got pranked and probably it is also not okay for god or gods to test others for real until there were many rape victims if there is god or gods so people should stop giving childbirth to stop all crimes immediately as murder crime equals death penalty and killing animal for food including childbirth the deadliest murder crime as there is no childdeath if there is no childbirth and of course words have never killed people except many people that have killed many people and many animals deliberately or undeliberately whether it was purposely committed consciously or unconsciously has no purpose at all as killing animal for food can be convicted as murder crime and death penalty can be convicted as revengeful murder crime and childbirth can be convicted as the deadliest murder crime as many people died from childbirth so childbirth is more dangerous than coronavirus.

  • @shaoaili4837
    @shaoaili4837 4 ปีที่แล้ว

    Deep Learning TH-camr Patrick :)

    • @patrickkidger1285
      @patrickkidger1285 4 ปีที่แล้ว

      Hey Aili! :D

    • @mr.kyairey1679
      @mr.kyairey1679 4 ปีที่แล้ว

      Freedom of art of drawing that can be shared by many people to many places freely so obviously this is not writing: "In my opinion where I think the intention to stop the spreading of fake news is similarly the same like the intention to stop the use of sharp knives just because many tragedies have been obviously caused by sharp knives so is it yes or no to the spreading of fake news as there is no piracy if there are no buyers and there are no drug dealers if there are no buyers which means there is no problem at all with fake news?" - freedom of expression is also not about freedom to kill people as dead people had no freedom of expression so death penalty is murder crime including killing animal for food et cetera as there are still too many veganists promoting veganism on internet as punishment is torture so all people have rights to turn off the commenting on their post or on their posts as the international rules of Article 19 of Universal Declaration of Human Rights or also known as UDHR said freedom to hold opinion without being interfered so all people can or may impart any information without being disturbed or without being restricted or without being intervened et cetera; if it is not okay for people to test others for real through making a prank such like to rape someone until they say you got pranked and probably it is also not okay for god or gods to test others for real until there were many rape victims if there is god or gods so people should stop giving childbirth to stop all crimes immediately as murder crime equals death penalty and killing animal for food including childbirth the deadliest murder crime as there is no childdeath if there is no childbirth and of course words have never killed people except many people that have killed many people and many animals deliberately or undeliberately whether it was purposely committed consciously or unconsciously has no purpose at all as killing animal for food can be convicted as murder crime and death penalty can be convicted as revengeful murder crime and childbirth can be convicted as the deadliest murder crime as many people died from childbirth so childbirth is more dangerous than coronavirus.

    • @mr.kyairey1679
      @mr.kyairey1679 4 ปีที่แล้ว

      @@patrickkidger1285 Freedom of art of drawing that can be shared by many people to many places freely so obviously this is not writing: "In my opinion where I think the intention to stop the spreading of fake news is similarly the same like the intention to stop the use of sharp knives just because many tragedies have been obviously caused by sharp knives so is it yes or no to the spreading of fake news as there is no piracy if there are no buyers and there are no drug dealers if there are no buyers which means there is no problem at all with fake news?" - freedom of expression is also not about freedom to kill people as dead people had no freedom of expression so death penalty is murder crime including killing animal for food et cetera as there are still too many veganists promoting veganism on internet as punishment is torture so all people have rights to turn off the commenting on their post or on their posts as the international rules of Article 19 of Universal Declaration of Human Rights or also known as UDHR said freedom to hold opinion without being interfered so all people can or may impart any information without being disturbed or without being restricted or without being intervened et cetera; if it is not okay for people to test others for real through making a prank such like to rape someone until they say you got pranked and probably it is also not okay for god or gods to test others for real until there were many rape victims if there is god or gods so people should stop giving childbirth to stop all crimes immediately as murder crime equals death penalty and killing animal for food including childbirth the deadliest murder crime as there is no childdeath if there is no childbirth and of course words have never killed people except many people that have killed many people and many animals deliberately or undeliberately whether it was purposely committed consciously or unconsciously has no purpose at all as killing animal for food can be convicted as murder crime and death penalty can be convicted as revengeful murder crime and childbirth can be convicted as the deadliest murder crime as many people died from childbirth so childbirth is more dangerous than coronavirus.

  • @Hannah-cb7wr
    @Hannah-cb7wr 4 ปีที่แล้ว

    Very interesting! And thanks for publishing the code, sadly not the standard for ML research.

    • @patrickkidger1285
      @patrickkidger1285 4 ปีที่แล้ว

      Thankyou! I think publishing code is becoming more frequent these days, thankfully.

  • @makesh8772
    @makesh8772 4 ปีที่แล้ว

    Excellent work and excellent presentation. Conveyed the key idea in just 13mins. A good tutorial to me on how to convey ideas very clearly. Unfortunate there is only one like so far :(

    • @patrickkidger1285
      @patrickkidger1285 4 ปีที่แล้ว

      Thankyou! Perhaps more likes will come in the future :)

    • @mr.kyairey1679
      @mr.kyairey1679 4 ปีที่แล้ว

      Freedom of art of drawing that can be shared by many people to many places freely so obviously this is not writing: "In my opinion where I think the intention to stop the spreading of fake news is similarly the same like the intention to stop the use of sharp knives just because many tragedies have been obviously caused by sharp knives so is it yes or no to the spreading of fake news as there is no piracy if there are no buyers and there are no drug dealers if there are no buyers which means there is no problem at all with fake news?" - freedom of expression is also not about freedom to kill people as dead people had no freedom of expression so death penalty is murder crime including killing animal for food et cetera as there are still too many veganists promoting veganism on internet as punishment is torture so all people have rights to turn off the commenting on their post or on their posts as the international rules of Article 19 of Universal Declaration of Human Rights or also known as UDHR said freedom to hold opinion without being interfered so all people can or may impart any information without being disturbed or without being restricted or without being intervened et cetera; if it is not okay for people to test others for real through making a prank such like to rape someone until they say you got pranked and probably it is also not okay for god or gods to test others for real until there were many rape victims if there is god or gods so people should stop giving childbirth to stop all crimes immediately as murder crime equals death penalty and killing animal for food including childbirth the deadliest murder crime as there is no childdeath if there is no childbirth and of course words have never killed people except many people that have killed many people and many animals deliberately or undeliberately whether it was purposely committed consciously or unconsciously has no purpose at all as killing animal for food can be convicted as murder crime and death penalty can be convicted as revengeful murder crime and childbirth can be convicted as the deadliest murder crime as many people died from childbirth so childbirth is more dangerous than coronavirus.

    • @mr.kyairey1679
      @mr.kyairey1679 4 ปีที่แล้ว

      @@patrickkidger1285 Freedom of art of drawing that can be shared by many people to many places freely so obviously this is not writing: "In my opinion where I think the intention to stop the spreading of fake news is similarly the same like the intention to stop the use of sharp knives just because many tragedies have been obviously caused by sharp knives so is it yes or no to the spreading of fake news as there is no piracy if there are no buyers and there are no drug dealers if there are no buyers which means there is no problem at all with fake news?" - freedom of expression is also not about freedom to kill people as dead people had no freedom of expression so death penalty is murder crime including killing animal for food et cetera as there are still too many veganists promoting veganism on internet as punishment is torture so all people have rights to turn off the commenting on their post or on their posts as the international rules of Article 19 of Universal Declaration of Human Rights or also known as UDHR said freedom to hold opinion without being interfered so all people can or may impart any information without being disturbed or without being restricted or without being intervened et cetera; if it is not okay for people to test others for real through making a prank such like to rape someone until they say you got pranked and probably it is also not okay for god or gods to test others for real until there were many rape victims if there is god or gods so people should stop giving childbirth to stop all crimes immediately as murder crime equals death penalty and killing animal for food including childbirth the deadliest murder crime as there is no childdeath if there is no childbirth and of course words have never killed people except many people that have killed many people and many animals deliberately or undeliberately whether it was purposely committed consciously or unconsciously has no purpose at all as killing animal for food can be convicted as murder crime and death penalty can be convicted as revengeful murder crime and childbirth can be convicted as the deadliest murder crime as many people died from childbirth so childbirth is more dangerous than coronavirus.