Panel Discussion: Open Problems in the Theory of Deep Learning

แชร์
ฝัง
  • เผยแพร่เมื่อ 14 ธ.ค. 2024

ความคิดเห็น • 12

  • @SalehElm
    @SalehElm 16 ชั่วโมงที่ผ่านมา

    Interesting that Tommy's statement about AlphaFold
    deserving of Nobel Prize finally came true :)

  • @haniel_ulises
    @haniel_ulises 3 หลายเดือนก่อน +5

    We need better theoretical frameworks for deep learning, in order to keep deep learning away from pseudoscience

  • @AlgoNudger
    @AlgoNudger 3 หลายเดือนก่อน +2

    This field needs stronger interdisciplinary collaboration (neuroscientist, cognitive scientist, physicist, mathematician, statistician, linguist, theoretical computer scientist, etc.) to avoid marketing buzzwords (hype).

  • @neuralbrew2976
    @neuralbrew2976 3 หลายเดือนก่อน

    Backpropagation/gradient descent is not the end all for deep learning algorithms. It is just one tool in the tool box. The brain uses correlation, logarithmic and non-linear signal processing

  • @willd1mindmind639
    @willd1mindmind639 3 หลายเดือนก่อน

    The problem is not deep learning in general as opposed to who is defining what the purpose and goals of these specific models are. For example, what is the purpose of a language model other than generating grammatically and syntactically correct language? That is not necessarily the same thing as generating factually correct information. Children learning subject, verb and noun relationships are using sentences to practice this all the time which are not factual statements but represent the fundamental rules of language. So from that perspective an LLM generating an answer that isn't factual but linguistically correct is actually working as designed. The reason humans are able to distinguish real versus fictional or theoretical is because we have experience outside of simply reading text or understanding language that an LLM does not. So what is missing from a language model training paradigm is the ability to treat certain pieces of text as "ground truth" and embed it as such within the distribution. And along with that, be able to set a context within a prompt that only wants answers based on "ground truth" or verified facts. And no, this is not about technological dictatorships, but basic things like using a dictionary and other sources of information as agents of authority for certain "knowledge" or collections of tokens in the model.

  • @deeplearningpartnership
    @deeplearningpartnership 3 หลายเดือนก่อน +1

    Be good if Tommy paid attention to what the speakers were saying rather than looking at his phone all the time.

  • @Sirmrmeowmeow
    @Sirmrmeowmeow 3 หลายเดือนก่อน

    'the brain has not changed significantly over the past 100,000 years' this statement may not be true. The size hasn't changed much. but there is indeed evidence that the brain has been undergoing changes to some extent due to various pressures. :x

  • @richardnunziata3221
    @richardnunziata3221 3 หลายเดือนก่อน

    It seems to me that the difference between the natural sciences and a brain while they are both physical at the base implementations the ontologies for a mind are incongruent quantitative. A new science of network ontologies is needed as well as development :networks embryogenesis.

  • @abdulshabazz8597
    @abdulshabazz8597 3 หลายเดือนก่อน

    I wonder if Dr. Andreas matriculated from a Mathematics background.
    @00:21:15 Notice how Jacob Andreas is quick to poke holes into, and to point out logical errors in, Santosh Vempala's suggestion of dispensing a Confidence Interval along with each completion generated by a generative transformer model. It's a very subtle way to intimate the funding of his research is more significant than the Dr.'s. Keep in mind both parties have millions in public and private funding at stake.
    This was instructed to be a brainstorming session!!
    Such behavior is rife in Mathematics and Physics! and it discourages public collaboration.
    This in my opinion is the reason engineering flourishes with newly published papers daily, and mathematics has few breakthroughs from academia -- and only from culturally approved and politically connected individuals, when compared to the private (business) sector.

  • @tankieslayer6927
    @tankieslayer6927 3 หลายเดือนก่อน +1

    MIT's CSAIL did try the ARC challenge with their Dreamcoder approach. From what I understand, it did not work. Now they pretend they never heard of the challenge.

  • @TheJimbosan
    @TheJimbosan 3 หลายเดือนก่อน +2

    How rude!!! Tomaso is ON HIS PHONE during the panel discussion?!! What the hell is wrong with you???!!!