Valence Labs
Valence Labs
  • 295
  • 389 839
Neural Entropy | Akhil Premkumar
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg
We examine the connection between deep learning and information theory through the paradigm of diffusion models. Using well-established principles from non-equilibrium thermodynamics we can characterize the amount of information required to reverse a diffusive process. Neural networks store this information and operate in a manner reminiscent of Maxwell's demon during the generative stage. We illustrate this cycle using a novel diffusion scheme we call the entropy matching model, wherein the information conveyed to the network during training exactly corresponds to the entropy that must be negated during reversal. We demonstrate that this entropy can be used to analyze the encoding efficiency and storage capacity of the network. This conceptual picture blends elements of stochastic optimal control, thermodynamics, information theory, and optimal transport, and raises the prospect of applying diffusion models as a test bench to understand neural networks.
Paper: Neural Entropy arxiv.org/abs/2409.03817
มุมมอง: 992

วีดีโอ

scPRINT: pre-training on 50 million cells allows robust gene network predictions | Jérémie Kalfon
มุมมอง 47721 วันที่ผ่านมา
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg A cell is governed by the interaction of myriads of macromolecules. Such a network of interaction has remained an elusive milestone in cellular biology. Building on recent advances in large foundation models and their ability to learn withou...
Learning Equivariant Non-Local Electron Density Functionals | Nicholas Gao
มุมมอง 628หลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg The accuracy of density functional theory hinges on the approximation of non-local contributions to the exchange-correlation (XC) functional. To date, machine-learned and human-designed approximations suffer from insufficient accuracy, limit...
Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data
มุมมอง 732หลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg Ambient diffusion is a recently proposed framework for training diffusion models using corrupted data. Both Ambient Diffusion and alternative SURE-based approaches for learning diffusion models from corrupted data resort to approximations wh...
BindCraft: one-shot design of functional protein binders
มุมมอง 1.5Kหลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg Protein-protein interactions (PPIs) are at the core of all key biological processes. However, the complexity of the structural features that determine PPIs makes their design challenging. We present BindCraft, an open-source and automated pi...
Meta Flow Matching: Integrating Vector Fields on the Wasserstein Manifold | Lazar Atanackovic
มุมมอง 1.1Kหลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg Numerous biological and physical processes can be modeled as systems of interacting entities evolving continuously over time, e.g. the dynamics of communicating cells or physical particles. Learning the dynamics of such systems is essential ...
Derivative-Free Guidance in Continuous and Discrete Diffusion Models | Xiner Li and Masatoshi Uehara
มุมมอง 524หลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg Diffusion models excel at capturing the natural design spaces of images, molecules, DNA, RNA, and protein sequences. However, rather than merely generating designs that are natural, we often aim to optimize downstream reward functions while ...
A long-context RNA foundation model for predicting transcriptome architecture | Ali Saberi
มุมมอง 4572 หลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg Linking DNA sequence to genomic function remains one of the grand challenges in genetics and genomics. Here, we combine large-scale single-molecule transcriptome sequencing of diverse cancer cell lines with cutting-edge machine learning to b...
Fine-tuning Flow and Diffusion Generative Models | Carles Domingo-Enrich
มุมมอง 8752 หลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg Dynamical generative models that produce samples through an iterative process, such as Flow Matching and denoising diffusion models, have seen widespread use, but there has not been many theoretically-sound methods for improving these models...
Probabilistic Inference in Language Models via Twisted Sequential Monte | Rob Brekelmans
มุมมอง 9832 หลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg Numerous capability and safety techniques of Large Language Models (LLMs), including RLHF, automated red-teaming, prompt engineering, and infilling, can be cast as sampling from an unnormalized target distribution defined by a given reward o...
MolGPS - A Foundational GNN for Molecular Property Prediction
มุมมอง 4213 หลายเดือนก่อน
Scaling deep learning models has been at the heart of recent revolutions in language modelling and image generation. Practitioners have observed a strong relationship between model size, dataset size, and performance. However, structure-based architectures such as Graph Neural Networks (GNNs) are yet to show the benefits of scale mainly due to the lower efficiency of sparse operations, large da...
Geometric deep learning framework for de novo genome assembly | Lovro Vrček
มุมมอง 6383 หลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg The critical stage of every de novo genome assembler is identifying paths in assembly graphs that correspond to the reconstructed genomic sequences. The existing algorithmic methods struggle with this, primarily due to repetitive regions cau...
An Open MetaGenomic corpus for mixed-modality genomic language modeling | Andre Cornman
มุมมอง 4383 หลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg Biological language model performance depends heavily on pretraining data quality, diversity, and size. While metagenomic datasets feature enormous biological diversity, their utilization as pretraining data has been limited due to challenge...
Propensity Score Alignment of Unpaired Multimodal Data
มุมมอง 4603 หลายเดือนก่อน
Multimodal representation learning techniques typically rely on paired samples to learn common representations, but paired samples are challenging to collect in fields such as biology where measurement devices often destroy the samples. This paper presents an approach to address the challenge of aligning unpaired samples across disparate modalities in multimodal representation learning. We draw...
Tokenized and Continuous Embedding Compressions of Protein Sequence and Structure | Amy X. Lu
มุมมอง 7453 หลายเดือนก่อน
Portal is the home of the AI for drug discovery community. Join for more details on this talk and to connect with the speakers: portal.valencelabs.com/logg Existing protein machine learning representations typically model either the sequence or structure distribution, with the other modality implicit. The latent space of sequence-to-structure prediction models such as ESMFold represents the joi...
Discrete Flow Matching | Andrew Campbell
มุมมอง 1.4K3 หลายเดือนก่อน
Discrete Flow Matching | Andrew Campbell
Day 5 - Introducing Bioptic | Vlad Vinogradov
มุมมอง 2243 หลายเดือนก่อน
Day 5 - Introducing Bioptic | Vlad Vinogradov
Day 5 - Hackathon Introduction | Cas Wognum
มุมมอง 1413 หลายเดือนก่อน
Day 5 - Hackathon Introduction | Cas Wognum
Day 5 - Open-Source Initiatives & Benchmarking Efforts | Karmen Condic-Jurkic
มุมมอง 2253 หลายเดือนก่อน
Day 5 - Open-Source Initiatives & Benchmarking Efforts | Karmen Condic-Jurkic
Day 5 - Protein Folding & Design | Alex Tong
มุมมอง 7693 หลายเดือนก่อน
Day 5 - Protein Folding & Design | Alex Tong
Day 5 - LLMs in Drug Discovery | Andres M Bran
มุมมอง 6723 หลายเดือนก่อน
Day 5 - LLMs in Drug Discovery | Andres M Bran
Lab 4 - Target Deconvolution Explanation | Ali Denton & Kristina Ulicna
มุมมอง 2673 หลายเดือนก่อน
Lab 4 - Target Deconvolution Explanation | Ali Denton & Kristina Ulicna
Day 4 - Modeling Population Dynamics | Charlotte Bunne
มุมมอง 6273 หลายเดือนก่อน
Day 4 - Modeling Population Dynamics | Charlotte Bunne
Day 4 - Causal Discovery & Representation Learning | Jason Hartford
มุมมอง 3793 หลายเดือนก่อน
Day 4 - Causal Discovery & Representation Learning | Jason Hartford
Day 4 - Multimodal Omics & AI | Sébastien Lemieux
มุมมอง 3163 หลายเดือนก่อน
Day 4 - Multimodal Omics & AI | Sébastien Lemieux
Day 4 - Phenomics in Drug Discovery: Microscopy and Machine Learning | Anne Carpenter
มุมมอง 1.1K3 หลายเดือนก่อน
Day 4 - Phenomics in Drug Discovery: Microscopy and Machine Learning | Anne Carpenter
Lab 3 - De Novo Generation Explanation
มุมมอง 2293 หลายเดือนก่อน
Lab 3 - De Novo Generation Explanation
Day 3 - Harnessing Geometric ML for Molecular Design | Michael Bronstein
มุมมอง 3473 หลายเดือนก่อน
Day 3 - Harnessing Geometric ML for Molecular Design | Michael Bronstein
Day 3 - Synthesizability & Molecular Synthesis | Connor Coley
มุมมอง 6043 หลายเดือนก่อน
Day 3 - Synthesizability & Molecular Synthesis | Connor Coley
Day 3 - Exploring Molecular Space & Active Learning | Yoshua Bengio
มุมมอง 6043 หลายเดือนก่อน
Day 3 - Exploring Molecular Space & Active Learning | Yoshua Bengio

ความคิดเห็น

  • @pauljones9150
    @pauljones9150 10 วันที่ผ่านมา

    I don't know enough to understand this. Someone recommend me a textbook to read

    • @akhilpremk
      @akhilpremk 6 วันที่ผ่านมา

      Which part do you need help with? Knowing your background might help me recommend some material.

    • @pauljones9150
      @pauljones9150 6 วันที่ผ่านมา

      @akhilpremk I graduated with a computer science degree and I have an interest in AI, so I got most of the diffusion conversation, but got completely lost in the math/statistics

  • @wwkk4964
    @wwkk4964 12 วันที่ผ่านมา

    This was really eye opening! Though, we should be careful about whether our machine learning systems are picking up on algorithmic "noise" because those sources of noise have a signal and are abundant enough that the network can learn about it due to homogeneity of the noise generating source.

  • @wwkk4964
    @wwkk4964 13 วันที่ผ่านมา

    🎉 lovely talk!

    • @akhilpremk
      @akhilpremk 13 วันที่ผ่านมา

      Thank you!

    • @wwkk4964
      @wwkk4964 13 วันที่ผ่านมา

      @akhilpremk I am still absorbing and learning from your presentation, it is still open and am listening to many times. Thank you truly! 🙏

  • @surfaceoftheoesj
    @surfaceoftheoesj 15 วันที่ผ่านมา

    stochastic optimal control, thermodynamics, information theory, and optimal transport,

  • @GioiaTsunoda
    @GioiaTsunoda หลายเดือนก่อน

    Thanks for sharing such valuable information! A bit off-topic, but I wanted to ask: I have a SafePal wallet with USDT, and I have the seed phrase. (alarm fetch churn bridge exercise tape speak race clerk couch crater letter). How should I go about transferring them to Binance?

  • @achunaryan3418
    @achunaryan3418 หลายเดือนก่อน

    This channel is gold.

  • @rainemendez4729
    @rainemendez4729 หลายเดือนก่อน

    I hope this article is free access.

  • @achunaryan3418
    @achunaryan3418 หลายเดือนก่อน

    Why are you denying use of graph neural networks when you are using edge architectures in distributional regression embeddings? Are you restricting to mathematics than going ML? 25:34

  • @kaliyan1391
    @kaliyan1391 หลายเดือนก่อน

    Deepchem dont have any licensing model ? can I use for commercialization purposes ?

  • @JanKowalski-st6sj
    @JanKowalski-st6sj หลายเดือนก่อน

    really cool shirt!

  • @eduardocesargarridomerchan5326
    @eduardocesargarridomerchan5326 2 หลายเดือนก่อน

    Tutorial de Kolmogorov Arnold networks en castellano: th-cam.com/video/Jb9wMCPUlnc/w-d-xo.html

  • @oswack
    @oswack 2 หลายเดือนก่อน

    So based on my understanding, this model's usability is hitched on the assumption that one has a "perfect" mapping function whereby no information is lost when applying Kolmogorov's theory to return the 1D edges? Because that in itself can be extremely difficult, even a near approximation.

  • @juniornicolodi
    @juniornicolodi 2 หลายเดือนก่อน

    Thank you a lot for the content! Great video!

  • @CapucineLamoubiv
    @CapucineLamoubiv 2 หลายเดือนก่อน

    Robinson Christopher Anderson Paul White Matthew

  • @godzikc
    @godzikc 2 หลายเดือนก่อน

    Great job @AnneCarpenter

  • @t0s0jain
    @t0s0jain 2 หลายเดือนก่อน

    Can you explain why you used a Ca-Ca prior of 3.8 A for this model where you are denoising Cb coordinates?

  • @Gaib_al_lisan
    @Gaib_al_lisan 2 หลายเดือนก่อน

    What I am not able to fully explain/justify is the accuracy schedule. I understand the existence on an accuracy altogether, since we chose to work with distributions across the entire workflow. I speculate that increasing the accuracy has to do with the "smoothness"/stability of training; if we were to start with a very informative message that the sender transmits to the receiver, then the update step would he very large, propelling the theta parameters to high values. This is just my intuition. It's not explained in the paper

  • @robertobruzzese8830
    @robertobruzzese8830 3 หลายเดือนก่อน

    i am interested in this subject of drug discovery! fine

  • @aminamoudjar4561
    @aminamoudjar4561 3 หลายเดือนก่อน

    I’de like to join this lab as phD student

  • @ShiqianTan
    @ShiqianTan 3 หลายเดือนก่อน

    Very touch and informative

  • @agranero6
    @agranero6 3 หลายเดือนก่อน

    I didn't expect to se a mention to Jone polynomials...last time I talked about that was...well in the 80s.

  • @jaimehibbard2114
    @jaimehibbard2114 3 หลายเดือนก่อน

    This is super helpful! I have a molecular biology background, and this talk explained ML for molecular representation and scoring in a very accessible way.

  • @shoubhikdasguptadg9911
    @shoubhikdasguptadg9911 3 หลายเดือนก่อน

    I have been following Bharath for some time on his Discord channel. This talk makes me want to fly across continents to meet him in person and thank him for sharing his knowledge with the community.

  • @benzoutandy8435
    @benzoutandy8435 3 หลายเดือนก่อน

    cool

  • @zapy422
    @zapy422 4 หลายเดือนก่อน

    thank you

  • @davidlearnforus
    @davidlearnforus 4 หลายเดือนก่อน

    Thanks a lot! Based on my current understanding I would say quickest starting mental image for me to understand BFN's would be telling me that it is a standard NN with Bayesian teacher forcing.

  • @julienblanchon6082
    @julienblanchon6082 4 หลายเดือนก่อน

    This Hannes Stärk guys is a legend for organizing all this

  • @Ammz1010
    @Ammz1010 4 หลายเดือนก่อน

    How can i join

  • @Harirtaylorversion
    @Harirtaylorversion 4 หลายเดือนก่อน

    How can we join research position in the lab?

  • @mpetersen4823
    @mpetersen4823 4 หลายเดือนก่อน

    Thanks, Boyuan! Really interesting work!

  • @yuguo4786
    @yuguo4786 4 หลายเดือนก่อน

    interesting topic!

  • @alexiscao8749
    @alexiscao8749 4 หลายเดือนก่อน

    Mathew sounds almost identical to the biologist Mike Levin from Tufts

  • @vipulverma3640
    @vipulverma3640 4 หลายเดือนก่อน

    Can we use this for time series to forecast the future value ?

  • @wangchentong5623
    @wangchentong5623 4 หลายเดือนก่อน

    Thank you for making this, it will be better if there’s less ad. There’s at least 4-5 through this video

  • @rylieweaver1516
    @rylieweaver1516 4 หลายเดือนก่อน

    Great conversation, thank you!

  • @rylieweaver1516
    @rylieweaver1516 4 หลายเดือนก่อน

    Could you elaborate more on how the allegro model allows for more parallelization? You say that there is no message-passing, but then indicate that there are layers for tensor products. I’m curious what the layers for tensor products are for if there isn’t message-passing?

  • @rylieweaver1516
    @rylieweaver1516 4 หลายเดือนก่อน

    Great research, thank you for the presentation!

  • @marcelhedman7391
    @marcelhedman7391 4 หลายเดือนก่อน

    Great video

  • @marcelhedman7391
    @marcelhedman7391 4 หลายเดือนก่อน

    Great video

  • @jihochoi_cs
    @jihochoi_cs 4 หลายเดือนก่อน

    Thank you!

  • @djalu-wahyu
    @djalu-wahyu 4 หลายเดือนก่อน

    the formula at screen 's right corner , can't read

  • @benzoutandy8435
    @benzoutandy8435 5 หลายเดือนก่อน

    cool

  • @ntej7927
    @ntej7927 5 หลายเดือนก่อน

    Interesting.......Thanks.

  • @johndewey7243
    @johndewey7243 5 หลายเดือนก่อน

    All of these posts are solid platinum. Keep it all coming!

  • @drvanon
    @drvanon 5 หลายเดือนก่อน

    Martin is having a lot of fun in the background 😂

  • @luke.perkin.inventor
    @luke.perkin.inventor 5 หลายเดือนก่อน

    I really enjoyed this, strong generalisation! The previous papers on bias free CNNs are also really good, there's a few good talks given by Eero Simoncelli on youtube. Do you think you could do a talk on some follow up papers? It feels like something in this space might be a "Deep Image Prior" moment for 2024! That's cited 3000+ times now.

  • @yes4653
    @yes4653 5 หลายเดือนก่อน

    Very useful ❤ and nice explanation @Nicholas Gao

  • @LaiannePlácidoLima
    @LaiannePlácidoLima 5 หลายเดือนก่อน

    vícios de linguagem e um raciocínio descontínuo.

  • @jakubpodsiadlikowski1897
    @jakubpodsiadlikowski1897 5 หลายเดือนก่อน

    Why am i seeing this?

  • @Pingu_astrocat21
    @Pingu_astrocat21 5 หลายเดือนก่อน

    Was waiting for this thankss a lot!!