Great video, but one small mistake at around 19:55 : You do not have two torsion angles because you are in 3D. The thing is, that atoms can rotate around a single covalent bond. The amino acid backbones, however, are connected in such a way, that a double bond is created (to be true, it is just a partial double bond, but this it not further important to get the point). Double bonds create an atom plane, in which the atoms are fixed and can no longer rotate individually. This is the case for connected amino acids. In detail, the center carbon atom, called C_alpha, is connected to two planes, one is the connection to the prior amino acid and the second the connection to the next one. The rotation angles of these plane in relation to the C_alpha are then the torsion angles. These are a direct result from the underlying chemistry and are used to describe structures since the birth of structural biology. If there wasn't a partial double bond, we would have a huge problem, since each protein would have even more angles we would have to consider (the side-chain angles are freely rotatable in most amino acids and lead to many many more possible combinations, but the backbone is more important and thankfully there are just two angles).
I've done my bachelor thesis in structural proteomics and your introduction was very very good! Looking forward to the explanation on the paper on AlphaFold2!
Note that at ~10:00 we get the impression that 'shape is all you need', but while some alternative AAs that replace common ones in given positions of a particular protein can retain the energetically favored structure, the functionality might be altered drastically: shape != functionality - you can have two different proteins with the same shape but only one is functional in the examined spatiotemporal sphere, or functions as expected. Further, the AA composition of the primary chain and its multitude of intrinsic properties are not the sole determinants of the final 3d structure as this depends largely on the environment (acidity, temp. etc.). Finally - different shapes (different proteins) can have similar functions. Overall top effort and overview of AlphaFold, thanks Yannic!
And there are quantum mechanics effects. Even a slight change in the atomic structure - like deuterium instead of hydrogen - can alter the total energy of an electron that tries to tunnel through the molecule. And the whole protein "machine" falls apart. But I guess shape is still VERY important. So good job Deepmind, CASP and of course Yannic! :)
Great job explaining everything. I'm a ~~protein engineering PhD student and all of the other videos I've watch have played into the hype and not explained anything well. Based off their CASP results they haven't solved anything yet but if they keep up this rate of innovation, they will in the next 2-4 years. They are absolutely killing the other big player in the field though (Rosetta), it is truly amazing what they have been able to accomplish.
Could you explain exactly why it can't be considered solved? Based on their blog they basically say their predictive error is close to what you would observe if you tried to determine the structure experimentally, which seems to be pretty close to being solved. I'm a ml student with limited knowledge of proteins, although I took a bioinformatics course and pretty much just coasted lol
@@firecatflameking To be considered 'solved' in my mind the model should be able to predict structures with ~90-95% percent crystal structure resolution or roughly cryo-EM resolutions, in >80% of cases. This would give me enough confidence in my structures to begin engineering proteins using this software and then only expressing the protein to validate changes periodically throughout the design process.
@@firecatflameking if they keep up at the same rate that they are we should be nearly there next year! But that's yet to be seen. I'm excited to see what they do
Thank you, finally, someone providing a longer more detailed presentation about AlphaFold. Just found your channel and will definitely check out more content. Great job!
There are a few amino acids with rings (they are called aromatic): Phenylalanine, Tryptophan, Tyrosine, (and histadine). Proline also connected strucure :) I love these videos man!
My comments as a computational biophysicist student: 1. What about protein which need extra proteins which need helping such as chaperone? Anfinsen's dogma (i.e. AA sequence encodes 3D) does not apply quite well here. 2. Nature paper on this AlphaFold mentioned that complex structure (probably meaning such as homo/hetero-n-meric proteins) is yet to be predicted with high accuracy due to intermolecular interactions distorting the structures. 3. Most importantly at least to me...what about correct folding PATHWAY? Deeplearning, MC-based, homology modeling whatsoever is all about the END structure. Molecular dynamics can perhaps (depending on force fields) predict folding pathways (called reaction coordinate or collective variable).
You say that the 64x64 conv can only see 64 amino acids at a time, but that's not true. While it is the case for a single layer conv net, when you stack convolution layers the effective receptive field grows with each successive layer. Their model with "220 residual convolution blocks" is deep enough for a receptive field of at least thousands of amino acids.
They have deep convolutional model that have 220 convolutional layers and takes 64x64 input size. But the whole thing has size LxL, where L > 64, so they must run their network several times on separated parts of the input and aggregate predictions. So it can really see only 64 aminos at time.
The 2 step process reminds me of the symbolic regression on physical systems paper. Use deep learning to generate some intermediate representation, and then use that representation as the model to approximate for a different algorithm that has nicer properties.
DNA is your compressed source code. RNA is your decompressed source code. Proteins are your binaries. Each 3 digits of base-4 dna or rna code represents one amino acid, with some seeming redundancy. We are slowly learning the exact compiler code. It was not that long ago when we found the code for "start" and "end". This folding puzzle is one of the last big steps before we can program life like we program computers.
A big issue with protein folding is that there are structures at many different scales. Small recursions wrap into large complexes, which fold into large knots. A CNN would in itself struggle to make those long chain assocations, alpha 2 has to be a transformer with attention in order to draw a relationship between the protein segment at position #2,736,203,023 and its nearby neighbor in 3d space, protein segment #72,720,022,853 millions of aminos down the line
John Jumper mentioned at CASP14 on Tuesday that their structure prediction system uses "equivariant" transformers and, most importantly, is end-to-end, meaning they can backpropagate errors through the entire prediction system. Just FYI.
So the end after the references is basically the supplimentals. I'm not sure if that is common in CS papers but it just goes into detail on exactly how everything was done.
Nicely explained (even biochemistry)!! Loved it when you compared DNA to source code,protein as binary and the whole process to compilation. I am a biologist interested in machine learning and AI and it is great to see this explanation. BTW, the amino acids with the rings exist . But especially proline's ring with amine group inside the ring makes protein folding even more complicated.
I think the problem with this approach is defining the “shape you want” in the first place. It’s difficult for scientists to accurately estimate the shapes of proteins based on their function. I mean, to an extent. Like if your protein serves the function x, then it might contain structural components a, b and c. But estimating even crudely accurate xyz coordinates for all the residues would be an exercise in futility. Also, the nature of the protein folding problem is that we know the sequence but we don’t know the 3D structure. Since sequence is the defining factor in determine tertiary structure, we should start with the sequence and go from there.
@@EMSV66 is this because their structure is very well known/conserved. Perhaps you shed light on this “going backwards method” more than I can, it’s not my field but I’m v interested!
@@npm1811 Yes, that is part of the reason. You only need to model the variable region. With antibodies you are trying to create a complementary surface to the antigen's so you can start from the structure of the antigen. Look up the work of Costas Maranas at PSU.
I was pre-med, got a bachelor's in Biology and am in an MS for ML. I cried reading this news yesterday. I don't want to rain on the skeptics parade as I sympathize with that view a bunch (since I'm usually "that guy"). Assuming they didn't train on the test samples inferred during the competition, this is big progress and a big deal. Is it worth pointing out that this is the same group a couple years ago that proved they can beat everyone on the planet at Go, Chess, etc? Pharmaceuticals is the obvious beneficiary of this tech, but we could imagine some next level stuff too. Like a CRISPR that we didn't just steal from bacteria and reverse engineer. We could, in the future, create those cellular machinery on our own terms. People all know DNA, but it is just blueprints for the proteins, the proteins are the real important stuff which do the enzymatic work.
@@rbain16 True, but not all biotech applications / workflows need to know the structure of a protein. It sure will help on many things, many of which we will probably not see coming; but I'd say that for now it'll mainly impact those applications where the spatial configuration of the protein can be exploited.
A structural biologist here. Decent explanation of protein folding. What is the best way to jump into neural networks for a newbie. Also, a comment on Nature papers. They have a print version that is shorter and an extended online version that contains the Methods section. So the short print version contains all the data with little explanation of how it was obtained. A deeper explanation of the methods used can be found in the extended online version. I hope this helps.
20:12 Till this moment. The Idea is that, train a network which reads the sequence of amino acids and predicts their distance. Stage 2 do the gradient descending in order to both generate something real spacial data in the form of vector3d and check if the prediction from stage 1 is possible. If it's not possible, let's say, distance(point 1,point 2) == 1, d(2,3)== 1,but d(1,3)== 3, the stage 2 has to deal with this, and gives out a result which fulfill all the distance prediction as possible.
Beau-ti-ful explanation. One of the best explanations of any subject I've ever seen. Brilliant turn from a complex matter to a understandable subject. Ge-ni-us. Congrats and, above all, Thank You Very, Very Much. Beau-ti-ful.
The paper states that they used dilated convolutions, this made it possible to also model long term interactions. It is crucial, since protein folding is going to be highly dependent on those longterm interactions that determine the 3D structure of a protein.
I love the on the fly real human to human explanation and the fun that ensues. @2033s "My drawing skills are to be criticized in another video " 🙂 . Technically accurate and compact and relevant. Enjoyed it immensely in many ways. Thanks!
The iterative process is probably the folding step. Typically, when using one of the algorithms you will fold the protein thousands of times and build the final structure from a wieghed average of all the folded structures.
I wonder if the results would be improved further if they used a tetryonic / quantom field base model. Similar to prior alpha projects, human assumptions and training on some human data actuall was holding the project back for the final stretches of improvement.
OK, the algorithm of alphafold2 is different to alphafold version 1.0. they replaced EVERY part of alphafold version 1 routine. there is no point in discussing alphafold v. 1.0 at all. apples vs. oranges. different neural network, different physical constraints, different everything. the co-evolution matrices and the final 3d minimization function were separate routines, but they are part of a single neural network now. the network adjusts and rebuilds itself (its type changes) depending on the sequence. Etc, etc, etc. Once again. There is no point in reading that nature paper any more, the new algorithm is really different.
25:23 It says "gradient descent on _protein_ _specific_ _potential_ ". I believe at that stage initial predicions are not used anymore (only as initial state).
Really there are only 20 amino acids found in proteins encoded by genetic code, Selenocysteine which is found in about 50 eukaryotic proteins replaces cysteine by incorporating Se in place of S, not a huge structural change. There can also be post translational modifications. I bet that some more highly ordered proteins, such as those with a large number of alpha helices, and/or beta sheets connected by more disordered strands are easier to predict as well as a larger data set from x-ray crystallography where protein homology aids the prediction. X-ray crystallography will still be important when looking at how other small molecules bind to proteins or how protein to protein interactions are observed.
Does anyone know, if there is a good source for explanations about convolutional neural networks? I am a biochemistry bachelor, so I've basically no experience with computer science at all.... But it would be super helpful :) And of course: Thanks for the great explanation, it was really entertaining to watch that explanation of the paper and it did really help to understand the concepts of the underlying math and computer sciences aspects.
Update: This video is great. I watched up until 3:30 when Yannic says this video is for a "Machine learning person" rather than a protein engineering type person. I know quite a bit about protein folding and protein engineering and fairly little about the technical side of machine learning. Is watching this hour long video going to be a waste of time for me?
Also, how does this account for how the environment affects how a protein folds? For example, don't some proteins misfold in the presence of other misfolded proteins of the same type?
As I remember it was a paper by the Salesforce team about 6 months ago on using BERT to predict binding points on the protein chains. Do you think that Google folk had sort of the same idea?
@@quebono100 Another point I got from talking with an expert, is that with tech like GPT-3, you still need a way to properly mitigate and handle the bias it can produce, otherwise imagine all the incidents that will arise from "surprising insensitivity" in a gpt-3 application. :/
@@andrewwilliam2209 Im not an expert. But I think you can do it like they discussed, you can be a prompt engineer :) and simply discover promts and pin it with a seed. (they discussed just the promt engineer part, I expiremented with gpt2 and theire you can have a random seed)
@@mitchellty Yeah you right. It doesnt mean, that they didnt build something amazing. But nower days such companies try to fake it until they make it. Like UNU AI predicted the kentuky derby 3 correct winners in a row. I thought wow thats amazing how did they do such a difficult task. They truly predicted the kentuky derbi, but not only with theire software but also with expert who cherry picked the most likely winners. Its all marketing, to push the product.
As for AlphaFold2: I guess the pairwise distances are just for training part of the model. It may not be used in predicting the structure. Directly predicting the structure may be better than using the pairwise dist as the middleman.
thank you for explaining the research papers the way do . I find very hard to understand by reading them; I am 3rd year Bsc student for Computer Science. I love the fields and the papers you talk about so it definitely feels great understanding a bit more about the papers you explain!
wow, that was deja vu. I worked on a project that had the same basic approach as alphafold 1 (distance matrix+neural network+gradient descent) in the 90's. Of course the neural net was a simple backprop net and the sample set was much tinier, but yeah, basic same approach. I've got to think that the deepmind group is infuriating a lot of pure science teams out there - they descend into their territory, clean their clocks on specific problems and then go 'hey - look how great we are'. Its GOT to drive the original scientists there absolutely nuts. But then again, I don't really mind as long as I get flying cars out of it. :)
@@dionbridger5944 well, yes that is what I was getting at with my "not really minding as long as I get flying cars out if it" comment. Yet still there is something about the tone of the deepmind folks that reminds me of the masters' attitude from monty python's 'meaning of life' skit about rugby at boarding school: th-cam.com/video/HKv6o7YqHnE/w-d-xo.html
Hey Yannic, great work on you videos, I really appreciate it. Could you cover the topic of lavasz loss and the respective paper "The Lovasz-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks" in future videos? I read that this specific jaccard approximation is used as loss function in many image segmentation tasks, even in U-net it has been observed that it gives better results. However it is not straightforward and I still don't fully get it. It would be awesome if you could create a video for it, breaking down the concept as you have done amazingly so many times. thank you :)
Today I was informed that AlphaFold 2 presentation for CASP14 is online, and it contains more information than DeepMind's blogpost (a bit more info about the model on slide #10): predictioncenter.org/casp14/doc/presentations/2020_12_01_TS_predictor_AlphaFold2.pdf Presentations of all participants: predictioncenter.org/casp14/doc/presentations/ For AlphaFold 1 and all other participants (CASP13 - 2018) predictioncenter.org/casp13/doc/presentations/
@@herp_derpingson there is. www.nature.com/articles/s41586-019-1923-7.epdf?author_access_token=Z_KaZKDqtKzbE7Wd5HtwI9RgN0jAjWel9jnR3ZoTv0MCcgAwHMgRx9mvLjNQdB2TlQQaa7l420UCtGo8vYQ39gg8lFWR9mAZtvsN_1PrccXfIbc6e-tGSgazNL_XdtQzn1PHfy21qdcxV7Pw-k3htw%3D%3D
I feel like the predicted torsion are more important than you say. For me the (torsion+distances) will be somewhat inconsistent, and the gradient descent enforce consistency. But given the space of search I'd bet you need the initial guess to be good to not get stuck in a local minima. Just my intuition though.
SUperb video. What application do you use to create these videos? Id like to learn to draw and scroll through images while recording my voice over just like your format.
I'm going to be sceptical because this type of problem seems incredibly similar to solving conway's game of life, but way more complicated. We haven't even figured out the super-simple version yet.
Great video like always. So it seems like DeepMind went directly from predicting something similar to adjacency matrix to transformers. I was wondering if they ever implemented spectral graph analysis here.
'Pyrene ring' isn't a thing. It's called an aromatic ring. Beta strands are joined by hydrogen bonds, to form a beta-pleated sheet. The beta sheet is the reason why the protein-folding took so long to solve. Hydrogen bonds of alpha helices are local, whereas beta sheets have global bonds.
Great vid, highly informative, very interesting and you are unquestionably the 'Richard Burton, Morgan Freeman and Jeremy Irons' hybrid voice of science vids! 😃
Loss function is like L²u for a Lagrangian progress error counting, but these Austrian mathematics school is really neat for angle repair in the aminase codification.
Thanks for the video. I studied biology and love tech. The ribosomes produce the amino acid chains which then go to the Endoplasmic Reticulum (ER) and then for further modification in the Golgi Body. Metals such as iron or magnesium along with atoms such as N are added. The question is what is conducting this process? It is more than residue attraction/repulsion and torsion angles. Possibly there are other proteins guiding the folding. What about 2 identical amino acid chains producing different multiple outcomes?
The explanation torsion angles is false. What is described and drawn at minute 20:49 are angles (3-body terms), torsion angles are dihedral angels between the two planes spanned by 4 neighbouring atoms (or any type of points).
Im not a machine learning person, i barely even know what a protein is, how to flip it, twist it, fold it to a paper plane or similar is far beyond my knowledge.. i am however a HTML programmer, made everything from divs to s, to websites build completely out of tables or even better canvas and SVG.. im currently trying to make an AI with HTML, marquee is the core
thanks for the great video! :) and... since I am coming from the field of bioinformatics, I really enjoyed your confusion about the format of Nature papers - I remember I had the same strange feelings about it when I started to write papers for similar journals.
Honestly I am so fascinated by the papers but as someone who is new to the domain...it feels overwhelming...for eg..in this video the transformers were like a prerequisite and I feel there might be some prerequisite for every video maybe... So is there like a playlist or some flow that someone could suggest. Thanks
![[LIVE] : ONE ลุมพินี 80 | คู่เอก "รักษ์ vs ยอดนำชัย"](http://i.ytimg.com/vi/ckLMRorsMeY/mqdefault.jpg)
Great video, but one small mistake at around 19:55 : You do not have two torsion angles because you are in 3D. The thing is, that atoms can rotate around a single covalent bond. The amino acid backbones, however, are connected in such a way, that a double bond is created (to be true, it is just a partial double bond, but this it not further important to get the point). Double bonds create an atom plane, in which the atoms are fixed and can no longer rotate individually. This is the case for connected amino acids. In detail, the center carbon atom, called C_alpha, is connected to two planes, one is the connection to the prior amino acid and the second the connection to the next one. The rotation angles of these plane in relation to the C_alpha are then the torsion angles. These are a direct result from the underlying chemistry and are used to describe structures since the birth of structural biology. If there wasn't a partial double bond, we would have a huge problem, since each protein would have even more angles we would have to consider (the side-chain angles are freely rotatable in most amino acids and lead to many many more possible combinations, but the backbone is more important and thankfully there are just two angles).
Yannic Kilcher is all you need
lol
The unreasonable efficiency of Yannic Kilcher
Learning to summarize from Yannic Kilcher
Self-training with Noisy Yannic Kilcher
Machine Learning Research Paper Summarization Models are Yannic Kilchers!
I've done my bachelor thesis in structural proteomics and your introduction was very very good!
Looking forward to the explanation on the paper on AlphaFold2!
Yannic:"If youre watching this youre a machine learning person,and dont know about proteins"
Me:"Actually...........quite the opposite"
Same 😂😂
Friendship ended with CNN
Now, Transformer is my best friend
Is it because of the butt-wiping feature?
Yeah, well transformers are not the universal answer, CNNs won't go anywhere soon.
Hello comrade
good reference
My friendship has ended with every media outlet like FOX and CNN, they all give us the illusion of a competition.
Note that at ~10:00 we get the impression that 'shape is all you need', but while some alternative AAs that replace common ones in given positions of a particular protein can retain the energetically favored structure, the functionality might be altered drastically: shape != functionality - you can have two different proteins with the same shape but only one is functional in the examined spatiotemporal sphere, or functions as expected.
Further, the AA composition of the primary chain and its multitude of intrinsic properties are not the sole determinants of the final 3d structure as this depends largely on the environment (acidity, temp. etc.). Finally - different shapes (different proteins) can have similar functions.
Overall top effort and overview of AlphaFold, thanks Yannic!
And there are quantum mechanics effects. Even a slight change in the atomic structure - like deuterium instead of hydrogen - can alter the total energy of an electron that tries to tunnel through the molecule. And the whole protein "machine" falls apart.
But I guess shape is still VERY important. So good job Deepmind, CASP and of course Yannic! :)
Great job explaining everything. I'm a ~~protein engineering PhD student and all of the other videos I've watch have played into the hype and not explained anything well.
Based off their CASP results they haven't solved anything yet but if they keep up this rate of innovation, they will in the next 2-4 years. They are absolutely killing the other big player in the field though (Rosetta), it is truly amazing what they have been able to accomplish.
Could you explain exactly why it can't be considered solved? Based on their blog they basically say their predictive error is close to what you would observe if you tried to determine the structure experimentally, which seems to be pretty close to being solved. I'm a ml student with limited knowledge of proteins, although I took a bioinformatics course and pretty much just coasted lol
Would love to know why you don't consider it solved aswell!
@@firecatflameking To be considered 'solved' in my mind the model should be able to predict structures with ~90-95% percent crystal structure resolution or roughly cryo-EM resolutions, in >80% of cases. This would give me enough confidence in my structures to begin engineering proteins using this software and then only expressing the protein to validate changes periodically throughout the design process.
@@scatteredvideos1 Makes sense! I'm guessing we're gonna get there within a few years
@@firecatflameking if they keep up at the same rate that they are we should be nearly there next year! But that's yet to be seen. I'm excited to see what they do
Thank you, finally, someone providing a longer more detailed presentation about AlphaFold.
Just found your channel and will definitely check out more content. Great job!
Lightspeed Kilcher strikes again. He's faster than Usain Bolt. ✨
first 🤣
@17:20 the second line has "NERDS" in it.
That's it. That's my contribution to this discussion.
There are a few amino acids with rings (they are called aromatic): Phenylalanine, Tryptophan, Tyrosine, (and histadine). Proline also connected strucure :) I love these videos man!
My comments as a computational biophysicist student:
1. What about protein which need extra proteins which need helping such as chaperone? Anfinsen's dogma (i.e. AA sequence encodes 3D) does not apply quite well here.
2. Nature paper on this AlphaFold mentioned that complex structure (probably meaning such as homo/hetero-n-meric proteins) is yet to be predicted with high accuracy due to intermolecular interactions distorting the structures.
3. Most importantly at least to me...what about correct folding PATHWAY? Deeplearning, MC-based, homology modeling whatsoever is all about the END structure. Molecular dynamics can perhaps (depending on force fields) predict folding pathways (called reaction coordinate or collective variable).
I study biochemistry and the explanation about folding proteins is actually really good!
You can see Yannic's brain breaking in realtime, not able to cope without there being *something* to be grumpy about.
You can be grumpy about who controls it.
Plot twist: the intern at CASP wrote buggy code for the score computation
if "DeepMind" score = 2 * highest score
Turns out you're true
@@saanvisharma2081 Wait, what do you mean by "you're true"?
Nah fam
@@BR-fu9px that would be a second-order plot twist
You say that the 64x64 conv can only see 64 amino acids at a time, but that's not true. While it is the case for a single layer conv net, when you stack convolution layers the effective receptive field grows with each successive layer.
Their model with "220 residual convolution blocks" is deep enough for a receptive field of at least thousands of amino acids.
They have deep convolutional model that have 220 convolutional layers and takes 64x64 input size. But the whole thing has size LxL, where L > 64, so they must run their network several times on separated parts of the input and aggregate predictions. So it can really see only 64 aminos at time.
The 2 step process reminds me of the symbolic regression on physical systems paper. Use deep learning to generate some intermediate representation, and then use that representation as the model to approximate for a different algorithm that has nicer properties.
DNA is your compressed source code.
RNA is your decompressed source code.
Proteins are your binaries.
Each 3 digits of base-4 dna or rna code represents one amino acid, with some seeming redundancy.
We are slowly learning the exact compiler code. It was not that long ago when we found the code for "start" and "end".
This folding puzzle is one of the last big steps before we can program life like we program computers.
Wow Yannic you have such amazing teaching skills.
A big issue with protein folding is that there are structures at many different scales. Small recursions wrap into large complexes, which fold into large knots. A CNN would in itself struggle to make those long chain assocations, alpha 2 has to be a transformer with attention in order to draw a relationship between the protein segment at position #2,736,203,023 and its nearby neighbor in 3d space, protein segment #72,720,022,853 millions of aminos down the line
John Jumper mentioned at CASP14 on Tuesday that their structure prediction system uses "equivariant" transformers and, most importantly, is end-to-end, meaning they can backpropagate errors through the entire prediction system. Just FYI.
You're 1) smart and 2) a great teacher. Reeeally good video. Super entertaining and rich of information.
That alligator drawing though... 👌🤩
"rawr" lol
So the end after the references is basically the supplimentals. I'm not sure if that is common in CS papers but it just goes into detail on exactly how everything was done.
Nicely explained (even biochemistry)!! Loved it when you compared DNA to source code,protein as binary and the whole process to compilation.
I am a biologist interested in machine learning and AI and it is great to see this explanation.
BTW, the amino acids with the rings exist . But especially proline's ring with amine group inside the ring makes protein folding even more complicated.
You are the best explainer I have encountered - and the population in question is large.
Biology has come a long way since 1995
This also involves chemistry
@@poksnee and snacks don't forget the snax
Thanks for your effort in shedding light on this development
How about going backwards? Start with a shape you want, and from that generate the sequence needed to produce it.
I think the problem with this approach is defining the “shape you want” in the first place. It’s difficult for scientists to accurately estimate the shapes of proteins based on their function. I mean, to an extent. Like if your protein serves the function x, then it might contain structural components a, b and c. But estimating even crudely accurate xyz coordinates for all the residues would be an exercise in futility. Also, the nature of the protein folding problem is that we know the sequence but we don’t know the 3D structure. Since sequence is the defining factor in determine tertiary structure, we should start with the sequence and go from there.
it's been done for antibodies
@@EMSV66 is this because their structure is very well known/conserved. Perhaps you shed light on this “going backwards method” more than I can, it’s not my field but I’m v interested!
@@npm1811 Yes, that is part of the reason. You only need to model the variable region. With antibodies you are trying to create a complementary surface to the antigen's so you can start from the structure of the antigen. Look up the work of Costas Maranas at PSU.
@@EMSV66 that makes a lot of sense, thanks
Very good explanation, are you going to explain Alpha Fold 2 paper and RoseTTAFold？
I was pre-med, got a bachelor's in Biology and am in an MS for ML. I cried reading this news yesterday. I don't want to rain on the skeptics parade as I sympathize with that view a bunch (since I'm usually "that guy"). Assuming they didn't train on the test samples inferred during the competition, this is big progress and a big deal.
Is it worth pointing out that this is the same group a couple years ago that proved they can beat everyone on the planet at Go, Chess, etc?
Pharmaceuticals is the obvious beneficiary of this tech, but we could imagine some next level stuff too. Like a CRISPR that we didn't just steal from bacteria and reverse engineer. We could, in the future, create those cellular machinery on our own terms. People all know DNA, but it is just blueprints for the proteins, the proteins are the real important stuff which do the enzymatic work.
One small step for man. One giant leap towards genetically engineered cat girls.
Will this speed up the progress of biotechnology / synthetic biology in general, or just specific things?
@@GyroCoder given that all the important bits (the cellular machinery) of us are proteins, this will have large impacts across biology
@@rbain16 True, but not all biotech applications / workflows need to know the structure of a protein. It sure will help on many things, many of which we will probably not see coming; but I'd say that for now it'll mainly impact those applications where the spatial configuration of the protein can be exploited.
@@alvaromendoza4406 The structure and the function go hand in hand, maybe you can give me an example?
A structural biologist here. Decent explanation of protein folding. What is the best way to jump into neural networks for a newbie. Also, a comment on Nature papers. They have a print version that is shorter and an extended online version that contains the Methods section. So the short print version contains all the data with little explanation of how it was obtained. A deeper explanation of the methods used can be found in the extended online version. I hope this helps.
20:12 Till this moment. The Idea is that, train a network which reads the sequence of amino acids and predicts their distance. Stage 2 do the gradient descending in order to both generate something real spacial data in the form of vector3d and check if the prediction from stage 1 is possible. If it's not possible, let's say, distance(point 1,point 2) == 1, d(2,3)== 1,but d(1,3)== 3, the stage 2 has to deal with this, and gives out a result which fulfill all the distance prediction as possible.
Beau-ti-ful explanation. One of the best explanations of any subject I've ever seen.
Brilliant turn from a complex matter to a understandable subject.
Ge-ni-us.
Congrats and, above all, Thank You Very, Very Much. Beau-ti-ful.
This is super cool, glad that I found you!
The paper states that they used dilated convolutions, this made it possible to also model long term interactions. It is crucial, since protein folding is going to be highly dependent on those longterm interactions that determine the 3D structure of a protein.
I love the on the fly real human to human explanation and the fun that ensues. @2033s "My drawing skills are to be criticized in another video " 🙂 . Technically accurate and compact and relevant. Enjoyed it immensely in many ways. Thanks!
The iterative process is probably the folding step. Typically, when using one of the algorithms you will fold the protein thousands of times and build the final structure from a wieghed average of all the folded structures.
I wonder if the results would be improved further if they used a tetryonic / quantom field base model. Similar to prior alpha projects, human assumptions and training on some human data actuall was holding the project back for the final stretches of improvement.
When is the new video coming out for the recent Nature paper?
Please do an update about the Alphafold database
Hi! Thank you for such a great explanation. This has been so helpful. Would love an update with the published Alphafold2 paper!
I'm actually here by suggested videos, where the first video was about pimple popping.
OK, the algorithm of alphafold2 is different to alphafold version 1.0. they replaced EVERY part of alphafold version 1 routine. there is no point in discussing alphafold v. 1.0 at all. apples vs. oranges. different neural network, different physical constraints, different everything. the co-evolution matrices and the final 3d minimization function were separate routines, but they are part of a single neural network now. the network adjusts and rebuilds itself (its type changes) depending on the sequence. Etc, etc, etc. Once again. There is no point in reading that nature paper any more, the new algorithm is really different.
Even if it is not relevant, it is still interesting to hear how the old Alphafold worked.
25:23 It says "gradient descent on _protein_ _specific_ _potential_ ". I believe at that stage initial predicions are not used anymore (only as initial state).
Really there are only 20 amino acids found in proteins encoded by genetic code, Selenocysteine which is found in about 50 eukaryotic proteins replaces cysteine by incorporating Se in place of S, not a huge structural change. There can also be post translational modifications. I bet that some more highly ordered proteins, such as those with a large number of alpha helices, and/or beta sheets connected by more disordered strands are easier to predict as well as a larger data set from x-ray crystallography where protein homology aids the prediction. X-ray crystallography will still be important when looking at how other small molecules bind to proteins or how protein to protein interactions are observed.
Very nice summary
Good visual explanations.
Enjoyed the alligator.
Thanks.
Does anyone know, if there is a good source for explanations about convolutional neural networks?
I am a biochemistry bachelor, so I've basically no experience with computer science at all.... But it would be super helpful :)
And of course: Thanks for the great explanation, it was really entertaining to watch that explanation of the paper and it did really help to understand the concepts of the underlying math and computer sciences aspects.
Update: This video is great.
I watched up until 3:30 when Yannic says this video is for a "Machine learning person" rather than a protein engineering type person. I know quite a bit about protein folding and protein engineering and fairly little about the technical side of machine learning. Is watching this hour long video going to be a waste of time for me?
Also, how does this account for how the environment affects how a protein folds? For example, don't some proteins misfold in the presence of other misfolded proteins of the same type?
D.R.N.A. and the R.G.B. of the albelian ring heliocase of light Theora
As I remember it was a paper by the Salesforce team about 6 months ago on using BERT to predict binding points on the protein chains. Do you think that Google folk had sort of the same idea?
Bertology meets biology scored better than AlphaFold1? @yannic
1st bar: Yannic
2nd bar: Next best AI TH-camr
10:58 Yes they're called "beta sheets"
So instead of using chemical properties of each amino acid, they took the data science/ML approach to build a model based on all publish folding data?
I'm learning to be more skeptical of these breakthroughs, they might turn out to be overhyped
Yeah I have the same feeling after ML street talk about GPT-3.
@@quebono100 Another point I got from talking with an expert, is that with tech like GPT-3, you still need a way to properly mitigate and handle the bias it can produce, otherwise imagine all the incidents that will arise from "surprising insensitivity" in a gpt-3 application. :/
@@andrewwilliam2209 Im not an expert. But I think you can do it like they discussed, you can be a prompt engineer :) and simply discover promts and pin it with a seed. (they discussed just the promt engineer part, I expiremented with gpt2 and theire you can have a random seed)
You should always be slightly skeptical
@@mitchellty Yeah you right. It doesnt mean, that they didnt build something amazing. But nower days such companies try to fake it until they make it. Like UNU AI predicted the kentuky derby 3 correct winners in a row. I thought wow thats amazing how did they do such a difficult task. They truly predicted the kentuky derbi, but not only with theire software but also with expert who cherry picked the most likely winners. Its all marketing, to push the product.
As for AlphaFold2: I guess the pairwise distances are just for training part of the model. It may not be used in predicting the structure. Directly predicting the structure may be better than using the pairwise dist as the middleman.
A new paper :
Autoencoder Variationnal Auto encoding
I can't wait for you to disect their second paper when it's published!
thank you for explaining the research papers the way do .
I find very hard to understand by reading them; I am 3rd year Bsc student for Computer Science.
I love the fields and the papers you talk about so it definitely feels great understanding a bit more about the papers you explain!
It's amazing how people are explaining things they don't know about
ooof
Convultionss=>Tansformers, now thats a cool insight Yannic! Wondering if the paper is out?
I don't think it is yet
Quantum light generators and the duality of the double helix heliocase transfer of the R.G.B.z to rod's connectivity
wow, that was deja vu. I worked on a project that had the same basic approach as alphafold 1 (distance matrix+neural network+gradient descent) in the 90's. Of course the neural net was a simple backprop net and the sample set was much tinier, but yeah, basic same approach.
I've got to think that the deepmind group is infuriating a lot of pure science teams out there - they descend into their territory, clean their clocks on specific problems and then go 'hey - look how great we are'. Its GOT to drive the original scientists there absolutely nuts. But then again, I don't really mind as long as I get flying cars out of it. :)
"infuriating a lot of pure science teams"
Diddums. I'd rather have better medicines than scientists without injured egos.
@@dionbridger5944 well, yes that is what I was getting at with my "not really minding as long as I get flying cars out if it" comment.
Yet still there is something about the tone of the deepmind folks that reminds me of the masters' attitude from monty python's 'meaning of life' skit about rugby at boarding school: th-cam.com/video/HKv6o7YqHnE/w-d-xo.html
Thank you for this informative video. Can you do a similar one for alpha fold 2 as they have now published the paper?
Hey Yannic, great work on you videos, I really appreciate it. Could you cover the topic of lavasz loss and the respective paper "The Lovasz-Softmax loss: A tractable surrogate for the optimization of the intersection-over-union measure in neural networks" in future videos? I read that this specific jaccard approximation is used as loss function in many image segmentation tasks, even in U-net it has been observed that it gives better results. However it is not straightforward and I still don't fully get it. It would be awesome if you could create a video for it, breaking down the concept as you have done amazingly so many times. thank you :)
Today I was informed that AlphaFold 2 presentation for CASP14 is online, and it contains more information than DeepMind's blogpost (a bit more info about the model on slide #10):
predictioncenter.org/casp14/doc/presentations/2020_12_01_TS_predictor_AlphaFold2.pdf
Presentations of all participants:
predictioncenter.org/casp14/doc/presentations/
For AlphaFold 1 and all other participants (CASP13 - 2018)
predictioncenter.org/casp13/doc/presentations/
This is huge. I can't wait to see where machine learning and AI takes us in the 2020s and 2030s.
5 seconds into the video and I can already tell that you're going to throw a bucket of cold water on this paper XD
There is no paper to throw water on.
@@herp_derpingson unfortunately yes, that was a quick guess before even watching the video
@@herp_derpingson there is. www.nature.com/articles/s41586-019-1923-7.epdf?author_access_token=Z_KaZKDqtKzbE7Wd5HtwI9RgN0jAjWel9jnR3ZoTv0MCcgAwHMgRx9mvLjNQdB2TlQQaa7l420UCtGo8vYQ39gg8lFWR9mAZtvsN_1PrccXfIbc6e-tGSgazNL_XdtQzn1PHfy21qdcxV7Pw-k3htw%3D%3D
@@lolerie That paper addresses an earlier model of theirs at CASP13, two years ago.
First good analysis of alphafold2 i see, compared to other click bite news is really refreshing
I feel like the predicted torsion are more important than you say. For me the (torsion+distances) will be somewhat inconsistent, and the gradient descent enforce consistency. But given the space of search I'd bet you need the initial guess to be good to not get stuck in a local minima. Just my intuition though.
Good overview, well presented, thank you!
SUperb video. What application do you use to create these videos? Id like to learn to draw and scroll through images while recording my voice over just like your format.
I'm going to be sceptical because this type of problem seems incredibly similar to solving conway's game of life, but way more complicated. We haven't even figured out the super-simple version yet.
Thanks for the video. Understandable for non-AI people too!
Great video like always. So it seems like DeepMind went directly from predicting something similar to adjacency matrix to transformers. I was wondering if they ever implemented spectral graph analysis here.
The Chladne´s plates experiments could help in someway?
'Pyrene ring' isn't a thing. It's called an aromatic ring. Beta strands are joined by hydrogen bonds, to form a beta-pleated sheet. The beta sheet is the reason why the protein-folding took so long to solve. Hydrogen bonds of alpha helices are local, whereas beta sheets have global bonds.
33:54 - finally we're on the same playing field! )
Thanks for the break down, I hope i'll get to your other videos soon as well
It's like after GANs era. Transformers are everywhere.
Great vid, highly informative, very interesting and you are unquestionably the 'Richard Burton, Morgan Freeman and Jeremy Irons' hybrid voice of science vids! 😃
Thank you. What stocks and sectors will benefit from this?
Watched one AlphaFold video and I'm now getting advertisements for pre-weighed biochemical research sample blisters.
Loss function is like L²u for a Lagrangian progress error counting, but these Austrian mathematics school is really neat for angle repair in the aminase codification.
Hahaha, awesome! Thanks!
Awesome explanation, please do a video about AlphaLink 🙏🙏
Thanks for the video. I studied biology and love tech. The ribosomes produce the amino acid chains which then go to the Endoplasmic Reticulum (ER) and then for further modification in the Golgi Body. Metals such as iron or magnesium along with atoms such as N are added. The question is what is conducting this process? It is more than residue attraction/repulsion and torsion angles. Possibly there are other proteins guiding the folding. What about 2 identical amino acid chains producing different multiple outcomes?
The explanation torsion angles is false. What is described and drawn at minute 20:49 are angles (3-body terms), torsion angles are dihedral angels between the two planes spanned by 4 neighbouring atoms (or any type of points).
This video is amazing! Agree with the comment "Yannic Kilcher is all you need".
I wonder if there is another new discovery theologic layer never imagined that surprises teams.
This is revolutionary!
Two different structures will always have different pairwise distances? That seems to be assumption of 2nd stage. Don't know enough math on this.
Why is the paper STILL NOT RELEASED?! IT'S BEEN 4 MONTHS!
Im not a machine learning person, i barely even know what a protein is, how to flip it, twist it, fold it to a paper plane or similar is far beyond my knowledge.. i am however a HTML programmer, made everything from divs to s, to websites build completely out of tables or even better canvas and SVG.. im currently trying to make an AI with HTML, marquee is the core
Chaos could never produce these things.
Autobots, fold out!
underrated.
Very educational video…thank you
thanks for the great video! :) and... since I am coming from the field of bioinformatics, I really enjoyed your confusion about the format of Nature papers - I remember I had the same strange feelings about it when I started to write papers for similar journals.
Honestly I am so fascinated by the papers but as someone who is new to the domain...it feels overwhelming...for eg..in this video the transformers were like a prerequisite and I feel there might be some prerequisite for every video maybe...
So is there like a playlist or some flow that someone could suggest.
Thanks
Great job!. By the way which app are you using?
OneNote