How To Simulate The Universe With DFT
ฝัง
- เผยแพร่เมื่อ 11 ม.ค. 2022
- PBS Member Stations rely on viewers like you. To support your local station, go to: to.pbs.org/DonateSPACE
Take the Space Time Fan Survey Here: forms.gle/wS4bj9o3rvyhfKzUA
*****
Sign Up on Patreon to get access to the Space Time Discord!
/ pbsspacetime
If you used every particle in the observable universe to do a full quantum simulation, how big would that simulation be? At best a large molecule. That’s how insanely information dense the quantum wavefunction really is. And yet we routinely simulate systems with thousands, even millions of particles. How? By cheating. Using the ultimate compression algorithm: Density Functional Theory (DFT). Let’s learn how to cheat the universe.
Check out the Space Time Merch Store
www.pbsspacetime.com/shop
Sign up for the mailing list to get episode notifications and hear special announcements!
mailchi.mp/1a6eb8f2717d/space...
Hosted by Matt O'Dowd
Written by Graeme Gossel & Matt O'Dowd
Post Production by Leonardo Scholzer, Yago Ballarini, Pedro Osinski, Adriano Leal & Stephanie Faria
GFX Visualizations: Ajay Manuel
Directed by Andrew Kornhaber
Assistant Producer: Setare Gholipour
Executive Producers: Eric Brown & Andrew Kornhaber
Executives in Charge (PBS): Adam Dylewski, Maribel Lopez
Director of Programming (PBS): Gabrielle Ewing
Spacetime is produced by Kornhaber Brown for PBS Digital Studios.
This program is produced by Kornhaber Brown, which is solely responsible for its content.
© 2021 PBS. All rights reserved.
End Credits Music by J.R.S. Schattenberg: / multidroideka
Special Thanks to Our Patreon Supporters
Big Bang Supporters
David Taiclet
Ben Dimock
Daniel Alexiuc
Nenado763
Pravin Mansukhani
Peter Barrett
Nils Anderson
David Neumann
Charlie
Leo Koguan
Sandy Wu
Matthew Miller
Ahmad Jodeh
Alexander Tamas
Morgan Hough
Juan Benet
Vinnie Falco
Fabrice Eap
Mark Rosenthal
David Nicklas
Henry Van Styn
Quasar Supporters
Alex Kinsey
OnlineBookClub.org
Alex Kern
Ethan Cohen
Stephen Wilcox
Yogi B
Christina Oegren
Mark Heising
Hank S
Hypernova Supporters
william bryan
drollere
Joe Moreira
Marc Armstrong
Scott Gorlick
Nick Berard
Paul Stehr-Green
Alexander Price
Adam Walters
Russell Pope
Ben Delo
Nicholas Newlin
Scott Gray
Антон Кочков
John R. Slavik
Mathew
Donal Botkin
John Pollock
Edmund Fokschaner
Joseph Salomone
chuck zegar
Jordan Young
m0nk
Daniel Muzquiz
Gamma Ray Burst Supporter
Avi Yashchin
Michael H Lev (SHS '78)
Kory Kirk
Terje Vold
Anatoliy Nagornyy
comboy
Brett Baker
Jeremy Soller
Jonathan Conerly
Andre Stechert
Ross Bohner
Paul Wood
Kent Durham
jim bartosh
Nubble
Chris Navrides
Scott R Calkins
The Mad Mechanic
Ellis Hall
John H. Austin, Jr.
Diana S
Ben Campbell
Lawrence Tholl, DVM
Faraz Khan
Almog Cohen
Alex Edwards
Ádám Kettinger
MD3
Endre Pech
Daniel Jennings
Cameron Sampson
Pratik Mukherjee
Geoffrey Clarion
Nate
Darren Duncan
Russ Creech
Jeremy Reed
Eric Webster
David Johnston
J. King
Michael Barton
Christopher Barron
James Ramsey
Justin Jermyn
Mr T
Andrew Mann
Isaac Suttell
Devon Rosenthal
Oliver Flanagan
Bleys Goodson
Robert Walter
Bruce B
Ismael Montecel
Simon Oliphant
Mirik Gogri
Mark Delagasse
Mark Daniel Cohen
Brandon Lattin
Nickolas Andrew Freeman
Shane Calimlim
Tybie Fitzhugh
Robert Ilardi
Eric Kiebler
Craig Stonaha
Martin Skans
The Art of Sin
Graydon Goss
Frederic Simon
Tonyface
John Robinson
A G
Kevin Lee
justahat
John Funai
Cass Costello
Tristan
Bradley Jenkins
Kyle Hofer
Daniel Stříbrný
Luaan
AlecZero
Vlad Shipulin
Cody
Malte Ubl
King Zeckendorff
Nick Virtue
Scott Gossett
Dan Warren
Patrick Sutton
John Griffith
Daniel Lyons
DFaulk
GrowingViolet
Kevin Warne
Andreas Nautsch
Hey Space Timers! At 10:30 Matt accidentally misspoke and called the Hohenberg-Kohn Theorem, the Kohn-Hohenberg Theorem. We know you're all an incredibly thorough bunch and would never miss a technicality like that, so unfortunately for us it's time to reset our "Days Without An Accident In Explaining the Universe Counter" back to "Big Bang."
8:45 "Make sure you get the order right or it won't work."
What an energy efficient civilization used all the energy even the info red. . Avoid would be the thing to look for. . What if aliens us the heat . How about star ship
What about traveling back in time as far as time is moving forward
aww not going with "The quantum simulation that was Matt had a statistically improbable event that was totally not his fault" or something?
You mean set the counter from "Big Bang" to "today"!
Thought this was going to be about Discrete Fourier Transforms. Great to learn
Me too!
I know, right? I was wondering how they would get sufficient time resolution.
Same!
Lol me too!!
Glad DFT didn't turn out to be Discrete Fourier Transforms. Was a bit worried about how that was going to work.
(Engineer who's led a sheltered life)
This sounds like the how calculus was invented trying to measure the area under a curve by making a rough estimate with a bunch of rectangles making them smaller and more numerous until you get close enough to reality.
Discretizing something that is continuous is often the first step to come to a solution.
It looks like a very similar thing we do with NP-complete problems in computing science. Most of the NP-complete problems in the real world have a specific less-dimensional estimate solution that you can iterate and get more accurate results instead of consuming the entire life-time of the universe multiple times over.
Its really like a topological Lie128 super-connected group in mathematics, but in reality, things have a super-structure that's not entirely complex, the graph has a lot of unconnected vertices, so you can guess and then iterate. Its very similar to the method demonstrated on the video. I does really have something to do with compression of information.
I think there's something very, very fundamental about computing and physics that's being missed by the excessive overuse of continuous calculus and analysis (its just an abstraction, its not the real thing after all). The universe doesn't seem to like being continuous, but physicists are a bit stubborn on their continuous equations.
@@monad_tcp This might be a bit of a stretch, but in networking, it's practical to have a routing table for the entire IPv4 address space where each different address could be programmed to route to a different output on a router. It only takes 4GB of RAM to handle 256 possible output ports because there's only 32 bits of address.
With IPv6 the space is so large that you're forced to use non-brute force schemes to split it up. Sort of like the fewer dimensional superstructure mentioned in the video.
@@big0bad0brad I think that we are fundamentally unable to fully understand how the universe works...just like a simulated self aware character in a computer game. It would be able to find an aproximate solution for the behaviour of their universe but will never learn how the computer that runs the simulation works.
@@santiagotomasso5184 thats because the higher universe in which the simulation is running is much more complicated. You cannot simulate more complexity because then you run out of memory which requires some crazy amounts like all particles in star system. Maybe the higher universe has 20 normal dimensions and our 3 dimensions are child’s play for them.
Wow, finally one that I'm familiar with, I use DFT daily in my job! (Edit: I had said some stuff which was incorrect, deleting that bit. Thanks to John Ferrier for the correction). Anyone interested in reading about all of this can look first at the Hartree Fock method which is the starting point for basically all modern computational chemistry, which then each have various tricks to include the correlation. DFT scales pretty well though, which is why someone has done that virus capsid (although was that strictly DFT? It scales as N^3-N^4 still... Which is unspeakably better than 3^N but still expensive as N gets into the thousands). There are so many interesting methods out there. Semi empirical methods are so cool too, moderately accurate and blazingly fast. And just the way all of these methods actually do the math in the calculations, by stacking up "basis sets" of simpler functions to approximate atomic orbitals but which have really nice properties for calculating the various integrals, is just so cool.
If you want to continue the computational chemistry episodes, you should do an episode on the PPP (Pariser Parr Pople) method - it's simpler than DFT (it came earlier. People actually used to do these by hand), is used to calculate electronic transitions in aromatic molecules and is surprisingly accurate for such a simple method - and this is the real kicker - it was later worked out that it's an approximation to quantum field theory calculations which is why it's so accurate. I've always been super interested to find out how they relate, but the math has gone above my head and I haven't found any sources that attempt to explain it in an approachable way. I think that would tie in really well to this channel!
NEEEEEEERRRRRRRDDDDDDDD
Great comment. Thanks for sharing
@@jeromegreen3654 *looks at title of video and channel*
Thanks! I'm trying to get into bioinformatics and since I have a special interest in protein engineering, I was trying to learn more about computational chemistry. I had a hard time separating the "intro material" from the stuff that's actually being used today and figuring out what advancements were currently being worked on in the field to make QM/MM calculations faster and/or more accurate, especially for larger scale systems like proteins.
Thanks a lot for your insight. This and the video together cleared a lot of confusions I've had in the brief time I've began to study DFT to better understand one of my professors' research papers. I also love your explanation on the semi-empirical methods - it sounded a lot like Fourier analysis but for many-particle quantum systems.
You mentioned that the Hartree-Fock method is the starting point for modern computational chemistry. Does that mean the HF method predates DFT? In many of my studies, the DFT calculations were compared to HF counterparts, and that's where I've heard the name and it's pretty much all I've ever known about the method. Never digged any further thinking it was far more complex than DFT.
I'm poorly educated and I only understand little if anything from this channel, yet I can't miss a single upload and know they'll be inspiring other kids out there to one day become a scientist too.
yaeh you and everyone else in the American education system. But by watching shows like this you're slowly but surely changing that for yourself. You will absorb stuff over time even if you don't actively understand it as you watch it.
@@TestECull It's social engineering the US education system. Give them the 3 Rs reading, riting, rithmetic and a bunch of the most profound lies about their history and you're good to go. The other social engineering is low health care and low or no unemployment benefits. Combined you get cheap labor and it makes your products competitive in the domestic and global markets. Downsides? A permanent underclass of poor people and the associated crime that goes with it, but they have guns to control those types.
@@ThePaulv12 lol you were doing great right up until you brought guns up. Then you just fell off the rails totally.
Go read our constitution and try again. Specifically, the 2nd Amendment. It spells out in crystal clear text why we have so many guns. Hint: It isn't to keep the poor under control.
And yet guns are primarily used to kill poor people, regardless of the 2nd amendment
@@noyou1114 Doesn't mean that's why we have them. And they aren't really used to subjugate anyway; that's done with gaslighting via Fox News. Get the poor people to vote for politicians that cram them into the dirt under one bootheel then demand they lick the other while claiming they're angels and it's the other party being cruel to them.
But I wouldn't expect someone exuding insane amounts of trans-atlantic smugness to realize this. Just latch onto 'hurr americans guns bang bang shooty' like every other motherfucker that tries to criticize us from abroad. Protip: If you wanna know why we are so quick to disregard such criticisms and/or respond to them with mockery and derision, there's your answer.
I did a few different kinds of DFT calculations in grad school. Never thought I'd see it discussed in a video for the lay person.
I have used the program VASP once or twice but the theory or the procedure inside the program is too complicated for me to understand
@@hyperduality2838 ???
We had WIEN2k at the start because that's the license we could afford as a research group and I ran it just on an old personal computer that I brought in. It was enough to calculate a few different stoichiometries of tin oxide and link a feature in the electron density to the size and nature of the bandgap thus explaining the coexistence of conductivity (p-type I believe) and transparency and how to potentially tune the properties with oxygen concentration. Later on I worked on running some code that ran on top of VASP in the background that attempted and mostly succeeded in predicting the O K-edge X-ray absorption spectra of oxides which is actually kind of hard. It required basically putting back some of the extra configuration space information due to high-ish electron correlation if I recall correctly. I only ever knew enough to setup and run the code and to know if the results could be trusted. I was interested and read up on the basic theory but was never well versed enough to contribute anything to actual coding or the theory of DFT itself.
I did all my DTF research in undergrad
as someone whose research uses density functional theory every single day this episode makes me so happy 🥰🥰🥰
my thoughts exactly :D
DFT Gang rise up!
Me too, and being a beginner the video really helped me understand the theorem better.
@@manjeetgodara2225 me watching this video: i am very smart, i am very good at this [VS.] me, staring at a segmentation violation for a simple B3LYP optimization: why won’t my molecules dance ))):
@@thelocalsage Let me guess, you are using Gaussian?
The animator is absolutely _crushing_ it lately; there's a very artful, demoscene vibe to these little animations and there's _so_ much detail 🌌🖥️🌇
“Make sure you get the order right or it won’t work.” It took me a solid 4 second pause to figure this out. And then I laughed so hard I choked. Well done, guys. 🤓😂😂😂
Really neat that this channel is going places no one else on TH-cam is. A lot of interesting things to hear about for the first time for me. Great topic.
Obviously you don’t know many channels then
@@MasterBlaster3545 lmao
@@MasterBlaster3545 Perhaps you can recommend other channels. Thanks
Physics videos by Eugene Khutoryansky
Science Asylum
Veritasium
Anton Petrov
Up and Atom
Sabine Hossenfelder
3 blue 1 brown
Those are just the ones popping to mind right now, there are tons more.
@@RandyJames22 You Tube does that already. Smh
Currently using DFT for solid state physics. So happy to see this reach the mainstream community! No surprise that it was PBS Spacetime and that they nailed it :) this channel is great
@@hyperduality2838 not sure what you’re getting at
@@hyperduality2838 you haven't actually expressed any thermodynamic law. at best, you're closer to some mathematical idea. at worst, you're just stating relationships between quantities
@@lattice737 don't worry about him, he's just some delusional guy who goes around youtube comment sections of pop-sci videos spouting non-sense like that
I am curious do we calculate the error +- in our DFT calculations?
@@v44n7 Yes, the precision of a "good enough" electron density can usually be decided by the user with an input file. In that way, it's similar to replicating a theoretical result with an experimental procedure. You can be reasonably sure that your experiment has been done well enough if you get within some range of error. Otherwise, you will need to spend more time tuning your instruments and reducing noise. More accurate DFT calculation means more computational time
The deeper discussion of quantum mechanics is much appreciated. Many scientific communicators shy away from talking scientific complexity in front of lay audiences but this channel strikes a great balance between explanation and technicality.
One of the most interesting (albeit disappointing) results of DFT is what happens when you apply computational complexity theory to it. As it turns out, whatever the exact functional looks like, it cannot be simple enough to be evaluated in polynomial time, not even by a quantum computer (!!) Paper: 10.1038/nphys1370
Interesting. That's assuming P =/= NP?
@@pavlenikacevic4976 Yeah, I think. Although the relationship between QMA, BQP and the classes relevant on non-quantum computers is a lot more fuzzy.
Trying to see if I understand this: the exact functional is the "universal functional" right? which is supposed to map from any input charge density to info about the ground state structure and we try to approximate this functional with DFT and other wavefunction methods. And what the paper is saying is that the universal functional cannot be computed in polynomial time, quantum computer or not? I feel like multiple points of what I just said was wrong lol.
Thanks for sharing
Really love these practical videos that show equations and methods most physicists actually use!
@@matthewhanson239 No, a lot of physicists also use DFT and many other methods not related to quantum field theory.
@@tybaltmercutio Hmmmm... what about QED?
@@davidpalmer9780 What do you mean?
I didn‘t say physicists don‘t use quantum field theory at all. It depends on the field you are working in.
However, a lot of physicists only work with DFT or some higher level ab initio approximations to solve the Schrödinger equation without ever touching quantum field theory.
The same holds for relativistic corrections and adiabatic corrections: If the systems you are working on don‘t require these corrections/theories, you don‘t use them.
@@hyperduality2838 blink if you need help
@@tybaltmercutio very true, I mean the whole field of ‘condensed matter’/low temp physics hardly if ever uses QFT lol. There are a few instances (circuit QED in quantum computing I think uses fields in calculations) but other than a few select others most of the time you’re working with density matrices and master equations that spin off from the basic schrodingers equation when doing research.
8:37 I'll be honest, first legit laugh out loud moment watching this channel in quite a while.
Omg SAME. It took me a solid few seconds and then I laughed so hard I choked. 👏👏👏
8:44 He could have said "If you don't get the order right, you're screwed."
I loved this excursion into Physical Chemistry!
I am a quantum chemist, I'd like to add a few important points here:
1) The video seems to imply that DFT is the only way to avoid this unrealistic grid size. Other approaches, so called Wave Function methods, do not use grids, instead the basis set approximation is invoked. In fact, the basis set approximation is also used in DFT.
2) DFT is highly dependent on the function choice. This choice is very hard because one functional maybe better for one problem while terrible for another, and there is no obvious way to know it without benchmarking. Wave function methods, in that sense, are more robust and less arbitrary.
3) Saying that DFT results are accurate raises the question, what is accurate?. It depends on your target accuracy really. I guess in general it can give you a decent qualitative picture, good structures, etc. I have never heard of precise thermochemistry being performed with DFT. In fact, it is normal to compute structures using DFT and refine the energies using wave function approaches.
4) In both DFT and the Hartree-Fock method, an effective interaction between electrons replaces the instantaneous one. They still interact though.
5) Now what bothers me the most... Many DFT functions are not really ab initio. In other words, they design the method to return the correct answer! Thus, you are not really getting the right answer for the right reason, you are getting it because you baked it into the model! Nothing wrong with it as a pragmatic tool, but it is hardly a show off of quantum mechanics solving chemistry problems.
Yes, DFT is by no means a "holly grail". In fact, from a computational pov, the most popular version of DFT, namely Kohn-Sham DFT, is not that different than Hartree-Fock. They're virtually identical to code, except for an extra term in DFT.
Exactly! Very well put.
Wow! The visualization of dimensions with dice just baffles my mind! That is the best way i ever heard to imagine more dimensions. My mind is so blown that i have to stop the video and watch it again tomorrow, because i cant think of anything other than dice!
I love how you specified the letters DFT and specifically in that order seemingly because you didn't want people trying to simulate the big bang with DTF.
I was honestly kinda surprised Matt knew what DTF stood for tbh 😂
@@ShadowWizard123I think you have the wrong read on the guy, I bet he knows all the acronyms.
I had to scroll way too far to find this comment. I think it went over alot of people's heads. I had a really nice chuckle over it.
Yay ... my Ph.D. is on Density Functional Theory (DFT) and Time-Dependent Density Functional Theory (TDDFT)
Cite this video as a reference in your thesis.
Wow!
Nice Im aiming at a PHD in DFT for computational Chem
So how many mistakes in the video then? =P
Needed DFT for my PhD too, although none of that ended up published
Oh! I recognise some of this!
In my Honours project we were simulating XPS and NEXAFS spectra for pentlandite. I remember reading up that previous studies had used DFT or Hartree-Fock methods but were insufficient for systems with strong correlation or charge transfer. I also had no idea what I was doing haha, so it's nice to get an idea of the fundamentals.
Ha! I was hoping it was Discrete Fourier Transforms - I work with Cellular Automata to try to understand and simulate artificial life & the conditions for abiogenesis, and DFT/FFT is one of the simulation techniques!
Still, never disappointed by a PBS Space Time video!
very cool. when you talk about simulating artificial life, do you mean you simulate the internal biochemical reactions of the cells (like systems biology) or do you investigate life more on the phenotypic properties/species interactions?
@@amentrison2794 More like systems biology - I know it's a virtually impossible goal, but I'd like to do my best to find and understand pre-life chemical evolution by using these bottom-up simulation techniques
WOW you where able to summarize in a single video almost the entire content of the first 11 chapters of MaQuarrie and Simon's Physical Chemistry: A Molecular Approach that took me almost a month to selfstudy. Mad respect brother!
Respect for self studying the details.
Ah! As a chemist myself, this episode really hit close to home.
Same! Reminds me of logging into my school's supercomputer and staring at screen for hours trying to figure out why my DFT code failed
"Is this physics or chemistry?"
"yes"
Watch Rick and morty Season 2 Episode 6
Too many years wasted running dft jobs on the supercomputer
Lol, a chemist
Love how I hear about how DFT is important for the first time near the end of my undergraduate degree and then soon later I find I am learning what it actually is from a "pop-sci" youtube channel. Love how I still find the content here relevant!
Wow that's the first time I've seen someone try to describe DFT on youtube. Cool. Probably better to stress more that we DON"T KNOW THE ENERGY FUNCTIONAL. So there are infinite DFT's that get us closer to the truth for some things and farther for others. At least we have the variational principle! You should explain that one. Really screw with people's imagination... Thanks!
Whoever does the animations for this show is a master: having to go through all sorts of different numeric simulations, pull them out of the relevant software, then get them into the video requires some very special knowledge.
Tensor networks are another great method of simulating the low-entanglement subspace of quantum mechanics, informed by quantum information theory. They are more general than DFT, able to support interesting entanglement structures.
I guess, almost any ab initio theory is more general than almost any DFT functional. 😁
I agree, TNS are super interesting.
Learnt about DFT last year in a 300-level physical chemistry paper and thought it was really cool! We got to do some computer work using DFT and a couple other types of approximations as well!
9:26 Technically, DFT itself is an exact theory and does not ignore the electronic correlation. Even many approximate functionals explicitly have correlation-exchange term.
„Bare“ DFT in its current still is a mean-field theory without any dynamic electron correlation. Of course, if you add MP2 to your DFT functional, you also take into account dynamic electron correlation.
@@tybaltmercutio You are talking about the DFA (Density Functional Approximation) which uses approximation to the universal (exact) functional of DFT.
@@meisam9592 These two things are usually used interchangeably, aren‘t they? I have never heard of anyone who actually uses DFT in practice (be it method development or applications) who makes a distinction between DFT and DFA (and I have read a ton of papers and attended many conferences). I guess, this is related to the fact that the exact functional is unknown and, hence, not relevant in practice.
Furthermore, there is no way to systematically improve the approximations towards the exact functional.
4:13
This is a similar reason on why the three body problem from gravitation is so hard to solve
I'm commander Shepard and this is the best science youtube channel on the Citadel
Good to see Space Time made it to 2022. It's looking well so far!
I remember my quantum II class when we learned to sinulat a single hydrogen atom. It was very difficult. I dropped the physics major that year. Granted, computers have come a long way in the 20 years since, but it was tough.
No computers were needed for what you did. I went further and built simulations of multi-electron molecules using the Schrödinger equation back in 1991. I can really say that computers to this much further.
@@idjles Yeah and also numerical methods have been used back then aswell
I just used DFT to calculate the 'greasiness' measure (Octanol/Water partition coefficient) of an array of molecules. The tools for using it have come a long way and now AI tools are starting to be developed which speed up the calculations even more.
I've worked on machine learning models for speeding up DFT calculations! Surprising to see PBS talk about DFT at all
@@treyshaffer That's really interesting. What kinds of things did you do to try and speed the calculations up and how did your models differ (since it sounds like there was more than one)?
13:29 You seem to have the entire artwork collection of the three most influential physicists ever...Nice!
This is such an amazing video, I never see videos about Hartree-Fock and DFT.
I understand it's not feasible to speak everything about these theories in a single video but there are some points i'd like to contribute:
1ª - It is not possible to solve the Schrodinger equation for a system with two or more electrons because there is no known solution. But it is possible to achieve an approximated wave function within 99,99% of accuracy compared to the true wave function using Configurarion Interaction, but again, currently we have no computer power to do that for big systems.
2ª - DFT is more efficient because the electronic density field is a three dimensional field instead of a 3N (with N being the number of particles or electrons) dimensional field.
3ª- Hartree Fock is not always more accurate than DFT. It really depends on the DFT functional you are using and what you are calculating. DFT is heavily parametrized and you can make it very accurate. Some hybrid DFT functionals use Hartree-Fock to some extent and they are pretty accurate for the general case.
With the incredible amount of power that would be required to compute even a minuscule number of these interactions... do the ACTUAL interactions also require that much effort from the universe? And is it effort we are able to measure in some form, or is it all still mysterious at this point?
I’ve always wondered this
I also commented on this, I believe the number of percales in the universe to be irrelevant we have an infinite number system, we can calculate what we want as long as we have enough processing power. I feel like its an infinite/ loop paradox? You have/require information to store information. Almost like what does the universe exist in, and what does that exist in and what does that exist in and what does that exist in....
What do you mean by „actual interactions“?
What do you mean by whether „we are able to measure“ it?
DFT != DFT but all functionals have a well-defined computational complexity which can range from O(N) to O(N^5). However, the linear scaling can only be a achieved with some additional approximations or for some very simple functionals. The O(N^5) scaling could probably also be reduced by using additional approximations based on the locality of electron correlation.
I feel like the universe does not care about the cost of computation, only the rules used, it’s like if god is just an indie dev, trying to fit the code for a game into a single tweet
Who knows really
I don't always have the time to sit and digest all of the information as well as I'd like but I love that you keep making these. I can definitely appreciate how quantum functions require ridiculous storage and computational power to perform these calculations but it's really eye opening that doing some of these calculations for a single iron atom (outright) is essentially impossible. Glad there are smarter people than me looking at these things.
Hi. Myself I came to understand this by watching every credible science video of this type for the last four years. And I have learned enough to hypothesis my own theory s that most people can understand . My theories combined all disciplines of science and religion.
I think it's funny how scientist s think they're smarter because they don't believe they believe in a creators of this very improbable universe.
.
@@michaelelbert5798 Impressive! Just a couple of hours to understand everything people usually need several years of dedicated studying to understand. And you even improved the existing theories ..
@@tybaltmercutio that's sarcasm, Right ?
No offense to you. If not then thanks for the compliment.
@@tybaltmercutio also thanks for your research time it took for you to know that about me.
@@tybaltmercutio but I am standing on the shoulders of those giants.
After programming a 1D DFT for semiconductor materials in the last few years, this is the first time I’ve really understood it :-)
DFT? More like "This has got to be" one of the coolest video on TH-cam! I love this series; never stop making these videos!
I've wondered about this for quite awhile. I have some vague understanding of quantum theory, but it always seemed to me that to calculate anything useful would absurdly and prohibitively expensive.
That depends on your definition of „useful“. DFT is not useful at all in many areas of quantum chemistry or physics since it can be rather inaccurate. Furthermore, the way DFT is presented here is totally misleading since what is described as „cheating“ here only works in „theory“ if the exact functional was known (which it is not). For actual calculations it still is necessary to introduce Hartree-Fock like orbitals which means the „cheating“ part as described here does actually not exist.
However, also with higher level approximations, which do not suffer from the weaknesses of DFT, it is already routinely possible to compute energies, ground and excited properties and so on and so forth for molecules with several hundred atoms.
@@tybaltmercutio okay but you’re clearly discussing things well beyond the scope of the video, which is not intended to be rigorous but accessible. I agree that “cheating” is a poor word choice, but every theory has limits of usefulness for certain conditions. OP appears to be commenting on how neat this idea is, and you’re replying pedantically
@@lattice737 In my eyes, the whole video gives people a totally wrong idea about DFT and, hence, what method development in quantum chemistry/physics is about and what the real challenges are to to reduce the computational complexity.
There are dozens of comments to this video which are talking about some kind of compression of information or how DFT is the theory which makes actual quantum calculations just possible. And if you just watch this video this is an understandable conclusion to draw, even though, it is totally wrong.
While DFT is a nice idea/theory it does not work like that in practice. I‘m not sure whether they didn‘t understand that themselves while making the video. But this point should have been made in the video.
A much better approach to this topic would have been (potentially in a series of videos) to start by explaining the most fundamental approximation, i.e. the Born Oppenheimer approximation. From there, one could then explain the difficulty of solving the N-electron problem (and also the nuclear Schrödinger equation). One part of that could then be DFT including the note that this is elegant in theory but in practice it does not work because …
On top of that, one could then start introducing the concept of correlated wavefunction based methods.
@@tybaltmercutio I think you should consider the intended audience of the video more carefully. The video presents DFT as a mathematical approach that reduces the computational expense of solving the many-body Schrodinger equation. Because the video keeps the discussion to the density functional principle and does not touch on implementation, I disagree that it is misleading. If the video had decided to give an example of a numerical implementation (DFT in practice), I would be in a better position to agree with you, because it would beg the questions of "why this functional?", "why this package?", etc.--"why choose this specific implementation to represent DFT as a whole?"
These videos are meant to raise questions about the content, not lead viewers to a complete understanding, so the fact that there are commenters who are drawing false or incomplete conclusions (e.g. information compression) is a reflection of those commenters' lack of education and/or curiosity. When I was first learning about DFT, my peers also tried to summarize it. My response was always many, many questions, and that's what the proper response to this video is. Viewers should ask: "Why is this approach valid?", "What about the time-dependent case?", "What's the difference between these equations (1:08)?", and many more, including "When does DFT break down?".
It looks like the approach that you suggest is supposed to be better because it's more complete. However, PBS Spacetime does not publish academic lecture videos, and your suggested presentation is how most, if not all, formal curricula introduce DFT. In that case, the professors can be confident that the students attending the lecture have the mathematical and scientific knowledge to understand the proof for the density functional principle and learn about different implementation techniques and their advantages/disadvantages. This channel targets an audience with limited mathematical and scientific experience, so the video avoids mathematical proofs while explaining the general principles simply, often using analogies and many visuals. And what I love about this channel is that it makes this topic accessible enough that I wouldn't be surprised if someone did ask one of the questions above and ended up watching one of those lectures after finishing this video. In my opinion, that would be a successful outcome for a video like this--introducing a topic accessibly and interestingly enough that someone watching decided to learn more.
Now we need a video on "how to stimulate the brain with DMT"
I love the content here! Most of the verbiage of these videos is beyond me, but I understand the content. I am not "properly" educated in these advanced theories, but I have been learning on my own for decades. Thanks!
OMG. For most of these videos, I can follow along for about 10-20% of the content. This one sent that number plummeting to no more than 5%. I love it!
lol DFT is awesome. I’m trying to use DFT for microkinetics using quantum chemistry [basically finding discrete transition-energies for chemical rxns per step]. Really hard to get into but really powerful and approxmating towards realistic results.
Nice me too. Im trying to set up NEB calculations but even for simple reactions there is alot to account for.
@@kingfisher1638 Oh really? Are you using any specific software? I tried Amsterdam Suite as a trial and thought it was worth paying for, if you have the reaction mechanism and crystal paramters ready to go. The GUI makes it easier, but its very expensive. I'm using orca after a lot of research, because I might need to do an aqueous/solvation model. Just asking because not many people are working on this at my institute.
@@sauskeuzumaki121 Yeah I'm using Orca and the ASE package. Using the solvation methods in Orca is super easy.
@@sauskeuzumaki121 What about e.g. pySCF?
Q-Chem and Turbomole are also quite nice but I‘m not sure now much effort it is to get the licenses.
@@tybaltmercutio I'll be honest I'm not sure. The team I am with mostly use siesta, orca, ASE and Avogadro which work for our purposes and we have pre built scripts for most of our needs.
'Just a simple evolution of the wavefunction can transform the universe innumerable ways'
As DFT is the basis of the comp chem modeling research I’m doing in undergrad, this was amazing
Once again, very clear and clever explanations. And once again, one of the most comprehensive english speach for a non-native english-speacker. Thanks.
In an information theory context, would the “compression” ratio of the bits required to store DFT to Schrödinger data be similar to that of a black hole surface area to it’s volume?
The way „standard“ DFT is presented here is somewhat misleading. Actually, there is no compression at all since you still need to introduce Hartree-Fock like orbitals in an actual implementation. And the way electron correlation is treated within DFT is by either fitting some data to experiments or to a higher level wavefunction approximation since the exact exchange-correlation functional is not known and there is no way to systematically improve DFT (contrary to some more elaborate theories which do not share these weaknesses of DFT but can also be more complicated/expensive).
Excellent description! I have been reasonably successful at understanding QFT up until now. 78 dimensions! Per electron! That must be Many Worlds, and Im not sure I am in the same world I was before I watched Matt describe it.
💯
2022 and I am still trying to figure out what world I am in and how to get to something similar to a familiar face of a friend.
Until then I’m just gonna kick back and pray.
This timing was perfect. I am going to give a intro talk on fluorescence, and that DFT video you showed of a pi-pi* transition was perfect!
Great episode. Another check for the possible importance of error correcting codes and the sporadic simple groups.
It amazed me when I was using DFT years ago just how limited computing is when it comes to simulating larger molecules. It would dramatically change how we approach chemistry and biology if quantum computing can break that limit.
It involves more approximations, but there are now "linear-scaling" DFT methods, where the computational complexity approaches linear scaling as problem size increases.
DFT will not gain anything from quantum computers.
Quantum advantage fomes into play for coupled cluster, (F)CI and so on.
I was under impression that if all particles in the simulation (e.g., electrons) are indistinguishable, then the number of dimensions to describe this system will not grow exponentially. Instead, one could use a joint wave function of all particles defined on a 3D domain.
Indistinguishable particles means all particles are connected to all orbitals. It doesn‘t mean the particles don‘t interact e.g. via Coulomb repulsion.
@@tybaltmercutio could you please explain that further?
You omitted highly relevant information:
1. DFT is by far not the only electronic structure theory available. There is MR-CI, CC, MPx, EOM-CC, CAS-SCF just to name a few.
2. While stationary DFT is well established, conventional TD-DFT is still derived from the Frenkel-Dirac action principle which leads to a causality paradox. Corrections by Vignale or van Leeuwen are rarely invoked.
3. The success of DFT is mostly due to the performance of hybrid functionals such as B3LYP. Those include HF-exchange terms, i.e.explicit electron-electron interaction and are based on Kohn-Sham-DFT, which inherently requires the "reintroduction" of wavefunctions for the calculation of kinetic energy and scales similarly to Hartree-Fock theory. Wavefunction-free "pure" DFT has been shown to perform abysmally for inhomogeneous electron systems in general.
Excellent attention to detail to specify all the atom in the “observable universe”.
All the atoms in the “entire universe” could be used to simulate anything.
If we did use this to simulate a universe, would our analogs inside that universe only discover physics up to the limitations of our abstraction of it in the simulation?
I realize there's no way to know the actual processing power of the system that runs the simulation we're in, so it could be arbitrarily powerful and time within the simulation could run arbitrarily slowly. But at what point will we be able to determine that what we experience is itself an abstraction of a more complicated system? And is there a version of Skyrim that runs on it yet?
And if we run Crysis on enough computers in our universal simulation, can we crash the computer that is running our universe?
I think that very quickly, you would find that DFT's predictions would not scale up well to the entire universe. We would "discover" things that don't exist in reality because the model they use is too wrong.
@@xBlueSkittlesx "Some people just want to watch the world... um... lag out"
I've created a simulation of the universe everyday since I was born, I don't see why you guys have so much trouble doing it
No trouble, directed evolution of proteins works with real sim speed ;) Still not too fast...
Who needs Space Time when you can explore infinite Protein Space ;)
🤣😂🤣
I was just thinking about dice probabilities a week or so ago and now you give me this amazing analogy, to think of every dice as an additional dimension. Dope 😎💙
Thank you for talking about no longer needing a universe size computer to simulate the universe. I always assumed some compression algorithm could be used to reduce the size of computer needed (if I’m understanding this episode correctly). Also thanks for existing Space Time! Much love in 2022!
Unfortunately DFT is not THAT good, if you double the number of electrons, you need 8-16 times the processing power. You could not simulate the whole universe within the universe... Only a small chunk of it. But a fat far larger chunk than vanilla QM
Are these techniques efficient enough for small-scale, interactable, real-time simulation, like in a video game?
Probably not, but with modern AI techniques this would never need to be implemented. Since the tech can more easily just guess, and be roughly better than the computation heavy simulator can generate, it's not worth it.
Check out: 2 minute papers, YT channel.
No but actually yes. These techniques can be used to train very fast AI networks which are getting better all the time. Some calculations which used to take days can be reduced to seconds on a sufficiently complex and well trained AI system.
Depends on what level.
You can probably have a video game that plays in a single benzene molecule and simulate that.
But you wouldn't simulate human protagonists on that level.
yes, with a few more approximations: th-cam.com/video/EIt5hDSbRvg/w-d-xo.html
Can't help but wonder how advanced our technology will be 10,000 years from now or 100,000 years from now. You gotta wonder how advanced technology gets when your civilization has been around for say, 5 million years.
Don’t forget the great filter.
Climate Change and its effects will see to prevent that.
@@plexiglasscorn The Great Filter is likely behind us tbph, although a Global Nuke Secondary Great Filter could still exist. Be real, where is everyone? I doubt there's other life in our Galaxy, the Universe is another story. If we're in an empty Galaxy, or even Universe, our goal should be to spread among the stars and evolve, soreading life to every corner of this vast cornerless universe, lol. Make the Doctor Who you wanna see in the Universe, ya know? And we should act as guides, an elder species, and teach peace and love.
@@lucemiserlohn climate change isn't a world ending event. Will probably make things harder but we'll survive
Well, humanity can go extinct in 10,000 yrs from now. Or, at least our civilization can turn into a pile of rubble.
Fantastic! So informative! Absolutely amazing how clever people can be to figure out something like this.
Finally! I've been waiting for some condensed matter and solid state stuff from PBS Space Time for an eternity!
What kind of scale of reality could DFT simulate with the kinds of computers we're likely to see in the future (classical or quantum)?
A protein or a few proteins, maybe a small virus, depends on how far future.
i think they did a video on a jupiter brain
Well, bad news is, even for small molecules like Heme molecules, or DNA bases, DFT isn't that accurate. DFT's highest reach so far is a few hundred base pairs of DNA.
SIESTA (DFT program) can run thousands of atoms at once on a supercomputer 😅
@@Willr7theSquid Yeah, I mean some programs can keep scaling as long as you keep adding more thousands of cores.
If someone had a device with enough power to simulate the universe, he would use it for crypto mining instead.
Maybe it is more datapower efficient to simulate a universe and have the inhabitants mine crypto than acually mine it yourself. #showerthought #meaningoflife
20:08 It seems I'm not the first one with such an idea 😄
The ability to build a supercomputer, requires that you have access to significant funding in the first place, so it actually works the other way around.
"Need Proof of Work"
*Points to planet sized computer and Dyson Sphere power supply* "Is this not proof enough?"
I love that you use a Zenith/Heathkit 19 inch terminal/computer from the 70s (with updated color screen) as your graphic for modern computing! Seriously, I still have one of these (in green screen of course). :)
👋
IT's DFT, not NFT, thankfully.
Literally one of the few episodes that I have little understanding. Time to study up. Gives me something new to learn.
This unlocked a bit of understanding that "multidimensional" aspect of particles for me.
The fact a wave function only collapses when observed makes me feel we are living in a simulation. It's almost like the system is saving resources by not simulating things it doesn't need to unless it becomes necessary to save computing power.
To me it seems like it’s very wasteful though. All that empty space, and wasted material with no life. If resources were only spent when things were observed, and that was done to save resources, then the more things are observed, the more strain is put on the system, maybe it lags, which would be interesting, but we probably wouldn’t experience it, as a computer running steps slowly still runs the same steps and produces the same output unless the current time is referenced. To me, I don’t know what simulation would mean in this sense, because the physics of the higher universe would be so different. It probably wouldnt be on a computer or any kind
Out of all the anthropomorphic religions ever conceived, CS graduates metaphysically projecting themelves as the creators of the universe has to be the lamest.
As a kid, I always wondered if that was why particles don't have a specific location until you look for it... to save on processing by not rendering them until the universe needs to.
All it needs to know is that they're there, and they follow rules, and interact in certain ways, so a generalized effect is all it has to know most of the time. It doesn't need to calculate every atom in the moon... just know that there are so many, and they do this and that in large groups, so it should look like this, and there should be this much gravity.. and it just processes that, unless you force it to by looking closer.
Like your computer not rendering the part of the scene behind your camera in a game, until you turn around. It knows the polygon structures are there, if your avatar can pass through them or not, and what textures they have... but it isn't being rendered pixel by pixel.
Thank you so much for this episode! Definitely needed that dopamine boost 😊
Very interesting! I am interested in computational physics myself, so it's great that you spend a video on it!
Could the solar system be "bleeding" (for want of a better term) dark matter due to the positive pressure from the sun's radiative nature? That is, could the solar wind cause enough "gravitational drag" to drag away some dark matter with it?
The V_{eff} is the key to DFT. In grad school, I had to set up by hand an early version of a potential used to calculate the band structure of the various semiconductors. Once I started research, my advisor paid for real fancy software that does DFT and the sheer amount of potentials in some of these software is astounding.
Hey Space Time viewers, Vic Chaos here! I know Matt talked a lot about DFTs, but he didn't talk about my favorite thing, NFTs! Now, you've heard people say that NFTs are terrible. But those people are flat earthers. You don't want to be a flat earther do you? Of course you don't, which is why you should buy some NFTs! See this cute puppy elephant with a propeller hat? HOW CUTE! Now, people who don't think the cute puppy elephant is cute are people that believe the universe is only as old as last Thursday. You're not a last Thursday-ist are you? Of course you're not! You believe the universe is old and NFTs are the future! Buy this puppy elephant from me and you'll be on your way to financial freedom. See, I have this cat eagle here with a bubble gun. How fun! Buy it now, otherwise, you want the universe to die in a big crunch, and we all know the universe will die in the big freeze, so buy NFTs now!
I'm amazed to know that folks who work with such advanced equations are here with in the YT comments. I feel a smart person already
You are great!, I have watched you for a while, and I continuously watch older and older videos. The only thing I cannot do is the Patreon thing.
i swear i always start off soooo good with these videos and realize it’s only been 2 min, by 5 min hes saying more big words then my mind can input
I spent nearly a full minute contemplating what FTD could stand for before realizing you meant the other order those letters could be arranged in lol. Full Throttle Delusions was probably my favorite.
I habe done DFT for 4 years in my Organic Chemistry doctoral Studies struggling hard with the theoretical Foundations. This Video Made it super clear in ca 15 min. Thank you so much.
What about a coupled Cluster video next? 😅
I am a student in in DFT and DMFT and this video defo resolved a curiosity
The "Matryoshka Bitcoin" jokes at the end are practically a plot synopsis of _Accelerando_ by Charles Stross! But he treated it with a straight face, as a nightmare dystopian future.
had to watch this twice which means it was very good. topic requires a bit of background on particle physics, qft, and qed though. I think I see what Matt did here setting us up with the previous video on Action though. Have to give it up to PBS Spacetime and the clever scientists devising ways to condense, or compress, and consildate myriad computational data points from the micro into marco.
Awesome, I always wanted to implement it to better understand it myself. Thanks for the explanation!
You got me there for a moment. I was terrified thinking you try to explain double field theory.
Excellent video. Very interesting, informative and worthwhile video.
This show challenges my ADHD and I love it so much.
just thought id say this..... your awesome and you have educated me and inspired me to persue a career in science.
Great explanation of DFT, it brought me back to having to do a lit review on it many years ago. I know you touched on it in the video but it would also be interesting to see something on methods like quantum monte carlo. Maybe I'm just biased though because I use it a lot now in my PhD research.
I CANT HELP but recommend-around epic science-youtubers
like Sci Show, tom scott and of course Hbomberguy.
Always helpful and mind expanding! Thankyou!
More formula explanations please. You do it the right way.
Great video. I am a master student in chemistry and loved learning about how creative theoretical chemists and physicists get in approximating these absurdly unsolvable problems. Hartree-Fock next? :D
One of the first of your videos, where i instantly new what your talking about 😂 thanks to three years of studying chemistry
I can't wait till you do videos on HF and CC. This was a really nice summary on DFT.
Can't wait for Quantum Computers, which should give a significant boost in those calculations without cheating.
As a theoretical chemist this video makes me very happy