"You need to hold on tight to your optimism. Because if you don't, You're never going to figure out how to do the hard things to answer the hard questions and make it really work"
The example at 12:14 , I’m curious if you could have used more descriptive language like “Only change the bed to be green while maintaining tan walls”? To be clear though, I agree it is a problem for a generator like this when you do not understand how it works and are trying to control it this way. Good work!
Congratulations Dr. Bau! It is a great and systematic research project and a very important one with many practical implications. I am sure it could be a great step towards a transparent general AI if you start combining and interconnecting such separate networks together at the interaction levels. I am fascinated! Thank you very much! It really inspiring!
Thanks for sharing, it was a great watch! Can you please comment on: 1. How do you search for "concept neurons" in giant models efficiently? 2. You mentioned in another comment that your group is working on interpreting GPT-X models, can you briefly comment on the concepts your group is trying to find out in GPT-X based models?
"PhD Defense at MIT" and for most people, he is "the amazing maze guy". Due to pandemics and KDP explosion, his mazes are now in demand. A lot of people is trying to figure out how to make solutions script.
Check out this link - web.mit.edu/PostScript/obfuscated-1993/labyrinth.ps - it is my submission to the obfuscated postscript contest web.mit.edu/PostScript/obfuscated-1993/WINNERS and it generates random mazes together with a solution, with all the computation done on the postscript printer. Edit the postscript as a textfile to change the options.
I am at 27 minutes mark and I am impatient to ask this question with out completing this video that does this method scale up to models which have millions and billions of parameters
Yes. We have found that the largest models, trained for a long time on massive data sets, tend to have very rich interpretable structure. Developing interpretable methods for massively parameterized models such as GPT-X is the topic of ongoing work in my group, and we have found that large models are a very target-rich environment. Oddly enough, one of the more difficult problems is to clarify is how interpretable structure emerges in the very simplest toy settings, trained on small problems, where the emergent structure is less obvious.
By the way, if you or somebody you know is considering a PhD, I am looking for students! (For Fall 2022.) Check out our papers davidbau.com/research/, apply to the Khoury school www.khoury.northeastern.edu/apply/phd-apply/, and drop me a note if you are interested.
"You need to hold on tight to your optimism. Because if you don't,
You're never going to figure out how to do the hard things to answer the
hard questions and make it really work"
This presentation has blown my mind. Thank you for making this publicly available!
We need more videos from you! If you have lectures available and are comfortable sharing them please do so.
Wow, Dr. Bau is really pedagogical and is pleasant to listen to.
The example at 12:14 , I’m curious if you could have used more descriptive language like “Only change the bed to be green while maintaining tan walls”? To be clear though, I agree it is a problem for a generator like this when you do not understand how it works and are trying to control it this way. Good work!
Congratulations Dr. Bau! It is a great and systematic research project and a very important one with many practical implications. I am sure it could be a great step towards a transparent general AI if you start combining and interconnecting such separate networks together at the interaction levels. I am fascinated! Thank you very much! It really inspiring!
Great watch. Thanks for sharing this, David. - Ervin
I can't join the chat without creating a channel, which I don't have time to do - but I wanted to drop in and congratulate you, Dr Bau!!
Thank you Kate!!!
Thanks for sharing, it was a great watch!
Can you please comment on:
1. How do you search for "concept neurons" in giant models efficiently?
2. You mentioned in another comment that your group is working on interpreting GPT-X models, can you briefly comment on the concepts your group is trying to find out in GPT-X based models?
"PhD Defense at MIT" and for most people, he is "the amazing maze guy". Due to pandemics and KDP explosion, his mazes are now in demand. A lot of people is trying to figure out how to make solutions script.
Check out this link - web.mit.edu/PostScript/obfuscated-1993/labyrinth.ps - it is my submission to the obfuscated postscript contest web.mit.edu/PostScript/obfuscated-1993/WINNERS and it generates random mazes together with a solution, with all the computation done on the postscript printer. Edit the postscript as a textfile to change the options.
Dear prof Dr sir
Thank you very 🙏 much 🙏
Great job! I really appreciate what you have achieved.
I am at 27 minutes mark and I am impatient to ask this question with out completing this video that does this method scale up to models which have millions and billions of parameters
Yes. We have found that the largest models, trained for a long time on massive data sets, tend to have very rich interpretable structure. Developing interpretable methods for massively parameterized models such as GPT-X is the topic of ongoing work in my group, and we have found that large models are a very target-rich environment. Oddly enough, one of the more difficult problems is to clarify is how interpretable structure emerges in the very simplest toy settings, trained on small problems, where the emergent structure is less obvious.
Congrats! This was really interesting!
Really cool results !
Congratulations!
Thank you!
the video has been greatly intriguing, and really enjoyed watching it. thanks for sharing 58:16
By the way, if you or somebody you know is considering a PhD, I am looking for students! (For Fall 2022.) Check out our papers davidbau.com/research/, apply to the Khoury school www.khoury.northeastern.edu/apply/phd-apply/, and drop me a note if you are interested.
Dr. Bau , what is the best way to reach you ?
Nice work. Bushy eyebrow kids isn't something I thought I would hear today 😅