What is mechanistic interpretability? Neel Nanda explains.

Art by ‪@hamishdoodles‬
Clipped from episode 19 of AXRP: • 19 - Mechanistic Inter...
Transcript of that episode: axrp.net/episode/2023/02/04/e...
---
AXRP patreon: / axrpodcast
AXRP ko-fi: ko-fi.com/axrpodcast

[@antigonemerlin]

The images are quite helpful, especially for a complete beginner to the field when it comes to terms like stochastic descent. This channel is very underrated.

[@axrpodcast]

Thanks - nice to hear!

[@Words-.]

Thank you!

[@Words-.]

What if we have an AI that does this for us? And an ai that interprets the interpreter and so on. Maybe an ai wave process in order to give us a constant state of interpretation of what is going on.

[@reidelliot1972]

There are approaches that use this tactic for outer alignment. I highly recommend checking out the classics: Christiano IDA and debate, etc. It's definitely a common motif in this area of research. But then again, I've seen people raise concerns that automating interpretability tools may enable deceptively aligned policies/agents to further entrench themselves.
Check out "AGI-Automated Interpretability is Suicide" by RicG

[@user-vt4bz2vl6j]

thats great but how would you know its doing it correctly...

[@Words-.]

@@user-vt4bz2vl6j That is a fair question, idk. But at least its a step