This is a concept I'd been considering myself, but I never thought of it as autodifferentiated text. Fantastic that research is being done in this direction. I knew it'd be a good idea.
I criticized this has to be done manually, but never thought of chaining 2 LLMs to achieve it. Though, it does make getting slightly better answers 3x more expensive. I guess it's useful for unsupervised learning though.
Here's an unpopular opinion: could this be considered a misuse of the notation for auto-differentiation and backpropagation? For any graph to be differentiable, it must be acyclic-like a Directed Acyclic Graph (DAG), which is typical for neural networks. However, in the LLM sphere, we see pipelines incorporating cycles, such as the RAG where blocks are repeatedly cycled through, forming what might be described as Directed Cyclic Graphs (DCGs). While using PyTorch's clean and modular syntax is appealing, applying auto-differentiation in this context could be seen as a stretch (personal opinion).
Great video, very informative. Textual Gradient is such a pretentious concept for me, but I do look forward to try TextGrad out. At least it is a systematic method to perform prompt optimization.....
Thanks for the video, it’s very insightful! I have 1 thought: 1. Textgrad and DSPy can be combined. As DSPy is mostly based on ICL and this framework focuses more on signature optimization. Additionally, the researchers in Stanford mentioned that the combined prompt on one occasion improved the prompt by 1% and it should be further studied.
You said that you used in on your tasks. Can you release part of that code in the wild? It would be really great to see a live example. That was the thing I found very challenging with DSPy. Only with the storm project I started understanding how it should work ;-)
Start with the four Jupyter Notebooks that I provided and you will see that you have immediately multiple new ideas for your specific tasks. I plan a new video on my insights, given my testing and maybe I have an idea how to optimize the TextGrad method further ....
Amazing video! But pseudo as in pseudo-code is pronounced like sudo (syuudo) Not smart enough to correct anything else in this video lmao, keep up the good work! Love the channel
This is a concept I'd been considering myself, but I never thought of it as autodifferentiated text. Fantastic that research is being done in this direction. I knew it'd be a good idea.
I criticized this has to be done manually, but never thought of chaining 2 LLMs to achieve it. Though, it does make getting slightly better answers 3x more expensive.
I guess it's useful for unsupervised learning though.
Thanks stanford, though I would have called it backpromptigation. ;)
Solid. I knew if the guy behind DSPy could build that, there was a better version imminent
Here's an unpopular opinion: could this be considered a misuse of the notation for auto-differentiation and backpropagation? For any graph to be differentiable, it must be acyclic-like a Directed Acyclic Graph (DAG), which is typical for neural networks. However, in the LLM sphere, we see pipelines incorporating cycles, such as the RAG where blocks are repeatedly cycled through, forming what might be described as Directed Cyclic Graphs (DCGs). While using PyTorch's clean and modular syntax is appealing, applying auto-differentiation in this context could be seen as a stretch (personal opinion).
Great video, very informative. Textual Gradient is such a pretentious concept for me, but I do look forward to try TextGrad out. At least it is a systematic method to perform prompt optimization.....
sounds like we need a middleware complexity assesor that can sit in the middle and auto reject if it doesnt meet that balance
Thanks for the video, it’s very insightful!
I have 1 thought:
1. Textgrad and DSPy can be combined. As DSPy is mostly based on ICL and this framework focuses more on signature optimization. Additionally, the researchers in Stanford mentioned that the combined prompt on one occasion improved the prompt by 1% and it should be further studied.
DSPy is ICL and prompt optimisation combined. I hope they add text grad in somehow though
@@matty-oz6yd yea, good correction. I hope they add Textgrad in as an optimizer.
Their mipro v2 optmizes literally doing the same
Thanks for the video. I missed the boat with DSPy but it's good to know you can just go ahead with TextGrad.
You said that you used in on your tasks. Can you release part of that code in the wild? It would be really great to see a live example. That was the thing I found very challenging with DSPy. Only with the storm project I started understanding how it should work ;-)
Start with the four Jupyter Notebooks that I provided and you will see that you have immediately multiple new ideas for your specific tasks. I plan a new video on my insights, given my testing and maybe I have an idea how to optimize the TextGrad method further ....
Thanks for another great video! I like your presentation style. What kind of software do you use for your slides?
26:51 what does 0 demonstrations mean? No examples of good output, only original prompt?
great video, hope for you more advanced explain and experience on TextGrad!
Seems like one can prompt optimize for the same level system and never lack coherence.
Superb explanation! Thank you!
Very informative.
Thanks
Thanks for the links to colabs…
Great! Thanks
How is this different from prompt tuning (not engineering)?
Explained in the video.
Amazing video!
But pseudo as in pseudo-code is pronounced like sudo (syuudo)
Not smart enough to correct anything else in this video lmao, keep up the good work! Love the channel