Thank you, I read a lot of papers working full time on GenAI projects, but missed the ground changing paper you presented. A comment on the economics of GenAI, it is clear to me that new models like Jamba, with linear complexity, with their equivalent performance to Transformer based LLMs, with quadratic complexity, will come to the fore. I have experience using Ray Tune, so that will be my optimizer :)
Regarding the comparison of complex agents and retry. Did the agents provide 1 answer or a choice from the top 10? It is incorrect to compare the top 1 with the top 10. I would like to see a comparison of the top 1. After all, in practical tasks, I most often need one specific correct answer, not a bunch of answers among which there is a correct one. Also, the agent explains its actions. They are divided into stages. It's easier to find errors in its reasoning. All else being equal, this can be an extremely important criterion for solving the task.
Some commercial agents can be black boxes. And it is not uncommon, that agents perform internal majority voting to present the "correct" answer to you, an answer with the highest probability score. As with the example of SWE, I can't follow several hundred of thousand tokens for a $4 run.
@@code4AI It seems that the agent-based approach does not improve the reasoning capabilities of networks, BUT: It allows for the decomposition of reasoning into stages, the correctness of which can be verified by instrumental means (checking the validity of the logical construction, code compilation, passing tests, etc.). It allows for an increase in the length of the correct reasoning chain, i.e., to improve the perplexity of the response in a long context. For example, to write a coherent, logically, and stylistically correct book. And the complexity of real tasks lies precisely in their multi-stage nature. This involves a long context of reasoning and actions, the correctness of which needs to be maintained. Are agent systems evaluated by the right benchmarks? However, I do have questions about the feasibility of agent systems. Won't they be eventually overtaken by LLMs that can maintain a very long context and independently generate requests for various actions? Are there any fundamental reasons to consider the agent-based approach as something unique and irreplaceable in the near future?
StoneAge, IronAge, ModernAge, AgenticAge❤
Thank you, I read a lot of papers working full time on GenAI projects, but missed the ground changing paper you presented. A comment on the economics of GenAI, it is clear to me that new models like Jamba, with linear complexity, with their equivalent performance to Transformer based LLMs, with quadratic complexity, will come to the fore. I have experience using Ray Tune, so that will be my optimizer :)
Regarding the comparison of complex agents and retry.
Did the agents provide 1 answer or a choice from the top 10?
It is incorrect to compare the top 1 with the top 10.
I would like to see a comparison of the top 1. After all, in practical tasks, I most often need one specific correct answer, not a bunch of answers among which there is a correct one.
Also, the agent explains its actions. They are divided into stages. It's easier to find errors in its reasoning. All else being equal, this can be an extremely important criterion for solving the task.
Some commercial agents can be black boxes. And it is not uncommon, that agents perform internal majority voting to present the "correct" answer to you, an answer with the highest probability score. As with the example of SWE, I can't follow several hundred of thousand tokens for a $4 run.
@@code4AI It seems that the agent-based approach does not improve the reasoning capabilities of networks, BUT:
It allows for the decomposition of reasoning into stages, the correctness of which can be verified by instrumental means (checking the validity of the logical construction, code compilation, passing tests, etc.).
It allows for an increase in the length of the correct reasoning chain, i.e., to improve the perplexity of the response in a long context. For example, to write a coherent, logically, and stylistically correct book.
And the complexity of real tasks lies precisely in their multi-stage nature. This involves a long context of reasoning and actions, the correctness of which needs to be maintained. Are agent systems evaluated by the right benchmarks?
However, I do have questions about the feasibility of agent systems. Won't they be eventually overtaken by LLMs that can maintain a very long context and independently generate requests for various actions?
Are there any fundamental reasons to consider the agent-based approach as something unique and irreplaceable in the near future?
Thanks
very insightful! thanks
Thank you.
Can I do optimization on langgraph agents?
You can run an optimization on almost any system ...
Get out of my bubble
See you.