Qwen-2.5 Coder 32B: The Ultimate Coding Assistant? Full Test Vs Sonnet 3.5

[@Codewello]

The Qwen 2.5 Coder 32B model is insane, and it’s open source! Go ahead and try it 😁
Free API:hyperbolic.xyz/
HF Playground :huggingface.co/playground?modelId=Qwen/Qwen2.5-Coder-32B-Instruct

[@shay5338]

01:03 awww absolutely

[@altyb_anony]

يا معلم 🥲🔥

[@Codewello]

منور

[@1brokkolibaum]

Its not working for me within cline. Such a headache..
//EDIT: It says "401 status code (no body)" using hyperbolic and using infra it says same just with 404 error. This is heavily annoying..

[@Codewello]

Upgrade Cline and make sure the API URL is correct, as I had the same issue

[@1brokkolibaum]

@Codewello I am already on v2.1.5 🥲 downgrading also didn't help

[@costatattooz840]

you should use hhao version that he released. it works for me in cline but since i have 12gb of vram and 32gb of ram it works sooooooo slowww

[@SejalDatta-l9u]

Im struggling to use Qwen 2.5 Coder 32B 8b quant version on a RTX 4080 with 12GB. It works be so slow and sometimes just gives up. Does anyone have any tips of ideas that have worked for them to speed up the process?

[@pensiveintrovert4318]

I have tested the Q8_0 version with Cline. Firstly, the most recent Ollama doesn't work with it, the runner dies. I had to downgrade to v0.3.11. Then Cline could interact with Qwen but Qwen does not seem to understand all the commands/prompts that Cline sends to it, so Cline complains that it has difficulty and loops over the same request.

[@SejalDatta-l9u]

@pensiveintrovert4318 100% exactly what happened with me. I'm sure others have been successful with it. Perhaps something is blocking us, or there may be a step we are missing (not yet revealed to us)

[@Codewello]

I think you need more RAM, around 24GB, to make it work better.

[@Codewello]

I had this issue with the Grok-Beta API; some of these models apis are not stable yet.

[@pensiveintrovert4318]

@@Codewello I have 49GB, over 4 GPUs. In any case, I got it working, more or less, used the template and system prompt from a discussion on Ollama GitHub. Qwen has a problem recognizing function definitions in some formats, how exports are done in Express.

[@SoloJetMan]

I've been pronouncing Cluade incorrectly all this time

[@Codewello]

The name Claude sounds like a common girl's name in Germany and Austria.

[@DanIel-fl1vc]

It takes so much Ram though, you need at least 16GB ram for full usage.
5GB model for 8 GB ram. Great for debugging.

[@Codewello]

24 GB of RAM is required to make it work, as far as I know.

[@Lapatate-s1l]

You can’t even compare that to Gemini flash . But it’s good for its size and price ( free)

[@Codewello]

Bro, I'm really confused. Why is everyone trashing this model so hard? 🤣Did I miss something? 🤔

[@Lapatate-s1l]

@ I didn’t say trash . What I want to say . Is that is a good Open source model . And one of the best for coding if not the best at 32b size . But we are very far from sonnet or gpt4 . I’m using it when offline . It does the job but it’s really not as good as sonnet . You should try it for a real world project

[@0xb1sh0p8]

This model is absolute trash though.

[@AdminOne-hj4dt]

😂

[@Codewello]

What are you talking about?

[@0xb1sh0p8]

@@Codewello I'm talking about how horrible it is at real world coding. I understand these were tests but nobody is building todo lists and quiz apps. I tested it against a React/Express/Prisma app and every response was a total hallucination.

[@r72762]

@@0xb1sh0p8I agree bro. In your opinion, what do you think is best model for coding at the moment?

[@TheReferrer72]

@@r72762 Claude Sonnet

[@brto]

🗑

[@Codewello]

🔴

[@o_glethorpe]

Just use cursor, so far everything else sucks

[@A-Jaradat-Z]

use Windsurf, the best so far or aider for vscode

[@Codewello]

Not really, but Cursor is amazing, not going to lie.

[@Codewello]

I'm going to make a video about this.