Qwen QwQ-32B Tested LOCALLY: An Open Source Model that THINKS

[@userou-ig1ze]

Commenting to feed the algorithm

[@Bijanbowen]

Hahah I appreciate it :)

[@antisony2008]

AI nerd here. dude so many AI created videos about AI these days, seeing a person check it out is so much better.

[@Bijanbowen]

Thanks very much! I feel the same way, I am not a fan of the stock footage voiceover style videos haha.

[@haileycollet4147]

What is the chat template? I try ChatML (in Kobold) and get total gibberish. Want to use Kobold so I can use a draft model (speculative decoding)

[@Bijanbowen]

I have never used Kobold, and only used this with ollama so far. Take a look at line 197 in this script as it shows the chat template for qwq from its huggingface page: huggingface.co/Qwen/QwQ-32B-Preview/blob/main/tokenizer_config.json Not sure if that will be of any use but I suppose it could help in figuring out a working kobold template

[@iseverynametakenwtf1]

I've been using it and trying to figure out what for.

[@Bijanbowen]

That's always the issue haha, I would like to use it to see if it does a better job at TinyTroupe but it is not on Ollama and I don't want to have to connect it to Webui lol. Edit: NVM It is on Ollama.... whoops hahaha

[@jim666]

would be interesting to test more jailbreaks from the "jailbreak community"

[@Bijanbowen]

I have been monitoring the localllama sub to see what new ones get posted there hahah, unfortunately really testing them on the channel would risk getting in trouble hehehe

[@jim666]

@@Bijanbowen woowww I guess you're right!

[@1conscience0dimension]

Seriously increase font size

[@Bijanbowen]

Great point, will do that!