วีดีโอ

That was a very clear introduction. Thanks!!

224 Juliet Shores

th-cam.com/video/3crJNnAXd6w/w-d-xo.html

Most clear definition of quantization, thx!!!!

I did the entire process but it's saying phi-2 not found

Amazing content. Thank you so much. 1) Do I have to use float16 on exl2 model, is there a way to use int4? For example Dracones/Midnight-Miqu-70B-v1.0_exl2_2.24bpw can be loaded on a 24 GB VRAM card on RunPod running OobaBooga with 4Bit cache enabled. I'm surprised why vLLM needs so much memory. 2) How would you secure the API with a key?

Are this limited? like you need to buy credits? or free without words or character limit?

thanks for the video

Thank you so so much !! lifesaver. Exactly what I was looking for !! just one question, in vllm, we can use batch to maximize the inference speed, just wonder if we set it up as an endpoint, can we still use batch ? can you give me a little bit hint ? and again thank you for your video !!

how to use as API

getting same response everytime

Leaked miqu before GTA6 💀💀

Looks quite expensive to run though

Good content, keep it up!

It is indeed very easy to set up. I am also surprised that getting a H100 pod is also very fast. (despite that downloading image and model might not be that fast)

Leaked Miqu in 5 minutes, that's so easy.

how is different from Ollama?

Super useful tool! It’s great to be able to play with closed source and open source models side by side.

Thank you for the Video. Could you please help with the hardware requirements(CPU, GPU etc) to run Phi-2.

Thank you for making this tutorial! It is very apprecaited. Can you recommend a good discord server to ask questions on topics related to this? Also, do you know if there is a way to send prompts via API? So I can make a PySimpleGUI UI for this?

Loved the clarity you bring

😇 'promo sm'

Great tool

Him: "She believes everything I say" Her in her mind: "He believes everything I say" 😂

The truth is that he shared his opinion about the current big military conflict somewhere in the world and he took the wrong side 😀

I think India is the reason

i want to work with you. i fell like i will learn a lot.

In the digital realm where startups aspire, Airtrain AI emerges, a narrative to inquire. Did OpenAI's move send ripples and shakes? A poetic exploration of the startup landscape's stakes. In the algorithmic dance, where innovations bloom, Airtrain AI faces a potential gloom. Did OpenAI's steps alter the startup game, A poetic question, where consequences may claim? Thousands of dreams in the startup cascade, Did OpenAI's decision cast a sudden shade? In the digital landscape, where possibilities gleam, A poetic pondering, in the startup dream. Airtrain AI, in the tech's ballet, Did OpenAI alter the startup play? In the entrepreneurial sphere, where visions align, A poetic inquiry, where destinies entwine. So let the questions echo, in the digital rhyme, Did OpenAI's move alter the startup climb? In the startup saga, where dreams partake, A poetic exploration, in the startup's wake.

A bit ironic for these startup people to be worried when these AIs are already replacing various jobs at a high pace. I think it's pretty sad to watch all of this unfold, as most of the people who act excited about these technologies are the ones putting themselves in a position where they will try not to be affected. Not sure what to think about this revolution and the speed at which it is happening. Also am wondering what the finality/logic is, where is all that supposed to lead to ?

Great analysis, thanks

Everyone should have anticipated this outcome.

nice video very interesting

how can one use the downloaded model with langchain?

Thanks for the video.