Hi James! I have a question about the Pinecone tool. I already have my database there and based on that, I want to create an advanced chat bot that will probably be used by max 5 people at once. Will the free option be sufficient? Should I be interested in paid plans? And if we are talking about paid plans, does anyone understand their price list? I don't really :(
I have a question - when it purely sparse search, What is the matching algorithm between query (list of token-ids:frequency) and the document (list of token-ids:frequency) ? Secondly, In case of hybrid, are the scores normalised and combined for every document ?
Hello James great video thank you a lot, I have a question, do you know if it’s also possible for milvus database ? And what kind of mlops we can use to do this for more real world use cases ? Thank you
I don't believe so, the page in their docs on "hybrid search" is actually a vector search with metadata filters, so not actually a hybrid search lol. I'd image they're probably working on it (or thinking of working on it) but it's still early days for hybrid search so I don't when to expect it. For vector embedding there is Hugging Face Inference endpoints, OpenAI's GPT-3 Embeddings, Cohere embeddings, etc for the dense vectors For the sparse vectors - I'm not actually sure what service would make sense, possibly Hugging Face inference endpoints with the SPLADE models, but I haven't gotten around to testing those yet so not 100% sure
Yes that is how I’ve understood, I’ll cover in more detail soon and explain better - but for example you can use a sparse embedding model like SPLADE on the sparse side (not only BM25)
![กิโลเมตรที่ 0 เดินทางรอบโลกไม่ใช้เครื่องบิน [Around The World Ep. 1]](http://i.ytimg.com/vi/toJG9oofLFY/mqdefault.jpg)
Incredible. This is exactly what i was looking for!
Hi James!
I have a question about the Pinecone tool. I already have my database there and based on that, I want to create an advanced chat bot that will probably be used by max 5 people at once.
Will the free option be sufficient?
Should I be interested in paid plans? And if we are talking about paid plans, does anyone understand their price list? I don't really :(
I have a question - when it purely sparse search, What is the matching algorithm between query (list of token-ids:frequency) and the document (list of token-ids:frequency) ? Secondly, In case of hybrid, are the scores normalised and combined for every document ?
Hello James great video thank you a lot, I have a question, do you know if it’s also possible for milvus database ? And what kind of mlops we can use to do this for more real world use cases ? Thank you
I don't believe so, the page in their docs on "hybrid search" is actually a vector search with metadata filters, so not actually a hybrid search lol. I'd image they're probably working on it (or thinking of working on it) but it's still early days for hybrid search so I don't when to expect it.
For vector embedding there is Hugging Face Inference endpoints, OpenAI's GPT-3 Embeddings, Cohere embeddings, etc for the dense vectors
For the sparse vectors - I'm not actually sure what service would make sense, possibly Hugging Face inference endpoints with the SPLADE models, but I haven't gotten around to testing those yet so not 100% sure
Does it mean that this is not a hybrid between BM25 and Vector search but it is hybrid between sparse and dense vector embedding?
Yes that is how I’ve understood, I’ll cover in more detail soon and explain better - but for example you can use a sparse embedding model like SPLADE on the sparse side (not only BM25)
Thanks James!
🙏
Is hybrid available now ?
as of ~5 minutes ago yes, see here www.pinecone.io/learn/sparse-dense/
So what you are saying is that this is just black box
Nice video. But don't know who gave you the idea of adding in those blips. Annoying asf
those noises make me want to jump out the window
Lol which noises I’ll tone them down?
@@jamesbriggs bro the constant blips! It sounds way nastier and disruptive when on 2x.
Yikes yeah it’s so bad I bailed after 2 minutes. Cannot expose myself to that kind of damage. Mental health comes first.