How to Fine-Tune and Deploy an Embedding Model

This video shows you how to fine-tune embedding models with Marqtune and deploy them using Marqo Cloud for real-world applications. It covers getting set up, preparing a dataset, fine-tuning a model, and deploying it to create a searchable index ready for production.
Main Article: www.marqo.ai/b...
Google Colab: colab.research...
Article (including score metric & evaluations): www.marqo.ai/b...
Generalised Contrastive Learning (GCL): www.marqo.ai/b...
Community: marqo-communit...
💡 More about this video 💡
Fine-tuning and deploying embedding models tailored to specific use cases has often been a challenge-until now. Marqo Cloud provides an end-to-end solution, allowing you to train, embed, retrieve, and evaluate all in one platform.
So, how does it work? Start by selecting your embedding model-Marqo offers hundreds of open-source options, or you can bring your own. Next, fine-tune and evaluate this model using Marqtune, Marqo’s fine-tuning embedding model platform. We built Marqtune on the foundation of our new training framework, Generalized Contrastive Learning (GCL). With GCL, you can fine-tune embedding models to rank search results not only by semantic relevance but also by a ranking system defined by your search team. This means better, more relevant search results that cater to your business.
Once your model is trained, deploy it seamlessly on Marqo Cloud. From there, create an index, add your documents, and voilà-you’re set to deliver powerful search and recommendations. Whether you’re building a fashion search tool, an ecommerce platform, or any multimodal application, Marqo’s tools get you up and running in no time.
In this video, we’ll walk through the entire process. Happy coding!
🔗 Follow us and stay updated 🔗
Website: www.marqo.ai/?...
LinkedIn: / marqo-ai
X: x.com/marqo_ai
Slack: marqo-communit...
Hugging Face: huggingface.co...
GitHub: github.com/mar...
Instagram: / marqo_ai
#genai #generativeai #llm #llms #embedding #vectorsearch #vectordatabase
Subscribers: 190

[@marqo-ai]

🔗Links 🔗
Article: www.marqo.ai/blog/how-to-fine-tune-and-deploy-an-embedding-model?
Google Colab: colab.research.google.com/drive/1r5Q3R9NbHLh2tWvNt9Xv4hHMPgnGEVPE?usp=sharing
Article (including score metric & evaluations): www.marqo.ai/blog/getting-started-with-marqtune?
Generalised Contrastive Learning (GCL): www.marqo.ai/blog/generalized-contrastive-learning-for-multi-modal-retrieval-and-ranking?
Community: marqo-community.slack.com/join/shared_invite/zt-2ry33y71j-H0WUeQvFaVlKuuZwl38BeA#/shared-invite/email
🔗 Follow us and stay updated 🔗
Website: www.marqo.ai/?
LinkedIn: www.linkedin.com/company/marqo-ai/
X: x.com/marqo_ai
Slack: marqo-community.slack.com/join/shared_invite/zt-2ry33y71j-H0WUeQvFaVlKuuZwl38BeA#/shared-invite/email
Hugging Face: huggingface.co/Marqo
GitHub: github.com/marqo-ai/marqo
Instagram: instagram.com/marqo_ai/

[@GabrielAlon-y6e]

Great level of detail and clarity

[@IshaaqChandy-f2i]

Thanks, Ellie! Great to see how simple it is to fine-tune and embed a model into Marqo Cloud.

[@martaesteban613]

Such a great video!

[@EllieSleightholm]

🔥🔥🔥

[@al.sargent]

Thanks great video!

[@nitinrai6093]

Great for production, but for developement I would still prefer an on-premise vector db. Can I download the finetuned model from marqo cloud for offline inference?

[@marqo-ai]

You absolutely can under our enterprise plan. If you'd like to learn more, reach out to me at [e-mail] and I can introduce you to the team for a chat.