I am blown away at how much information is densely packed into this. You got yourself a new subscriber, sir. It’s staggering to think about how these technologies will shape the landscape for data and analytics. This is just the beginning.
Damn, I spent like a week researching this shit on my own, and have been working on almost exactly the same thing. Processing MDX files into embeddings etc. It’s really cool to see somebody doing almost the same exact thing. Makes me think I am really on the right track!
@@automioai In this project no PDF files were used - all documentation had been written directly in MDX. You'll have to do some research on ways to extract text from PDF files. Once you have that, I wouldn't bother with MDX at all - just generate embeddings directly on that content.
Prediction: This is gonna get a million views. Just saw fireship video about vector databases and wanted to understand embeddings. Before I could even search, this video was in the page. Though I wasn’t interested in a 40 min video (had a feeling I’ll just stop after 5 mins like I usually do) I ended up watching it all. The rabbit hole 🐇 🕳️ format is so naturally elegant. Clear end to end use case. I secretly don’t want to share it with anyone but I am forced to fulfill my prediction.
The clarity of this video while maintaining detailed granularity of the subject is very impressive and very appreciated. Thank you for making this video.
Ive been looking for this information for months. Such an excellent tutorial and I love that Supabase's code is all open source so i can actually clone it and read how it works in detail later. Thank you so much for the walk through. Super talented dude too - love the blender stuff.
@@RabbitHoleSyndrome why is the generate embeddings file so different in the video then what is in the repo now? I can't find anything you talk about in minutes 10-13.
Hey @@fraternitas5117! Supabase moves pretty quick - the code I references has been refactored now to support multiple knowledge sources (ie. more than just markdown). You can find the markdown specific code here: github.com/supabase/supabase/blob/1b2361c099c2573afa1fe59d3187343bb8f1bcab/apps/docs/scripts/search/sources/markdown.ts
Found your channel while learning React Three Fiber, subbed with notifications immediately. Today I get a notification for a well-explained ChatGPT tutorial, right as I embark on building a similar thing. Fantastic continued work, thank you very much!
Wow. I'm so thrilled to know that you were one of the ones behind that great feature. I've been using Supabase for 6 months, and have been pretty happy with it. Except for the docs and the transition to 2.0. I was blown away when I saw that it generated the code for me when I started writing its documentation
@@RabbitHoleSyndrome So far so good! I think one challenge is to know how we can check if a user's email address exists. (Or other specific user's metadata) I couldn't find it in the docs. There was a GitHub issue which said to store the user's data in a separate table as the auth table was private. I ended up doing that and haven't had any problems. Btw, thanks again for all the awesomeness!
What a fantastic video and content. I’ve gone through multiple videos trying to better understand embedding and how to work with ChatGPT in the best way for querying large amount of content and producing an analyzed response. I’m not a developer, have a background in computer science but I’m a software sales person that is curious about technology and I was able to completely understand your video and content. Subscribed, liked and will be watching more of your videos. Thank you!
Fireship guy? Either way, its been pretty useful in terms of learning API's and how to connect them to my nocode builder. Spent hours trying to get things working and the Assistant basically told me what i was doing wrong and how to fix it. So well done with the implementation.
Dude, that's exactly what I was looking for. No more bullshit articles with clickbait titles, just DIY in the essence. Is there a way to support you through patreon or smth. ?
Insane, I love how you are able to do so many things. My laptop atm is unable to power every wish of mine (getting into 3D) but I hope I will soon be able to do so.
The most amazing think is that I basically made a chatbot app in less than a week with only the help of GPT4, I had no knowledge of AWS services, PostgreSQL or python. Everthing you told in the video is what GPT4 told me. All of the serves and database are setup, it has memory, STT, TTS and Cognito login/register.
This is a glorious illustration. Thank you very much! I've been trying to find an example of doing this, and yours has put it all together for me! Subscribed!
First time here. This is so well done. Subscribed. Your viewer number will explode! I like how you approached the topic in a very calm way without jumping on the "LLMs will take over the world" train :) You don't happen to have the clippy blender asset somewhere?
I have not seen such a great video in a while! How wonderful have you explained the whole process 👍👍! Could you explain a bit more about how did you choose 0.78 as threshold for embeddings comparision? have you statisticized that wether the most relevant sections can be found with it?
Glad you liked it! 0.78 was a first-stab threshold that worked best based on a limited sample of test queries. I wouldn’t claim that this number is universal - almost certainly this could change by domain.
28:37 Whenever I see examples of decoder (GPT) prompts starting with "You are a helpful finance advisor" or "You are an enthusiastic support rep", I can almost see the AI clearing its throat and sitting up straight and saying "right, ok". Gimme that can-do attitude, GPT!
This is a fantastic video! Thank you very much for sharing :D Quick question - Currently if the info is not in the documentation it responds "Sorry I don't know how to help with that". But how can we make it respond like this: "Sorry I don't have relevant info in the documentation but you can do something like this". For e.g. "I don't have any info about how to make banana pancakes in the documentation, but here is how you can make one...." Idea here is to make it act like chatgpt on top of the information provided. Keen to know more on this and thank you so much for making this video :D
ปีที่แล้ว +1
Great content. I wonder if it can generate code as well. Lets say you are doing this for cloudflare products, kv, workers, durable objects etc all documents injected. Then give prompt like "generate a worker for x" and it will specifically generate with given docs etc..
One thing that might help is if the question result shows the links to the documents that it acquired the information from. Since you are currently fetching which document to run chatgpt on based on similarity of features, maybe you can change the prompt so that it also returns the link of the document that was deemed as a "similar document"
Amazing video! This is exactly what I was looking for a long time. You basically explains everything I wanted to know about how to create a search engine using open ai. But I have a few questions: How much did you spend on open ai embending API building this? How much supabase spends monthly with searchs using the open ai api? It is possible to use an open source embedding API instead of calling the open ai api ? Wouldn't it be less expensive than the approach you took?
Glad it was useful! As for costs, you may be surprised how inexpensive OpenAI embeddings are (at least I was). To put it in perspective, for the Supabase guides we currently have around 1500 page sections which total just over 220000 tokens. At OpenAI's current embedding price ($0.0004/1k tokens), that brought us to just less than $0.10 for the entire guide knowledge base (~one-time pre-processing). After that the average query is likely
This is amazing. Super clear and short explanation of embeddings. Great walkthrough the code touching on the relevant parts. Subbed the channel👍 I see the code is still in the Supabase repo - but the Clippy is not there? what happened?
Glad it was helpful! Edit: Just realized you might have been talking about the Clippy graphic on the site, not the code. Search and Clippy have been combined into the same interface - you can find Clippy by clicking search, then switch to “Ask Clippy”. Original: Things move quick at Supabase 😆 The Clippy frontend code got moved when search was upgraded to also use embeddings. After the refactor everything is just under “Search”: github.com/supabase/supabase/blob/0ecc238ad6d81202bb2301f7919b166a98929697/apps/docs/components/Search/SearchModal.tsx The backend Clippy logic is still in the same edge function: github.com/supabase/supabase/blob/0ecc238ad6d81202bb2301f7919b166a98929697/supabase/functions/clippy-search/index.ts
@@RabbitHoleSyndrome cool. You can feedback to Supabase, that a) this video significantly increased my interest in adoption of Supabase b) clipy icon is tooo small , it should be large and obstructive. Serious question Prompt Engineering: With Chat Completion API - do you do system persona and pack all prompt into user? Or would you break prompt like here into example 3 user entries? E.g.: Context, question, use markdown. Does it make a difference?
Good feedback 🙌 and great question about prompt engineering in the chat API. We are actually experimenting with this right now and trying to understand what produces the best results. At the moment we do a bit of both (system message and user message with a bit of prompt overlap between both). OpenAI says that the model doesn’t pay strong attention to the system message, so it may be better to use system message strictly to set identity and provide instructions & context in a user message.
Loved it, came for the embeddings and left with that and whole lot more! Too bad that the clippy did not made to the web on supabase :). Copyright problems?
Thanks for watching! Clippy did make it - but things move fast at Supabase 😅. This feature has evolved into a unified cmd+k menu and just renamed as “Supabase AI”.
Good stuff! Curious how you are handling performance of the respons? We setup a similar pattern and have found that GPT 3.5 obviously returns faster, but GPT-4 returns with much better quality. Have you experienced similar results? In this context does the speed of 3.5 outweigh the GPT4 quality - for us we are seeing some GPT-4 responses take >30 seconds, which is pretty terrible from a UX perspective. Also, curious if you are still using Algolia for the other "basic" search experience? We were thinking about using Algolia for search, and then sending the top few most relevant results to OpenAI for summarization. Not sure if the quality of Algolia's search results (even with all of their AI synonnyms, dynamic re-ranking, etc.) was meaningfully different then creating your own vector database? I appreciate you taking the time to make this video, it's validating to some stuff I'm working on for sure!
Loved the video validated a lot of the decisions we are making at work. I have a question however on the section about context injection. You mention that you search for relevant information to inject into the prompt. How do you accomplish the search part ? Is it using an index or a sql query amongst all columns ?
Glad it was helpful! The search is done through embeddings - we perform a similarity search between the embeddings generated from user's query and the pre-generated embeddings on the knowledge base (stored in a column using pgvector).
@@RabbitHoleSyndrome ahh so: 1) call OpenAi embedding api for the query 2) use cos sim to compare the query embedding against the stored embeddings 3) utilize the top results to inject into a prompt that we compile to send to OpenAi completion api ?
Thanks! PDF to MDX will be a tough task. But if your end goal is embeddings, you could consider pulling the content out of the PDF and generating embeddings on it directly without getting MDX involved.
Thanks! Pinecone is a dedicated vector database so will likely outperform pgvector and have more specialized features like additional index types. pgvector is attractive when you are already using Postgres or you need Postgres for other things and don’t want to manage multiple databases. Performance can still be quite good with pgvector, especially if you add an index.
Great explanation! I was wondering how to perform similarity searches in my files, there you have it! I'm also wondering if there would be a way (or need??) to speed up the similarity check step? For this application it might be fast enough, but imagine having a billion or trillion entries which you need to check you query against.. That would be a nightmare. Something like clustering ,say, N similar entries and in the first step only checking your query against their average value. If they are similar enough (above a certain threshold) you could then proceed to check against each of them and you have made only 1 extra check. If they're not at all similar, then you don't have to check against each and you have saved yourself N-1 unnecessary checks!! What do you guys think?
@@RabbitHoleSyndrome Thanks you for making this content for us for free! I very much appreciate it and you motivates me to share knowledge with my friends :)
Hey - CSV files are definitely doable. It will mostly come down to how you plan to pre-process them. Perhaps you can import your CSV file into a table itself and generate embeddings on the content within it.
Hey great content! I was looking for exactly this. Do you have a discord for asking queries? I was making the same thing but for ".md" files not mdx and faced some issues.
Yo, thank you for putting out your knowledge :). But I have question regarding the data search for the context. So basically your creating a graphdatabse with the gpt models and then you input the users question into the database to find the relevant articles/information?
I saw your video and quite impressed with it. I want to create a similar tool like ChatGPT for docs to showcase internally. I created an account on Supabase and then created a new project. But then I am lost. I don't know where you got the "Explorer" and "OpenEditor" on the left side. Can you please guide how to get there?
Hey! The screens you're referring to are for VSCode, a popular IDE used by developers (nothing to do with Supabase). Feel free to download that and from there you can check out supabase.com/docs to learn how to integrate your own project with Supabase. Supabase is just meant to be the backend provider for your project. Let me know if that helps!
Can you make the completion also tell which chunks have been used and link to them or get “read more” links or would you do that by just listing “top 5” matches from the context?
So if I understood correctly: The embeddings were only used to check for similarity between the user's input and the doc's content, in order to provide the prompt with relevant (text) context, right? Is there a way to provide the GPT model with the embeddings instead?
That’s correct. You could have used an alternate search method, but embeddings have a nice alignment with LLMs since they also use language model themselves. Unfortunately no there is not currently a way to inject embeddings directly into GPT today. Maybe this will change in the future or become available in open source models like LLaMa in the same way we’ve seen it happen with Stable Diffusion.
Great video! How closely do the .mdx files have to match this structure before they can be processed into embeddings? Do they need to export the meta const, for example?
Hey! Context injection does a couple things: 1. Primes the prompt with specific information we want GPT to reply with 2. We can always use up-to-date information - anytime we need to add/remove/update information in our knowledge base, it's as simple as a DB update. No need to re-fine-tune the model all over again
Nice, tried to follow along but found out that there are a lot of assumptions about knowledge your viewers have: Supabase and pgvector takes a while to find, creating the same table definition as you have seems impossible (where to enter the 1536 dimensions???). Installing pgvector locally means recompiling the stuff from the commandline etc. etc.
Hey! Sorry to hear you had trouble getting setup. My recommendation would be a local Supabase stack + migrations (similar to the video). There's a decent guide here: supabase.com/docs/guides/cli/local-development Under the hood this will run a modified version of Postgres locally in a Docker container that has all the extensions pre-installed. You can then enable pgvector in a DB migration using the "SQL" approach here under "Enable the extension": supabase.com/docs/guides/database/extensions/pgvector#enable-the-extension
16:10 would love an explanation on how embedding work with a document structure? I.e query is “summarize chapter 3”. The embedding sans retrieval don’t seem to capture the structure of the chunks that are contained in title chunk “chapter 3 “. All explanations on embedding I’ve seen all rely on the text content within a chunk.
You may find this post helpful: mattboegner.com/knowledge-retrieval-architecture-for-llms/ The community is starting to settle with "retrieval-augmented generation" as the terminology for this now.
I am blown away at how much information is densely packed into this. You got yourself a new subscriber, sir.
It’s staggering to think about how these technologies will shape the landscape for data and analytics. This is just the beginning.
Thanks for the sub! I am continuously blown away by the possibilities of large language models 😀
Damn, I spent like a week researching this shit on my own, and have been working on almost exactly the same thing. Processing MDX files into embeddings etc. It’s really cool to see somebody doing almost the same exact thing. Makes me think I am really on the right track!
Nice! Glad to help give some validation 😄 are you also building for docs?
Hey! how had you turn your pdf files into a propper mdx format ? tnx
@@automioai In this project no PDF files were used - all documentation had been written directly in MDX. You'll have to do some research on ways to extract text from PDF files. Once you have that, I wouldn't bother with MDX at all - just generate embeddings directly on that content.
Prediction: This is gonna get a million views. Just saw fireship video about vector databases and wanted to understand embeddings. Before I could even search, this video was in the page. Though I wasn’t interested in a 40 min video (had a feeling I’ll just stop after 5 mins like I usually do) I ended up watching it all. The rabbit hole 🐇 🕳️ format is so naturally elegant. Clear end to end use case. I secretly don’t want to share it with anyone but I am forced to fulfill my prediction.
Thanks for the comment! Glad to hear the format is working 😃
The clarity of this video while maintaining detailed granularity of the subject is very impressive and very appreciated. Thank you for making this video.
Thank you for sharing . I was struggling on how to get started . This was very well presented . From this side of the world asante sana !
Glad it was helpful 😃
Wow, this is incredibile... I can see a future where every docs site does the same thing. This is truly powerful stuff! Well done 👏
I’m am both excited and blown away by the possibilities. Thanks for watching!
This was so valuable to just settle the thoughts into a clear action plan as to how to implement in production. Thank you!
Glad it helped!
Ive been looking for this information for months. Such an excellent tutorial and I love that Supabase's code is all open source so i can actually clone it and read how it works in detail later. Thank you so much for the walk through. Super talented dude too - love the blender stuff.
Glad to hear it helped! Agreed - open source is amazing! Let me know if you hit any road blocks along the way, happy to help 😃
@@RabbitHoleSyndrome why is the generate embeddings file so different in the video then what is in the repo now? I can't find anything you talk about in minutes 10-13.
Hey @@fraternitas5117! Supabase moves pretty quick - the code I references has been refactored now to support multiple knowledge sources (ie. more than just markdown). You can find the markdown specific code here:
github.com/supabase/supabase/blob/1b2361c099c2573afa1fe59d3187343bb8f1bcab/apps/docs/scripts/search/sources/markdown.ts
Found your channel while learning React Three Fiber, subbed with notifications immediately. Today I get a notification for a well-explained ChatGPT tutorial, right as I embark on building a similar thing. Fantastic continued work, thank you very much!
Awesome! Thank you!
Wow. I'm so thrilled to know that you were one of the ones behind that great feature.
I've been using Supabase for 6 months, and have been pretty happy with it.
Except for the docs and the transition to 2.0.
I was blown away when I saw that it generated the code for me when I started writing its documentation
Glad to hear it has helped you! Any feedback on the docs that you think should improve?
@@RabbitHoleSyndrome So far so good!
I think one challenge is to know how we can check if a user's email address exists. (Or other specific user's metadata)
I couldn't find it in the docs.
There was a GitHub issue which said to store the user's data in a separate table as the auth table was private.
I ended up doing that and haven't had any problems.
Btw, thanks again for all the awesomeness!
First time on this channel. The way you structured the video, the pace and the explanations are all on point. Keep up the good work. +1 subscriber.
Glad to hear that, thanks for the sub!
This 40-minute video looks like 10 minutes. I have been researching related engineering topics recently and they have been very inspiring to me.
The potentials with LLMs seem to be endless 🤯
What a fantastic video and content. I’ve gone through multiple videos trying to better understand embedding and how to work with ChatGPT in the best way for querying large amount of content and producing an analyzed response. I’m not a developer, have a background in computer science but I’m a software sales person that is curious about technology and I was able to completely understand your video and content. Subscribed, liked and will be watching more of your videos. Thank you!
Glad it was helpful!
This content is really top notch. I appreciate the clear and detailed explanations of everything!
Glad you found it helpful!
Seems to have found one great coding channel among the noise of today; back to good old days when coding was boring & nerdy. Great job!
Fireship guy?
Either way, its been pretty useful in terms of learning API's and how to connect them to my nocode builder. Spent hours trying to get things working and the Assistant basically told me what i was doing wrong and how to fix it. So well done with the implementation.
Glad it helped!
So you're the one who made this beautiful thing. Pretty nice.
Dude, that's exactly what I was looking for. No more bullshit articles with clickbait titles, just DIY in the essence. Is there a way to support you through patreon or smth. ?
Glad to hear it! You support by watching 🙌
This channel is awesome! Love the rabbit holes you take us one! keep them coming please!
I am grateful to this video. This open my eyes to Embeddings
Glad it helped 😃
fantastic content, didn't expect it will be so informative in just 40 mins. Looking forwards to the next one!
Thanks for watching 😀
Insane, I love how you are able to do so many things. My laptop atm is unable to power every wish of mine (getting into 3D) but I hope I will soon be able to do so.
Amazed by your content. Fantastic work here .
Thanks, glad it helps!
best tutorial i have ever watched u are a genius ty!
The most amazing think is that I basically made a chatbot app in less than a week with only the help of GPT4, I had no knowledge of AWS services, PostgreSQL or python. Everthing you told in the video is what GPT4 told me.
All of the serves and database are setup, it has memory, STT, TTS and Cognito login/register.
It is quite amazing - and I’m sure it will only get better!
You are awesome! Now I can understand how these things work together.
Happy to hear it! 😀 thanks for supporting the channel!
This is a glorious illustration. Thank you very much! I've been trying to find an example of doing this, and yours has put it all together for me! Subscribed!
Glad it helped, thanks for the sub!
excellent . you are hands-on & practical
Absolutely amazing quality here, glad I found your video. Subbed!
Thank you!
Wonderful primer on prompt engineering.
You do a such great job bro ! I love what youhave built and your video, keep build great things bro.
This really is/was an epic video clearly and well laid out!
This is amazing, you’ve created Clippy just as enthusiastic and helpful you are!
Thanks a lot
Thanks for watching 😄
@@RabbitHoleSyndromehow did you generate the mdx files?
The MDX files weren’t generated - the Supabase team wrote them as you would any markdown file.
incredible end to end tutorial, nicely done
Thank you! 😃 Have you explored embeddings or pgvector?
incredible value in this video
Thanks Greg for a great explanation. I like your presentational style!
Glad it was helpful!
This video was fantastic, lots of information given in an easily digestible way. Subscribing!
Glad it was helpful, thanks for the sub!
I'm doing something even cooler with the vector embeds I cant wait till I can share!
whaaaattt tell us??
I'm reading the comments, and I'm like.. Yeah WTF 🗿🔥
Many thanks for this insightful and helpful video!
Happy to help, thanks for watching!
I'm at 1:42 and this video already is 10/10
First time here. This is so well done. Subscribed. Your viewer number will explode! I like how you approached the topic in a very calm way without jumping on the "LLMs will take over the world" train :) You don't happen to have the clippy blender asset somewhere?
GREAT content. It explained everything you need to know about creating 'chat docs' or similar in one run, and all open source. Kudos! And subscribed.
Thanks for the sub! Great to hear 😃
I have not seen such a great video in a while! How wonderful have you explained the whole process 👍👍! Could you explain a bit more about how did you choose 0.78 as threshold for embeddings comparision? have you statisticized that wether the most relevant sections can be found with it?
Glad you liked it! 0.78 was a first-stab threshold that worked best based on a limited sample of test queries. I wouldn’t claim that this number is universal - almost certainly this could change by domain.
Amazing video! I'm going to put this in practice right now!
Awesome! Feel free to share as you make progress!
28:37 Whenever I see examples of decoder (GPT) prompts starting with "You are a helpful finance advisor" or "You are an enthusiastic support rep", I can almost see the AI clearing its throat and sitting up straight and saying "right, ok". Gimme that can-do attitude, GPT!
🤣
Just what I was looking for - thank you very much🥳
Great 😃 Thanks for watching!
This is a fantastic video! Thank you very much for sharing :D
Quick question - Currently if the info is not in the documentation it responds "Sorry I don't know how to help with that". But how can we make it respond like this:
"Sorry I don't have relevant info in the documentation but you can do something like this".
For e.g. "I don't have any info about how to make banana pancakes in the documentation, but here is how you can make one...."
Idea here is to make it act like chatgpt on top of the information provided. Keen to know more on this and thank you so much for making this video :D
Great content.
I wonder if it can generate code as well.
Lets say you are doing this for cloudflare products, kv, workers, durable objects etc all documents injected. Then give prompt like "generate a worker for x" and it will specifically generate with given docs etc..
I think this is where the industry is going next!
Cool video. I was thinking about doing something similar for some reference pdfs. Thanks for the video
Thanks for watching & best of luck!
This is truly valuable and useful content, thanks a lot.
You bet! Glad it was helpful
One thing that might help is if the question result shows the links to the documents that it acquired the information from. Since you are currently fetching which document to run chatgpt on based on similarity of features, maybe you can change the prompt so that it also returns the link of the document that was deemed as a "similar document"
Absolutely! This should definitely be the next progression.
that's great, you did great job! helps me a lot in this.
Glad it helped!
Amazing video! This is exactly what I was looking for a long time. You basically explains everything I wanted to know about how to create a search engine using open ai.
But I have a few questions:
How much did you spend on open ai embending API building this?
How much supabase spends monthly with searchs using the open ai api?
It is possible to use an open source embedding API instead of calling the open ai api ? Wouldn't it be less expensive than the approach you took?
Glad it was useful! As for costs, you may be surprised how inexpensive OpenAI embeddings are (at least I was).
To put it in perspective, for the Supabase guides we currently have around 1500 page sections which total just over 220000 tokens. At OpenAI's current embedding price ($0.0004/1k tokens), that brought us to just less than $0.10 for the entire guide knowledge base (~one-time pre-processing). After that the average query is likely
Excellent video and the whole methodology for deep dive into sou much information!!!! will you consider elastic as vector database?
Thank you! I haven't looked too much into elastic yet. I might consider it if I was already using Elastic in my stack. Have you tried it?
This is amazing. Super clear and short explanation of embeddings. Great walkthrough the code touching on the relevant parts. Subbed the channel👍
I see the code is still in the Supabase repo - but the Clippy is not there? what happened?
Glad it was helpful!
Edit: Just realized you might have been talking about the Clippy graphic on the site, not the code. Search and Clippy have been combined into the same interface - you can find Clippy by clicking search, then switch to “Ask Clippy”.
Original:
Things move quick at Supabase 😆 The Clippy frontend code got moved when search was upgraded to also use embeddings. After the refactor everything is just under “Search”:
github.com/supabase/supabase/blob/0ecc238ad6d81202bb2301f7919b166a98929697/apps/docs/components/Search/SearchModal.tsx
The backend Clippy logic is still in the same edge function:
github.com/supabase/supabase/blob/0ecc238ad6d81202bb2301f7919b166a98929697/supabase/functions/clippy-search/index.ts
@@RabbitHoleSyndrome cool. You can feedback to Supabase, that a) this video significantly increased my interest in adoption of Supabase b) clipy icon is tooo small , it should be large and obstructive.
Serious question Prompt Engineering: With Chat Completion API - do you do system persona and pack all prompt into user? Or would you break prompt like here into example 3 user entries? E.g.: Context, question, use markdown. Does it make a difference?
Good feedback 🙌 and great question about prompt engineering in the chat API. We are actually experimenting with this right now and trying to understand what produces the best results. At the moment we do a bit of both (system message and user message with a bit of prompt overlap between both). OpenAI says that the model doesn’t pay strong attention to the system message, so it may be better to use system message strictly to set identity and provide instructions & context in a user message.
Thanks for the knowledge share my friend. 💪🙏
You bet!
Cool project, would be cool to create a tool like this that you can embed in any documentation, reading directly the markdown or scraping the website.
Definitely!
Amazing video! Great and detailed information!
Glad it helped!
This is really helpful and valuable, thanks a ton!!!
You bet, thanks for watching!
Loved it, came for the embeddings and left with that and whole lot more! Too bad that the clippy did not made to the web on supabase :). Copyright problems?
Thanks for watching! Clippy did make it - but things move fast at Supabase 😅. This feature has evolved into a unified cmd+k menu and just renamed as “Supabase AI”.
Good stuff! Curious how you are handling performance of the respons? We setup a similar pattern and have found that GPT 3.5 obviously returns faster, but GPT-4 returns with much better quality. Have you experienced similar results? In this context does the speed of 3.5 outweigh the GPT4 quality - for us we are seeing some GPT-4 responses take >30 seconds, which is pretty terrible from a UX perspective.
Also, curious if you are still using Algolia for the other "basic" search experience? We were thinking about using Algolia for search, and then sending the top few most relevant results to OpenAI for summarization. Not sure if the quality of Algolia's search results (even with all of their AI synonnyms, dynamic re-ranking, etc.) was meaningfully different then creating your own vector database?
I appreciate you taking the time to make this video, it's validating to some stuff I'm working on for sure!
Watched because of the content, subscribed because of the dog. 👍🏻
Thanks for the sub! 🐶
Thanks for explaining everything so clearly :))
You bet! Thanks for watching 😃
Loved the video validated a lot of the decisions we are making at work. I have a question however on the section about context injection. You mention that you search for relevant information to inject into the prompt. How do you accomplish the search part ? Is it using an index or a sql query amongst all columns ?
Glad it was helpful! The search is done through embeddings - we perform a similarity search between the embeddings generated from user's query and the pre-generated embeddings on the knowledge base (stored in a column using pgvector).
@@RabbitHoleSyndrome ahh so:
1) call OpenAi embedding api for the query
2) use cos sim to compare the query embedding against the stored embeddings
3) utilize the top results to inject into a prompt that we compile to send to OpenAi completion api ?
You got it 👍
Thanks, amazing content. there is a way to junk pre-process PDFs to Mdx ?
Thanks! PDF to MDX will be a tough task. But if your end goal is embeddings, you could consider pulling the content out of the PDF and generating embeddings on it directly without getting MDX involved.
Excellent video!!! I have one question about Postgres/PgVector versus Pinecone. What do you think about Pinecone? Why did you prefer PgVector?
Thanks! Pinecone is a dedicated vector database so will likely outperform pgvector and have more specialized features like additional index types. pgvector is attractive when you are already using Postgres or you need Postgres for other things and don’t want to manage multiple databases. Performance can still be quite good with pgvector, especially if you add an index.
@@RabbitHoleSyndrome thanks!
Enjoyed this tutorial. Would love to know how you'd approach fine-tuning a model with this data to build a chat bot?
Thanks! Any reason you're looking to fine-tune vs. embeddings+context injection?
Such a great video.
Damn that's a great, helpful video!
Glad to hear it was helpful, thanks for watching!
Amazing new feature in supabase. Btw wdym by supabase hired you for building this tool? or do you mean they sponsor you?
They asked me to help develop it!
Great explanation! I was wondering how to perform similarity searches in my files, there you have it!
I'm also wondering if there would be a way (or need??) to speed up the similarity check step? For this application it might be fast enough, but imagine having a billion or trillion entries which you need to check you query against.. That would be a nightmare. Something like clustering ,say, N similar entries and in the first step only checking your query against their average value. If they are similar enough (above a certain threshold) you could then proceed to check against each of them and you have made only 1 extra check. If they're not at all similar, then you don't have to check against each and you have saved yourself N-1 unnecessary checks!!
What do you guys think?
This was very useful! Thanks a lot ofr sharing. Are there plans for keep it up? I'm glad to found this type of content
You bet! Definitely - videos will keep coming. Some take a bit longer than others - thanks for your patience 😃
@@RabbitHoleSyndrome Thanks you for making this content for us for free! I very much appreciate it and you motivates me to share knowledge with my friends :)
Subscribed, amazing content.
This is really interesting! I'm looking to build something similar.
Best of luck!
Hey, awesome video! Could I check with you if it is possible to search the database that contains CSV files?
Hey - CSV files are definitely doable. It will mostly come down to how you plan to pre-process them. Perhaps you can import your CSV file into a table itself and generate embeddings on the content within it.
I'd love to learn how to also incorporate user-feedback (thumbs up/down)
This will likely make it into the next iterations. Will be a good challenge!
@@RabbitHoleSyndrome That's awesome! Can't wait
@@RabbitHoleSyndrome amazing video! Please let us know if you get to it:)
@@RabbitHoleSyndromeany progress?
Hey great content! I was looking for exactly this. Do you have a discord for asking queries? I was making the same thing but for ".md" files not mdx and faced some issues.
Thanks! For now, feel free to reach out to the channel's email address and I'll do my best to give you a hand.
Yo, thank you for putting out your knowledge :). But I have question regarding the data search for the context. So basically your creating a graphdatabse with the gpt models and then you input the users question into the database to find the relevant articles/information?
I saw your video and quite impressed with it. I want to create a similar tool like ChatGPT for docs to showcase internally. I created an account on Supabase and then created a new project. But then I am lost. I don't know where you got the "Explorer" and "OpenEditor" on the left side. Can you please guide how to get there?
Hey! The screens you're referring to are for VSCode, a popular IDE used by developers (nothing to do with Supabase). Feel free to download that and from there you can check out supabase.com/docs to learn how to integrate your own project with Supabase. Supabase is just meant to be the backend provider for your project. Let me know if that helps!
Brilliant.
Can you make the completion also tell which chunks have been used and link to them or get “read more” links or would you do that by just listing “top 5” matches from the context?
excellent info, great presentation
Glad it was helpful!
You basically build llama-Index yourself, good job
So if I understood correctly: The embeddings were only used to check for similarity between the user's input and the doc's content, in order to provide the prompt with relevant (text) context, right? Is there a way to provide the GPT model with the embeddings instead?
That’s correct. You could have used an alternate search method, but embeddings have a nice alignment with LLMs since they also use language model themselves.
Unfortunately no there is not currently a way to inject embeddings directly into GPT today. Maybe this will change in the future or become available in open source models like LLaMa in the same way we’ve seen it happen with Stable Diffusion.
Great job 👍, this was really helpful
Glad it helped!
Really good video, nice job! 🎉
Cheers! 😀
This is awesome, thank you!
Glad it was helpful!
Great video! How closely do the .mdx files have to match this structure before they can be processed into embeddings? Do they need to export the meta const, for example?
The meta const is optional! You’re also free to tweak the pre processing logic to fit whichever format you need to work with
super helpful. thanks. love supabase
Happy to help!
Could you please explain why you chose context injection over fine tuning? What are the strenghts and weakness etc.
Hey! Context injection does a couple things:
1. Primes the prompt with specific information we want GPT to reply with
2. We can always use up-to-date information - anytime we need to add/remove/update information in our knowledge base, it's as simple as a DB update. No need to re-fine-tune the model all over again
amazing!
Thanks for the share !
Amazing Content..
this would be amazing for obsidian note taking application
Great idea!
Woah, wonder if you could make this work with a browser extension, could have either a private or public store of indexed website embeddings
Can’t see any reason this wouldn’t work. If you build this please report back 😃
Nice, tried to follow along but found out that there are a lot of assumptions about knowledge your viewers have: Supabase and pgvector takes a while to find, creating the same table definition as you have seems impossible (where to enter the 1536 dimensions???). Installing pgvector locally means recompiling the stuff from the commandline etc. etc.
Hey! Sorry to hear you had trouble getting setup. My recommendation would be a local Supabase stack + migrations (similar to the video). There's a decent guide here:
supabase.com/docs/guides/cli/local-development
Under the hood this will run a modified version of Postgres locally in a Docker container that has all the extensions pre-installed. You can then enable pgvector in a DB migration using the "SQL" approach here under "Enable the extension":
supabase.com/docs/guides/database/extensions/pgvector#enable-the-extension
16:10 would love an explanation on how embedding work with a document structure? I.e query is “summarize chapter 3”. The embedding sans retrieval don’t seem to capture the structure of the chunks that are contained in title chunk “chapter 3 “. All explanations on embedding I’ve seen all rely on the text content within a chunk.
Any hints on papers on "context injection" for LM or LLMs?
You may find this post helpful:
mattboegner.com/knowledge-retrieval-architecture-for-llms/
The community is starting to settle with "retrieval-augmented generation" as the terminology for this now.
🔥🔥🔥 This is amazing!