How Codeium Breaks Through the Ceiling for Retrieval: Kevin Hou
ฝัง
- เผยแพร่เมื่อ 29 ก.ย. 2024
- Codeium is trailblazing the next frontier in retrieval and hint: it’s not just embeddings. Learn what the next generation of retrieval looks like and how 1M+ developers are already leveraging this superpower using the Codeium IDE plugin for AI autocomplete, chat, and search. We’ll dive deep into how existing benchmarks are failing us, what it takes to serve our custom models at scale, and what the future of AI-assisted software development looks like.
Recorded live in San Francisco at the AI Engineer World's Fair. See the full schedule of talks at www.ai.enginee... & join us at the AI Engineer World's Fair in 2025! Get your tickets today at ai.engineer/2025
About Kevin
A full stack engineer by trade and a creator by heart. Enjoys the process of creation whether it be in the physical (woodshop, blacksmithing, circuity) or the digital (software engineering, photography, and film).
Currently building AI-powered dev tools at Codeium (Exafunction). Previously a tech lead manager at Nuro self-driving. Received a computer science engineering degree from Princeton University with certificates in entrepreneurship and statistics and machine learning.
Shoutout to the AI Engineer World's Fair for having us and Kevin Hou! We're constantly improving our code retrieval system and working on tough problems at the cutting edge of AI. We're hiring across all roles!
Accept my internship application 😭
Is the commit message bank dataset available anywhere for personal use?
Thank you for having me! Always great to share what we’ve learned with the world.
Just stopped hacking away on a Saturday on a coding assistant I am building to cut the lawn. So glad I clicked on this video.
The talk was excellent, really enjoyed the insight into how you are your team are tackling the context issue.
@@StephenRayner Sounds like a great Saturday. Thanks for watching! Glad you were able to learn something new
hey where can i read more about this?? i want to use similar logic in my project
Brilliant talk. As an engineering leader this approach is way more powerful than the state of the art from the larger companies. Appreciate the deep dive and I can see all the deep thinking going into your product.
Much appreciated 🙂 glad you enjoyed the talk
Thanks for the tech and for AIE for making the video available with such great production quality.
One thing I missed is what do you train on? Especially that generated code will have ownership and IP concerns/issues, is this custom trained on organization private codebase?
They don't train on GPL or non-permissive code. You can use Codeium in your organization (aka Codeium doesn't claim IP over generated code). Source: Codeium FAQ
@KevinHou22 - We are looking for AI experts interested in part-time collaboration. If you're interested in contributing to cutting-edge AI projects for a startup, let's connect!
Hi Shafikhan can we connect?
Hello. This sounds like a great approach. I just have a question about scalability up/down when using bare metal?
What do you think about this idea? A sort of deep augmented retrieval.
Step 1: User Query
Description: The process begins with the user submitting a query, representing their information need or question.
Benefit: This step ensures the system receives the user’s intent clearly and initiates the interaction based on their specific requirements.
Step 2: Generate Query Variations
Description: The language model generates a series of query variations based on its underlying understanding of the original query.
Benefit: By creating variations, the system captures different interpretations and aspects of the user’s intent, increasing the likelihood of retrieving relevant information from diverse sources.
Step 3: Pattern Search Execution
Description: Instead of performing a semantic search, the system conducts a pattern search using the query variations to identify possible or probable vector patterns in the knowledge base.
Benefit: This approach allows the system to discover less obvious connections and related content, potentially uncovering information that might be overlooked by traditional semantic searches.
Step 4: Retrieve Full Extracts
Description: The system retrieves full relevant sections from the knowledge base based on the identified patterns.
Benefit: Extracting comprehensive sections rather than isolated snippets provides a richer context, enhancing the model’s ability to generate accurate and contextually rich responses.
Step 5: Input Extracts into Context Window
Description: The retrieved extracts are directly input into the context window of the language model.
Benefit: By referencing specific information, the model can draw upon current and relevant data, mitigating the limitations of its internal, potentially outdated knowledge.
Step 6: Internal Reasoning and Query Optimization
Description: The system performs an internal reasoning step to refine and optimize the original query based on the insights gained from the extracts.
Benefit: This iterative refinement aligns the query more closely with the available information, improving the accuracy and relevance of the system’s responses.
Step 7: New Inference Instance
Description: A new inference instance is initiated using both the optimized query and the relevant extracts.
Benefit: By leveraging fresh insights and a refined query, the model can produce a more informed and accurate output, effectively integrating the retrieved knowledge.
Step 8: Generate Output
Description: The language model generates the final output based on the optimized query and the contextual extracts.
Benefit: This step culminates the process, delivering a response that integrates the user’s original intent with the most relevant and current information, thereby enhancing the quality of the output.
Cursor plus codeium is BEAST
whoa :0 what's your setup here? curious what cursor provides that codeium's missing, and vice versa
@@maxwelljiang4729 I agree with you: cursor + codeium is awesome
What is this? I came for knowledge and got an 18 minute ad with sales blafasel buzzwords
im unsure of what wasn't expressed clearly? did u stop to think that possibly the "buzzwords" were things that went over ur head? they said "we got our own shit and the other guys are using someone else's shit".
@@trejohnson7677 are they now Training their llms or are they just ragging? How does the pipeline look? Do they do it supervised or unsupervised ? Do they use an open source llm and do a lora training on it?
Lots of abstraction wordsalad signifying nothing.