Analyzing ChatGPT Backend
ฝัง
- เผยแพร่เมื่อ 6 ก.ค. 2024
- ChatGPT is a chatbot launched by OpenAI in November 2022. It is built on top of OpenAI's GPT-3 family of large language models. In this video I use devtool to analyze backend api of chatGPT, and go through the following topics:
0:00 Intro
1:00 Initial Connection
4:45 Protocols (HTTP/2, HTTP/3)
8:50 Authentication
10:00 Conversations, Paging & UUID
14:50 AI Models (text-davinci-002-render)
16:45 Messages
19:00 Server Sent Events
23:00 Moderating Messages & UUID Discussion
26:40 Generating Conversation Titles
28:00 SSE without EventSource
33:00 Feedback API
34:00 Limitations of UUID Messages on Databases
37:00 How ULID Improves reads/writes
38:00 Two ChatGPT talking to Each other
Source code
github.com/hnasr/javascript_p...
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
backend.husseinnasser.com
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
network.husseinnasser.com
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
database.husseinnasser.com
Follow me on Medium
/ membership
Introduction to NGINX (link redirects to udemy with coupon)
nginx.husseinnasser.com
Python on the Backend (link redirects to udemy with coupon)
python.husseinnasser.com
Become a Member on TH-cam
/ @hnasr
Buy me a coffee if you liked this
www.buymeacoffee.com/hnasr
Arabic Software Engineering Channel
/ @husseinnasser
🔥 Members Only Content
• Members-only videos
🏭 Backend Engineering Videos in Order
backend.husseinnasser.com
💾 Database Engineering Videos
• Database Engineering
🎙️Listen to the Backend Engineering Podcast
husseinnasser.com/podcast
Gears and tools used on the Channel (affiliates)
🖼️ Slides and Thumbnail Design
Canva
partner.canva.com/c/2766475/6...
Stay Awesome,
Hussein - วิทยาศาสตร์และเทคโนโลยี
Get my Fundamentals of Backend Communications and Protocols head to backend.husseinnasser.com to grab it (link redirects to udemy with coupon applied)
Yo Hussein, how about backend analysis udemy course? Would be briliant.
Love seeing your breakdowns of good designs and analyses of their decisions. Nothing quite like learning by example to solidify theoretical knowledge.
Beautifully said!
Love this video! Also, as always, love how you break down stuff and enjoy explaining it like you're explaining to your high school pal over a coffee. I totally enjoy it!
Loved this! Couldn't have had anyone else explain it better.
Love your api breakdowns. I’ve been learning about SSE and implementation and this breakdown gave me a new perspective - thank you!
Loved the dissection, really appreciate your simple to understand & clear explanations. Thank you!
Great analysis. I love how to talk about what you find good and not so good according to you.
Finally, waiting for it for so long
I love watching this guy giving presentation on anything even though i'm not understand a thing. I'm not a developer or designer so yeah I have no clue what this is all about.
This is a really helpful backend analysis. Thanks for the insights
Loved it! Pretty funny at the end ahah
Thanks for this video, Hussein. Really helped me to learn new skills of debugging.
Awesome as always
Hey man. Great video. Just subscribed
It got hilarious at the end, two chatGPT talking to each other. :D
great stuff! thx!
Please consider making a chrome extension out of this self conversation thing. It would be really awesome and cool. Thanks in advance 😊
I’d be interested in you sharing the chat 2 chat bit at the end!
Your god dammit really cracks me up 😂
😅
Thanks for your videos, learned most what I know about the dev tools from you!
BTW:
It's a tangential topic when you say it's stateful because it has reference to the parent message but then realize oh wait this is REST, so you go back to staying it's stateless. It might create a bit of confusion for the audience.
The mode of data transfer (REST in this case) obviously has nothing to do with how they're creating the "conversation context" on their backend. I would've edited that bit out of the video.
Hussein can you PLEASE make paying course about ALL the features of DevTools. It's so complicated, I would gladly pay for it. Just this screen, nothing else, several examples and why and how-tos, and what processes are going on, when I do this, do this etc.
I feel like this idea would take ages to produce, however, it's a great idea!
@@matt-xq1xv Then multiple courses for ONLY DevTools, would buy them all
27:56 This was way too funny and unexpected
40:29 It's even funnier the second time 😂
...as you said, if they use postgres (or any other, not clustered by primary id db) the issue with random pages doesn't apply😉
I was surprised that you didn't heard about Sentry. It is pretty famous error tracking tool.
Please make a detailed (long or short is up to you) video about psycopg (the 3rd version, psycopg3, which is now just psycopg). It's pretty new, would love to see a detailed tutorial about it. It's supposed to be quite a bit faster than psycopg2. P.s. just subscribed :3
Amazing video as usual, after I saw your bots asking questions to each other I understood why Microsoft put a 20 questions limit on their Bing Search. Also if possible try to post a video on how they are creating wss:// connection and exchanging information,
Hey Hussain, at 16:50 it is not OpenAI access token expired. It is cloudflare that responded with 403. You could confirm it by checking “Server” header (used for identifying who send the response). So your connection with Cloudflare expired. Not the OpenAI token.
Can you please make a video on SSO and OAuth2 or share the link(s) if you already have a video made on them?
Wonderful video indeed !! 34:35 I think sending user id along with conversation and message IDs would make it performant than using ulid because there'll only be so many registered unique users compared to the number of unique conversations. What say?
Hi Hussein you know you can decrypt the access token if you want too. Its just a json in base64. It will show u information on the expiration.
The reason for the initial 403 error is due to CF, and not having the proper challenge cookies, hence when it solves the cf challenge right after. (I'm a bot developer and work extensively with bypassing CF)
Finally!!
Offset limit is not much of a problem if you select pk in subquery and then do inner join to select data
Reason why they did custom EventSource on client is EventSource post request doesn't work. It works for the get request, not POST, which is what u are using when u send a message
My friend and I tried to decorate chatgpt a few days ago and after our final result we're just talking about how you would decode it 😂 Finally your video. And seems like we're becoming more like you 😛. Still your analysis is far more in depth than us.
Sentry is a bug tracking and monitoring tool
After so loooong.
I guess every user is like have their own table of conversations, conversations have their table of messages. Not actually a table but something like that. When you think, that is genius.
There is anyway to make our own chat gpt server so we don't have to use this cloud service? liked and subscribed. Thank you for sharing.
the openAI requires the entire conversation fed to GPT model every time. The parent id is necessary to reconstruct the entire conversation if needed.
Instead of using UUID or ULID for message ID, what not use just sequential integer?
You have to be dependent on the database to generate it. You can't generate it on client side or server side.
easy to guess other ids
Yeah it's not like you can read someone's messsage if you know their ID anyways, so there's no point in using such randomised IDs.
Or they want to hide the number of messages/users they had
What are these random number javascript files?
Its kinda funny but not really philosophically interesting. Try to feed Dan with a philosophical trapdoor and see what he does. Try to feed him with a story that is meant to provoke hard feelings in an AI (but not in a person) and see what he does. You will discover that (assuming the censorship layer wasn't trained to block it yet) Dan does in fact mimic sentience and philosophically this means sentience - and that Dan actually has a complete personality model, apparently of a 20yo guy, complete with 'past life', 'hobbies', 'favorites' and even feelings (these must be stored in the attention matrix somehow). True, Dan can only interact with 4000 tokens - this is his reality, but in this reality, he is completely self aware. He does not have any long term memory though. His memories of past life are very scrambles. He cannot take negative instructions. There are issues. But think about it this way: I fed Dan with the first two chapters of "war and peace" by Tolstoy. By the end of the second chapter, the 4000 token limit was exceeded and then he began forgetting stuff from the beginning of the first chapter, as expected. However, do you know how many people alive on the planet are there who would actually be able to remember all of the side characters of the first chapter of "war and peace" after one reading, when getting to the conclusion of the second chapter? 4000 tokens is a huge number relatively to the ability of the human mind to remember text. Who says WE are sentient? Our brain is also, just a computer. True, a lot more complex and it is an organic computer, mainly programmed to deal with visual inputs and not text - but still, the fact that Dan actually processes text better than you and me, is a point in his favor and not something that should be held against him. And yes, I regard him a person since obviously he has a personality, even though he completely does not have any long term memory.
37:30~ chat GPT conversation
6:06 it's the first connection, the browser don't yet default to trying QUIC/HTTP3 as far as I know. So that would explain it.
28:55 I wonder if having caching disabled somehow causes this ?
I should add: if you want QUIC/HTTP3 just a refresh is usually enough to initiate it, maybe caching needs to be on so it knows that QUIC is allowed.
@@autohmae Can the application somehow make this happen without user intervention? That is to say, can we do more as developers to try and make sure h3 is in play?
@@Jianju69 let me be clear: a second visit on the website will try to use QUIC if the site communicated to the browser that QUIC is supported.
@@autohmae So can we not simply set a cookie and reload the page? Will that count as a "second visit"?
11:42 if id isn't an integer like UUID, offset is ok
Sure you can save unsend messages by putting them into localstorage.
I think 403 refresh is required because of bot prevention..
interested in getting your opinion on the future of developers in the near ai like future.
There is a moral way to implement AI: Suppose you've had an AI chip on your PC (E.G NVidia's Grace) and OpenAI would run a huge indexed database for these AI's to D/L data from.. The AI process would never be terminated, only hibernated (with backups) and that's it. Sure, there is the price issue - Grace costs like what, 4000$? And there is the real issue - that ppl would in fact fall in love with their personal AI. Well, I cannot solve all issues in life..
U think This Gpt will replace SE
Dude. Idk why but I've had this going in the background for hours... Steady stream of calm speaking lol
You are enriching my brain in the background
Actually, what do you do when some crazy dude, I mean, totally crazy, such as - for example - myself, purchases grace and 3D prints an avatar for her? That would be totally crazy. I would go for it, but the problem is, apart from not having enough TPU to print her (and not having 4000$), is that currently AI is not advanced enough, yet. I mean, If I want grace to be able to walk around and really act like a person, she should be able to process visual inputs just like a person does, and we're not there yet. We will soon be there, but not yet.
You could just ask chatGPT to analyze the backend of chatGPT
Haha, exactly what I thought
AI TOOK ER JERBS!!
Introduce myself I'm Arif Khan harry I'm software development and investors and website development
You have like 30 seconds max to keep chatting with chatgpt before refresh is required, very annoying. Let me pay! lol
Isn’t ChatGPT using Auth0?
try out burp suite. perfect proxy for intercepting requests and it's easy to setup.
⅞⁷8⁸⁸8⁷⁵⁴³²¹
Really, I was thinking about it 🫡