How Instagram content is served through GraphQL - Devtooling Instagram
ฝัง
- เผยแพร่เมื่อ 15 ก.ค. 2024
- Welcome to another dev tools video, in this video I use dev tools to break down how Instagram works. We notice the mix use of legacy REST API vs GraphQL which is great. I give my thoughts on GraphQL and I use therock as a subject as always. It is an interesting episode, enjoy
0:00 Intro
1:00 First Request
9:30 Instagram legacy REST API
16:30 Instagram GraphQL
23:30 Instagram uses WebSockets?
24:20 Security and Origins
DevTool them All Playlist
• Dev Tool them ALL!
Dont use offset • don’t use “offset” in ...
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
database.husseinnasser.com
Introduction to NGINX (link redirects to udemy with coupon)
nginx.husseinnasser.com
Python on the Backend (link redirects to udemy with coupon)
nginx.husseinnasser.com
Become a Member on TH-cam
/ @hnasr
🔥 Members Only Content
• Members-only videos
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
database.husseinnasser.com
Introduction to NGINX (link redirects to udemy with coupon)
nginx.husseinnasser.com
Python on the Backend (link redirects to udemy with coupon)
nginx.husseinnasser.com
🏭 Backend Engineering Videos in Order
backend.husseinnasser.com
💾 Database Engineering Videos
• Database Engineering
🎙️Listen to the Backend Engineering Podcast
husseinnasser.com/podcast
Gears and tools used on the Channel (affiliates)
🖼️ Slides and Thumbnail Design
Canva
partner.canva.com/c/2766475/6...
🎙️ Mic Gear
Shure SM7B Cardioid Dynamic Microphone
amzn.to/3o1NiBi
Cloudlifter
amzn.to/2RAeyLo
XLR cables
amzn.to/3tvMJRu
Focusrite Audio Interface
amzn.to/3f2vjGY
📷 Camera Gear
Canon M50 Mark II
amzn.to/3o2ed0c
Micro HDMI to HDMI
amzn.to/3uwCxK3
Video capture card
amzn.to/3f34pyD
AC Wall for constant power
amzn.to/3eueoxP
Stay Awesome,
Hussein - วิทยาศาสตร์และเทคโนโลยี
Head to database.husseinnasser.com for a discount coupon to my Introduction to Database Engineering course. Link redirects to udemy with coupon applied.
Great content as always. Looking forward to "devtooling codedamn playgrounds" video 👀
That would be fun to watch
That subtle reel roast 😂😂
Maybe this will help somebody: you can pause network request capture using the little buttons on the top left 😌
Those OPTIONS requests are called preflight requests. It's required for enabling CORS.
I believe instagram uses Django as their backend framework. One of Django's core design philosophies is that URLs should be beautiful. Hence the trailing slash. Also I read somewhere sometime ago that the trailing slash helps with SEO. But yeah seems wasteful
There is also a default setting with Django that prevents doing a post request if the url does not end with a slash. I think it may be any request if it is not get
As a Django developer, I don't care about 100ms, I am trusting what Django like to do about trailing slash.
@@maheshprajapati9441 btw it's not about developer experience, it's about saving company's 100ms load time on every page request. So, you should care about it.
"something/" != "something" for SEO, it detect as a duplicate content, that's why Instagram use only one URL, not both with trailing slash and without
why does instagram cares about seo?
About the slash - yes they can detect and serve BUT from SEO perspective which is important you have to redirect. Loading content from both ways (with slash and without) will lower the rank of the page because google considers both URLs as different pages with equal content which is counted as duplicate and it's bad for Google and that's why redirection is needed and the price for redirection is acceptable. If Instagram wants a good SEO presentation they should pay the price of delay.
And a few other reasons. Importantly, if you cache this, these actually represent different resources which can be out of step with one another. Then you need the associated code and architecture to maintain both versions, or not cache it, which is, just, bad-sauce. Google will also drop a 301'd page from their index and crawl the new page. Just tidy. Many other reasons beyond this also..
And also I notice the canonical URL in the DOM which - to Google - takes care of the slash-no-slash issue :)
started recently and binge watching your videos. exactly the type of channel i was searching..
the power of graphql is allowing the client to request what they want in a very flexible manner without heavily being relient on some or some document to tell you how to pass data, it is really intuitive and the flexibility is amazing but with great power comes great responsibility, do not ask for too much data. it is really tempting to just ask for all the data in the structure you might need which could be uneccesary sometimes. but still just the ability to just ask for what you need how you need when you need is golden
GraphQL should never be used when working with non-fullstack frontend workers. They would basically generate SQL requests that would crash the backend.
Any frontend guy who uses graphql should be able to assess the backend strain each of his request is creating, thus should be a fullstack engineer.
one of the most useful programming channels on youtube as usual, thanks
Nothing like a Hussein upload during lunchtime man. Thanks bro 👍🏽
A fun fact I love; Instagram is one of the few social medias that started as a mobile app first, and then came out with a browser version
slash not slash maybe the same content to you, but not to google (they probably know something we don't.. cause they do get a lot of cross developers moving from google to fb and vice-versa)...
Man you don't even nudge even when criticizing the bk of a company like Insta 😂, honest as always 🙌
Sir, you seems to be fan of therock😂😂
Great content as usual, totally pumped up😅
24:20 Well, even when you are on home page you can get notifications from chat.
About the trailing slash : It's more of a semantic thing. A trailing slash usually represents a folder, or a directory. This is because you will have instagram posts under /therock/. A lot of navigation URLs use trailing slashes actually. What is wrong/not recommended, is to use a trailing slash for actual resources (images, files, etc)
In the main page, using the reals tray api, you can see the first story of an user and the user object is bigger
Well done Huss :)
Web socket to update real time data and likely messaging. I do wonder if it could be used for live interaction tracking somehow?
I think the trailing slash is enforced to sustain consistency.
Search engines could 'possibly' identify both pages as unique leading to duplicate content (they could probably easily detect it but it's better to make sure). Or maybe some logic processed on the page could depend on the current url, for concatenation or something. Like: current_url + '/friends'. (Maybe this is a 'problem' on its own but that's a different debate).
Regarding the lost ~100ms: I would favor stability AND consistency over such time. Especially considering the page request took ~1.5s (of which the redirect request makes up ~6%)
I'm also experienced the trailing slash in apiname ending on django
Hi Hussein, from where does all these security domains setup because on OPTIONS call to server they're only setting one domain?
Hi Hussein, What do you think about Wundergraph ?
"Hussein, that slash makes a huge difference"
- some marketing dude probably
love Devtooling series! can you make Devtooling Facebook?
the / at the end of the URL is a must for SEO reason, otherwise, google treat as a different URL,
wow, google is that dumb?
Can you do this with Telegram?
Now Instagram changed the whole API structure. just after this video!
They have change the protocol to h3 after your video :)
A lot of things they have changed after the video, highlight is missing now :p
yeah it's true, and now they're using QUIC instead of TLS 1.3, what a coincidence 😅
I don't think the redirect with the trailing slash is a big deal. I think barely anyone is typing instagram urls by hand and missing the trailing slash, so eventhough it's not optimal, it doesn't actually hurt any users (or barely).
And maybe there is some reasons for this redirection (maybe the trailing slash is better for SEO, and therefore they had to migrate the url structure to get better rankings ?)
Hi hussein, why that does not show the connection ID which means the TCP Connection in the connectio ID column when getting the html document? i saw most of the biggest sites does not show the connection ID
I believe Connection is of use only for HTTP2 (h2), not for HTTP1.1 nor HTTP3.
Do one on spotify !!
Why don't Instagram optimize? They got thousands of super powered software engineers!!
what's the reason for using VPN?
Perhaps because he does not want to show his own IP address? Idk
At around 7:47 he mentions that they calculate the profile's followers/following/posts numbers in the HTML. How exactly does this happen? Does it make a big difference in a page's performance?
It is probably a serverside rendered response, which will generate the html with the dynamic data on the server.
Server side calculations/lookups.
Is it right as a security wise, to expose the graphql endpoint in public, regardless the precautions taken to secure it ?
If it were not exposed there would be no use for Graphql and it has the same risks as a publicly facing REST api.
I think graphql is not meant to be exposed publicly, there are vulnerabilities and risks of doing so. There are articles taking about this subject though.
Facebook literally built it for their publicly facing APIs. All the articles talk about how you can write vulnerable code in graphql just like you can write vulnerable code in any other publicly facing application.
Save 100ms for typed in urls ... and mess with all relative urls? Not so sure how dumb it is
Can you please do Etsy next?
You're going to love their dev tools (and especially if you do it without adblock ad get exposed to Granify (A/B testing company). The Granify server gets pinged once a second on every Etsy page. Wild stuff (try with new sessionsuntil you get bucketed into a Granify group)
What is the alternative to an ORM? I‘m relatively new to backend development
This really depends on the language and db you are using. For example if I were looking for a postgres library for Javascript I would start by looking on the postgres website and then move on to Google after that. But I would warn you about security risks you should watch out for when using raw sql like injection
@@crustydev5561 my thoughts exactly, I mean isn’t that the whole point for an ORM? I wonder what one might have to implement to go the other route and be safe of sql injections
Those libraries usually have sanitization options but it would be hard if you didn't know about those pitfalls. My own opinion is that ORMs are helpful to buy you time until you can learn more about databases and sql but that's just me.
@@andylib actually no. ORMs do several different things.
1. SQL generation. It generated queries. A quarry builder alone does this.
2. Object Mapping. It maps the query results to your defined types in code
3. Migrations. Automatically generates updates to the Schema based on defined models.
@@nathanhedglin931 thank you for the explanation 🙏🏻 it helps to break it down