Interesting one. Thank yous Speaking of 2016, I just watched your talk Jake (Instant Loading: Building offline-first Progressive Web Apps - Google I/O 2016) Pretty cool presentation with very insightful material 👍 (even in 2020).
Has just watched this video over and over again these several days,it really fills in the emptiness of my knowledge of streaming in web and is so amazing,appreciate it！ 👍
I could see how you could use this to stream from a file input through some kind of gzip compression transformer to an upload request, all without storing the whole file in memory!
You guys are saviors - I spend hours looking for wasm zlib and I was also expecting your new video to come up sometime soon as well but didn't expect this same video to solve my problem OMG. (talking about compression streams of-course)
You could solve the redirect issue by also caching the stream for the first few seconds. If the server responds with a 3xx quickly enough you can pump the cached stream data again to the new endpoint. If no 3xx was received within the first few seconds then dump the cache but continue sending the streamed data If the server waits too long to send the 3xx, well tough luck.
Jake and Surma in this talk you didn't mention that chrome actually limits concurrent requests to a host at 6, meaning try streaming more than 6 real-time values across transfer-encoding: chunked and watch the browser just leave all your future requests on "pending".
I've missed you guys! I wonder if there's actually any advantage to doing it this way over websockets. I'd imagine websockets have less overhead too? Seems fun to try though.
You didn't get into how to feature-detect this. If anyone is going to use this, that should be one of the first things that come to mind ;-). But yeah, this would be really cool for websockets-like use: when the user focuses, already show "User is typing..." to others; notifications when someone reads a message (because there might be multiple messages). Most of this though, would be to skip the "authentication" part of the HTTP request, as a "warmup". The more interesting use-case would be streaming large files. We currently split up the files in chunks before sending it off to the server, and then patching it together again. It would be cool to just stream the file manually. Why manually? That way you can make resumable uploads. Disconnected? The server can tell you where to pick it back up and then stream those parts. Too bad origin-trials are limited by hostnames. We serve users' content on their own hostname, but might still like to make origin-trial features available.
`.pipeThrough()` takes a transform and returns a new `ReadableStream`. `pipeTo()` takes a writable and returns a Promise that resolves when the stream has finished.
We film the head shots at 24fps (consistency with film, although I'd rather it was 30 or 60), but the 'slides' are 60fps, which is why the animations & screencasts look nice and smooth. IMO, code and browser screencasts look unnatural at anything lower than 60fps. My code editor and browser run at 60fps.
One question: is fetch api suited for uploading videos, to be clear would it make sense to use fetch api for an app like tiktok or is the XMLHttpRequest() better suited for such an app.
XMLHttpRequest isn't better than fetch for these cases. Assuming the video is already created (not live), then you'll want to create some kind of system that allows you to resume the upload. Background fetch is a great way to do this in Chrome, but for a more cross-browser solution, I'd cut the video file up into chunks (blob.slice should be good enough) and upload them one by one, and stitch them together on the server. This means you can 'resume' uploads (there isn't a web standard for resuming uploads).
@@jakearchibald Thank you very much for the answer...if you don't mind answering two other questions, I would be very glad...what exactly should I reconsider when live should be also possible and the second question is, if the proposed solution of your with cutting the video file up into chunks is also a good way handling a video upload for mobile applications?
@@liramoney9565 for live, WebRTC is probably a better system. I think you should have the same system for uploading on web as you do on mobile, so if you go with chunking for one, do it on both. The other way to do it is to send it as one file, but if it gets disrupted, you need a way for the server to signal how many bytes it received, then resume sending from that byte.
10:20 "Not all browsers have transform streams" And these are..? Please make a list of relevant non-chromium based browser. Also, sing this opportunity, press F for Mozilla, they had a good run.
I find function callbacks more clean then coroutines. ... what if there is an error mid-way ? will the loop ever finish ? Or will there be a silent error ? vs stream.addEventListener("data", function read(chunk) {...}); stream.addEventListener("error", function handleErr(err) {...}); Philosophically; what will be the consequence of an (unhandled) error/exception in a "walkCrossTheStreet" function ? (ex: a car drives by). If you cover all await's with try/catch it will not look clean. Is our goal as engineers to write beautiful and terse code, or to solve actual problems?
"This not gonna turn into a framewok." Three new js streaming lib's were made while you said that.
"I'm getting into very difficult legal territory now"
Luckily the name of this show is pretty much a legal disclaimer in itself!
Interesting one. Thank yous
Speaking of 2016, I just watched your talk Jake (Instant Loading: Building offline-first Progressive Web Apps - Google I/O 2016)
Pretty cool presentation with very insightful material 👍 (even in 2020).
Has just watched this video over and over again these several days,it really fills in the emptiness of my knowledge of streaming in web and is so amazing,appreciate it！ 👍
First thing that comes to mind: using this for large file uploads.
I could see how you could use this to stream from a file input through some kind of gzip compression transformer to an upload request, all without storing the whole file in memory!
I have seldom considered the essential difference between web and native land.It really enlightens me in a flash when saying web is streaming
Paul! gone but not forgotten hahahaha
You guys are saviors - I spend hours looking for wasm zlib and I was also expecting your new video to come up sometime soon as well but didn't expect this same video to solve my problem OMG. (talking about compression streams of-course)
I can't imagine the work they put to have a decent background 😂
Zero effort from me, which is why it looks bad. I want to do an episode from my shed.
Paul was always the funniest and most entertaining to watch! Hope to see him make more video for Google Chrome
17:10 Hear me out, validate data server-side but display instant errors like it's client side
Great idea
You could solve the redirect issue by also caching the stream for the first few seconds. If the server responds with a 3xx quickly enough you can pump the cached stream data again to the new endpoint.
If no 3xx was received within the first few seconds then dump the cache but continue sending the streamed data
If the server waits too long to send the 3xx, well tough luck.
Defining the time until it gives up is hard
Like how you're using a switch controller as a slide remote 😆
Web Gamepad API ;)
he previously used a wii mote, so it is a strict upgrade
Never go away, http 203
Amazing! Could be really interesting to us ;) Will report if something will break (fingers crossed).
In light of recent events, you should do an episode on input masking 😅
Jake and Surma in this talk you didn't mention that chrome actually limits concurrent requests to a host at 6, meaning try streaming more than 6 real-time values across transfer-encoding: chunked and watch the browser just leave all your future requests on "pending".
That is only true for HTTP/1. Over HTTP/2 (the standard nowadays) you can have as many parallel requests as you like.
sorry I'm just wondering why is the console.log() allowed after 'break' ?
what constructor helps us to read video data chuck by chunk plz help if anyone reading this
The 'sockets' demo doesn't work in Chrome Version 97.0.4692.99 (Official Build) (64-bit). Am I right?
Have these guys done an episode about what gear they use?
I've missed you guys! I wonder if there's actually any advantage to doing it this way over websockets. I'd imagine websockets have less overhead too? Seems fun to try though.
Difficult to tell, but a websocket would require another connection
What happened to Paul Lewis?
He’s gone, but not forgotten.
Please share the link of the encoding episode of which you talked about at the starting of the video.
It was on the podcast http203.libsyn.com/how-to-avoid-getting-utfd-by-text-encodings
this sounds exactly like web sockets
You guys did a video on encoding? Can someone point me to it? Searched, couldn’t find ...
It was a podcast episode!
I'm still struggling with async await I'm not sure if I'm ready for this 😂
Keep writing console.log everywhere. That's how I learnt 😅
@@ericaskari console.log(1), console.log(2)..3. Everywhere and then see the order they get printed 😅
In case it helps: developers.google.com/web/fundamentals/primers/async-functions
@@jakearchibaldThanks! It was a nice read. I'm definitely guilty of going "too sequential" at times 😅
also liked the shots at the "smart" code.
You didn't get into how to feature-detect this. If anyone is going to use this, that should be one of the first things that come to mind ;-).
But yeah, this would be really cool for websockets-like use: when the user focuses, already show "User is typing..." to others; notifications when someone reads a message (because there might be multiple messages). Most of this though, would be to skip the "authentication" part of the HTTP request, as a "warmup".
The more interesting use-case would be streaming large files. We currently split up the files in chunks before sending it off to the server, and then patching it together again. It would be cool to just stream the file manually. Why manually? That way you can make resumable uploads. Disconnected? The server can tell you where to pick it back up and then stream those parts.
Too bad origin-trials are limited by hostnames. We serve users' content on their own hostname, but might still like to make origin-trial features available.
Feature detect here: web.dev/fetch-upload-streaming/#feature-detection
Why are there functions ".pipeThrough()" and ".pipeTo()" and not just ".pipe()"?
`.pipeThrough()` takes a transform and returns a new `ReadableStream`. `pipeTo()` takes a writable and returns a Promise that resolves when the stream has finished.
Yep! It's actually kinda weird that Node overloads these into the same function.
@@jakearchibald you know if Deno accept a streams in a fetch requisition?
why Chrome play HTTP live streaming with a huge delay when loading? on Firefox HTTP streams play instantly. WHY? How Chrome handle the HTTP streaming?
I'm a curious web developer. But can't really figure out why this video is in 60fps? 🤔
We film the head shots at 24fps (consistency with film, although I'd rather it was 30 or 60), but the 'slides' are 60fps, which is why the animations & screencasts look nice and smooth. IMO, code and browser screencasts look unnatural at anything lower than 60fps. My code editor and browser run at 60fps.
Couldn't get this to work in 2023: I always receive a TypeError: Failed to fetch. Anyone able to help?
i hope some framerok use this advantages of native http
example of ReadableStream doesn't seems to be right. Passing invalid object literal with key = start is missing colon.
Pretty sure it's right. developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Method_definitions
One question: is fetch api suited for uploading videos, to be clear would it make sense to use fetch api for an app like tiktok or is the XMLHttpRequest() better suited for such an app.
XMLHttpRequest isn't better than fetch for these cases. Assuming the video is already created (not live), then you'll want to create some kind of system that allows you to resume the upload. Background fetch is a great way to do this in Chrome, but for a more cross-browser solution, I'd cut the video file up into chunks (blob.slice should be good enough) and upload them one by one, and stitch them together on the server. This means you can 'resume' uploads (there isn't a web standard for resuming uploads).
@@jakearchibald Thank you very much for the answer...if you don't mind answering two other questions, I would be very glad...what exactly should I reconsider when live should be also possible and the second question is, if the proposed solution of your with cutting the video file up into chunks is also a good way handling a video upload for mobile applications?
@@liramoney9565 for live, WebRTC is probably a better system. I think you should have the same system for uploading on web as you do on mobile, so if you go with chunking for one, do it on both. The other way to do it is to send it as one file, but if it gets disrupted, you need a way for the server to signal how many bytes it received, then resume sending from that byte.
With Http3 in the mix, Would fetch use it or still fallback to h2?
Fetch uses the browser's network stack, so if the browser supports H3 then fetch will use H3
perfect funky intro :D
It's how we synchronise the recordings 😀
Framework rant please ;)
This was really interesting, I want to try it out but I have to get through with a few projects.
I also want to try it out but have to get through a few proxy servers 😏
Stop eating our RAM.☺️😂😀
The original PS3.. Nice!
nice
10:20
"Not all browsers have transform streams"
And these are..? Please make a list of relevant non-chromium based browser. Also, sing this opportunity, press F for Mozilla, they had a good run.
PHP has streaming since 4.3 if I'm not mistaken
Maybe even earlier than that. But this video isn't about server-to-client streaming, it's about client-to-server streaming.
I find function callbacks more clean then coroutines. ... what if there is an error mid-way ? will the loop ever finish ? Or will there be a silent error ? vs stream.addEventListener("data", function read(chunk) {...}); stream.addEventListener("error", function handleErr(err) {...}); Philosophically; what will be the consequence of an (unhandled) error/exception in a "walkCrossTheStreet" function ? (ex: a car drives by). If you cover all await's with try/catch it will not look clean. Is our goal as engineers to write beautiful and terse code, or to solve actual problems?
nice
No you