The Dangers Of Promise.all()
ฝัง
- เผยแพร่เมื่อ 30 ก.ย. 2024
- Please just use Promise.allSettled()
Keywords JAVASCRIPT JS TS TYPESCRIPT ASYNC SYNC ASYNCHRONOUS SYNCHRONOUS PROMISES
ALL MY VIDEOS ARE POSTED EARLY ON PATREON / t3dotgg
Everything else (Twitch, Twitter, Discord & my blog): t3.gg/links
![ผัวเถื่อน - ยูริ โตเกียวมิวสิค [ OFFICIAL MUSIC VIDEO ]](http://i.ytimg.com/vi/P08ORRVDGl8/mqdefault.jpg)
This video left me fulfilled. Thanks Theo
replies[0].content
@@meqativreplies?.at(-1)?.content
@@meqativ replies undefined
@@johnddonnet5151
const replies = [
{
user: "meqativ",
content: "replies[0].content",
},
{
deleted: true,
},
{
user: "johnddonnet5151",
content: "replies undefined",
mentions: { user: "meqativ", short: null, },
},
{
user: "meqativ",
content: [Circular *1],
mentions: { user: "johnddonnet5151", short: null, },
},
];
replies.at(-1).content
@@johnddonnet5151 if (Array.isArray(replies))
Promise.all is good for doing multiple queries with no side effects where you need all to succeed before the next step. You could do all settled and add retry logic but I’d rather have it in the work function or have the whole thing retried instead
Exactly. Devs just worry about BS these days 😂
wouldn't it be better to start a transaction instead
@@kartashuvit4971 that question is a bit orthogonal to how you handle promises. If a transaction would prevent significant bugs then yes, otherwise you get better performance committing a single statement at a time.
sounds to me this video is needed because people don't read the documentation thoroughly enough
@awmy3109 I think it's that they less worry about bullshit and more and more that the majority use case doesn't require a surgical solution like this.
If you have some really slow process that runs many times (let's say some api call that costs for time used), it would make sense to try and cache the successful results and retry the failures, because it could save a lot of time and money in the long-term.
For loading 1kb of json from your own api for a component to render "hello [user]", yeah that can be retried wholesale.
if you use all instead of allSettled, this video would be over at 0:10 because that's when the first Exception is thrown
Haha
You should have a function that catches that error, do whatever you want with it, null, undefined or whatever. Then it wouldn’t be a problem?
Also interested to know
I’m glad you got this all settled, thanks Theo 😉
4:09 Bruh, that's what Exclude is for. The best thing about Exclude is that it will exclude anything that EXTENDS the second type param... so you can exclude a whole object type from a discriminated union by specifying only the discriminating property. You don't have to know the types, just the difference.
didnt knew about this
nice one
Could you give an example of how you use this?
@@kevinmitchell6141 If he doesn't, ChatGPT probably does
nice tip thanks
To be fair the "^" operator being XOR and not exponent is common to most languages? As for the topic at hand, I'm guilty. I use Promise.all all the time.
I believe it's the standard. Most languages have something like Math.pow() or pow() instead of **.
Seems like another example of how returning errors results in better error handling than throwing errors.
Throwing is more OOP, returning is more functional approach. TS supports both, so it all depends how and when you use it.
@@alexandrucaraus233 Nothing about throwing or returning is inherently OOP or functional. Throwing is just the model most commonly used by Java, which happens to be the most widely taught OOP language. In an alternate universe, returning errors or result types could have been the standard way of doing things in Java. OOP != throwing.
That's why I'm slowly falling in love with Go
@@bryanleebmy yes, java, c#, c++ and all other other OOP languages use more the throwing approach, and all other pure functional prefer returning values. So blue!=green, like I said :D.
Said the Go developer, lol
“I hate JavaScript” ~ Theo, Aug 2023 1:34
I usually use .all and catch in the entry promises and handle any errors and cleanup there and return or push the errors to an array (I find this nicer and more flexible than allSettled). The final .all then only serves to send off /format the errors if needed, or to take the final decision on what to do, not handle the errors themselves.
I love this type of content where you share new discoveries about a language. Please make more videos like this.
Nothing dangerous about promise.all if you know what it does (You might argue with me because I said "if you know what it does" but I say that because the function itself is not dangerous, it's the misuse of the function that is dangerous). Promise.all requires all functions to resolve otherwise nothing is returned (which does resolve in discarded work being done). Promise.allSettled is a good alternative though but if the requirement is that all promises MUST resolve (and you don't have a retry mechanism which could potentially also cause other complexities because endless retries would cause your program to hang) then promise.all is the preffered choice.
summoning ts wizard matt to make a video on this type
Great video, but for the most part, I want Promise.all to reject if a single underlying promise rejects. For instance, I was recently writing some code that split PDFs into chunks for search indexing. If one single page cannot be chunked, I don't want to store incomplete data, I want to show a message like "example.pdf failed on page X." Promise.all is essentially like a database transaction in that way, and very convenient for that purpose (though I agree, allSettled has many great use cases).
Do you have a strategy for cancelling other tasks when one fails?
@@xinaesthetic like if you had promises foo and bar, and you want to init them at the same time, then cancel bar if foo fails? Personally, I wouldn't, I'd just let bar finish.
@@MKorostoff fair enough. No point coding it when you don't need it. In a situation where it did matter I suppose I'd probably have some shared reference to a bit of state that flags if the operation should be cancelled, but not sure what particular patterns there are that might be good to know.
Maybe this is an anti-pattern but what if you catch inside the promises? If there's an error return null, otherwise return the result. That way the promises always resolve and we can filter out results after
I am more into observables by now. There I have multiple choices to handle multi asyncs and it feels easier.
It was super hard not to notice and resist commenting, but why is waitFor declared as async? Seems kinda extra to wrap the return value when you're already returning a promise? Obviously, js handles it behind the scenes gracefully, but still, there's no await used inside.
I actually just used promise.all in a project that to me was appropriate. Essentially I'm running two async functions in parallel that return an array of values. Once I get back both arrays I compare the two arrays and keep the values that are shared between the two. If one promise rejects (which only happens because they found no values to return in their array) I don't need to know what the returned values in the other async function will be because I know there will be no matching values for me to pull. So the fact that it jumps to the catch statement, and I don't get the values for the other promises, all work out.
1:06 -> the carrot operator `^` is bitwise OR in JavaScript. You probably wanted Math.pow().
nvm, you caught it.
Yes, the error behavior isn't too inferable. Once, instead of reading the docs, I wrote my own makeConsecutively function. Which was fine.
New Title = I used the wrong method for what I really want.. but what I want is not what 95% of everyone else wants.
I don't get it. I'm still awaiting the catch. Let me know when this is all settled.
All is settled, nothing to catch, go home :D.
Well kinda made up problem. If this is your requirements then you might not need promise all after all.
Promise.all is good when youre using puppeteer. Great video!
It is easier to check if the item of the result has value as key, then you know that the status is fulfilled
How might our understanding and application of asynchronous functions change if we were to prioritize clarity and predictability of outcomes over simplicity or convention in our coding practices?
If people are interested for the partial success result from the promises, you can wrap the async call with a function which have a catch block inside. Then you may still get the results even some are failed.
I have this issue in some production apps that were not written by me that this will fix. Theo you are a godsend!
The type issue couldn't be solved by using Exclude?
If you use a for loop instead of a filter, you'll get the correct typing
I haven't finished the video yet, but I'm going to guess most of the time taking a less FP approach and using "for...in" within an already async function is the recommendation coming up, rather than managing the nesty confusion of Promise.all
I was so wrong and learned something useful.
4:54 - When seeing `.filter` followed by `.map`, using `.flatMap` will usually result in a cleaner code: `results.flatMap(r => r.status === 'fulfilled' ? [r.value] : [])`
When you use flatMap, filtering and mapping is done in a single step, so there is no loss of type information between these 2 steps.
True, but transduce also fixes this problem without introducing intermediate arrays, which will use more memory and power to merge.
@@NickServ which transducers library do you recommend for JS?
Because JavaScript Promises are fucking eager.
thank god typescript fixed this filter issue
Sorry did you say Exception? I think you mean Error ;)
The filter issue was solved with TS5.5 🎉
This behavior makes me like RxJS.
thanks for letting me know how bad js is
Basically if your requirement is to process a bunch of steps sequentially, which means one after the other, please forget Promise.all because it process everything CONCURRENTLY.
I can't reject this one. God bless
You could also implement error catching in your asynchronous function (e.g., WORK) and have it return the potentially caught error. Then you can filter the returned array by instanceof Error
This, I don’t know what’s wrong with the limited amount of creativity in devs these days.
Lol, Promise.all(promises.map(p=>p.catch(e=>e)))
Can I haz utoob job plz?
I've been awaiting this video
The typing on the settled promise is so gross 😭
JS error handling in general sucks IMHO, especially with promises. Even with typescript you have no way of knowing if a function could throw and what it would throw, I hate it
I use it with something that's not an array. When you request multiple things, but definitely need all of it. An array api in that situation is kinda bad, I even defined a dictionary version of Promise.all, where you give names to each entry.
Sorry for the *chain* of puns...
Don’t worry, the Promise puns will never not be funny… although I’m sure some people will… await the day when this isn’t true.
Use await in a for-in loop(you can indeed!) to do them sequentially and promise.all for all at once
Well not all is Settled in JS async yet
Thank you very helpful.
Thanks, I didn't know that allSettled existed.
Before this video, I only used Promise.All like this:
The function WORK would be an async func with a try/catch so, if the promise got resolved would returned the value if not returned a null. After that filter the array of nulls.
promises is sucks because they are not lazy and not cancellable
and this might leads to unexpected results, data-race in async useEffect in react for example
Thanks F#
I'm just here for the puns.
promise.All() seems more reasonable in a serverless backend env during boot / initialization (eg. AWS SSM)
Is it fine to do promise.all with tanstack query mutateAsync? Say I need to make 5 separate independent mutations on button click.
If I need all results to succeed, and I don't want to keep it running on failed as .all does, I just use an async for loop - works wonders ;)
Yavascript folks…
Hail allsettled() !!!!!
honestly this is the best argument I've seen for a Result type, and it wasn't even intended.
Who is this video for, who immediately started writing on React without learning JS basics?
oh no... dont do this to me theo
I usually use a library like p-all or p-limit to also ensure promise all runs with a specified amount of concurrency.
I prefer to just retry everything. That's the reason why I put them all together in one promise in the first place.
I just keep thinking of the song "Promises" by Fugazi...
its 'all or nothing'
wdym hard to work with async stuff in js? have you tried to do this in python?
The type issue is actually easy to solve
npm uninstall -g typescript
if u think async is hard in javascript. well.. it's freaking pain in other languages, even just Python it's quite a nightmare compared to Javascript.
python is a pain as a whole
another gem
Isn't the .filter and .value issue one of those things that ts-reset fixes?
Thank you
"Don't make promises you can't keep" seems to be the moral here
Futures > Promises because polling can be halted
Great
I use it when building my test environment on the backend. I don't really care if the server or database failed to standup, if either of them fails then its pointless to run the test suite. Same with performing a bunch of bulkCreates with sequelize. If one of the bulkCreates fails then the database is corrupted anyway and I'm going to have to find out whats wrong and fix it regardless so it doesn't matter if the promises still run in the background or send back results.
The bulk case if you work with a bunch of operations of that nature you might use a Sagas pattern and revert/delete the corrupted data and retry it in some cases, but yeah I get what you mean
Damn 😮
splendid moustache, theo 🤝🏼
Promise ALL is the best for select queries
I only used promise.all when theres a dependancy chain, a relies on b relies on c. If A failed we dont want to continue. Otherwise I agree settled for everything else.
Wouldn’t you need to await in sequence if they are dependent?
@@t3dotgg Right my example wasnt clear, they would rely on eachother or another dependant requires all 3. Not sequentially. not in fact a proper "chain" more like a web
I find using a promise pool is a good balance. They usually return results and errors as separate objects, and have other features such as setting concurrency
Use case: Start up app requirements being fetched in calls. If any fails, we need to start all over again.
Wouldn't it be nice to know WHICH failed?
@@t3dotgg You can add catch handlers to the individual promises and benefit from the short-circuiting behavior.
@@t3dotgg not really, I can't see that on network debugging. It's all strictly required so we start all over again.
Theo is close to embracing a properly typed language!
"JS Dev discovers Result and/or Option" is my favorite genre of programming video
Wow, this is actually an issue in one of my apps and I wasn't sure what the problem was. Ty Theo.
Great video 😍, Thanks
But I heard never settle
solid vid, good stuff.
That is unsettling
Another thing is that the Promise concurrency functions are only useful for IO / network related tasks, since the requests are in flight in parallel. However, if you just have a very expensive task computationally, splitting it up and Promise.alling it wont speed it up at all, since JS is single-threaded and it will only ever work on one task at a time. If, however, you used multiple threads operating on multiple cores, then it would make sense.
You kind always turn computation heavy task into io with webWorker. It just requires a lot more setup (create worker, send relevant data, await response from worker)
This is a bit of an oversimplification, since while it can result in performance issues compared to threads, single-thread concurrency with delaying Promises until next tick can still improve performance in many situations.
@@NickServ for example?
@@NickServ do you mean performance (how fast your code run end to end) or reactivity (how long your code take to react to an event)?
If reactivity, I agree. If performance, if you have say 1000 lines of code to run to complete a function, splitting it into 10 group of 100 lines won't make things end faster.
@@aredrih6723 Both. Promise concurrency still helps with e2e wall time, especially when getting results early with all/race/any.
Just realized one of my production bugs might be due to EXACTLY this. brb checking
I agree on the correctness side but disagree on the efficiency side. If you need _all_ request to complete to perform a task, exiting early is more efficient.
If you want some retry logic, placing it in the catch handler of each individual promise would allow you to retry immediately instead of having to wait on the slowest to resolve first.
Also, isn't that be the use case for AbortController ? You tie every request to a signal and as soon as the controller abort, you interrupt as early as possible because the result is no longer required.
That said, I remember node logging a warning for memory leak for trying to have more than 10 listener to a signal...
There are cases where all you care about is that the promises are done, not that they succeeded.
I have exactly such a case in a current project: It does a few async operations in parallel, all of which must resolve. In case of an error, the shared signal cancels the rest. Works very well since we do not need the individual results in case of an error - and an early result is more valuable than having all error messages should more than one fail. But that's clearly very situation dependent.
@@CottidaeSEA catch(() => {}) is always an option if you don't care what kind of error each promise encounters.
If you're looking for stat on which action succeeded and which one failed, then yeah, allSettled might fit better.
Also, at the end of the day, allSettled is just a promise.all with a well chosen then applied to each promise (e.g. .then((e) => ({e, ok:true}), (e) => ({e, ok:false})))
@@aredrih6723 That is not compatible with await syntax, but yes, that works. Or you could just use allSettled and go on with life.
Name that vs code theme please
poimandres dark theme
thanks fr@@calledtigermttv7184
Interesting video.
As useful as it is, JavaScript is such a horrible language for inconsistent rules and syntax.
Very cool video!
For interacting with library code is is a good idea. Thanks for pointing it out. In my own code I don’t use exceptions. So promise.all works fine
Great video, Promise me you leave the puns out next time
There’s an opportunity to make all these suggestions accessible. Something a kin to ESLint warnings / recommendations.
I’m imagining a plugin that shows brief descriptions of “avoid this” with some if “logical/situation”.
Then we the community can add their own, vote, attach TH-cam videos.
So if you write Promise.all you see this video / a code example.
Love it
@Theo telemetry is a common place for me. I usually push up data in a fire and forget manner and for some metrics (granted, these are not actually all that many) it's better to use all instead of allSettled just because it allows the publishing function to return faster, reducing the time the function has to run. Those promises in the background can fail and it'll just be treated as a missing data point, which for a lot of metric types is actually not critical.
How do you know whether or not you have systemic errors? If you have common errors, your data risks being biased.
@@NuncNuncNuncNunc it depends on the metric being sent. Something like network tx and rx for example it's okay if publishing fails once every so often. A user may also select a metric to be non essential and that would be treated the same way. Of course, if every publish fails then that's a different matter that we do look out for, though all we do in that case is log to console, as we can't be sure whether the problem is ours or if the user has something in their config or something like a proxy
Isn't the ^ (caret symbol) bitwise XOR and not exponentiation? From what I can remember ** would be exponentiation.
he says that in the video
I Promise to you Theo that this video is going to be resolved for the all-settled users.
Thats why you choose a language that has a good error handling
And good typing
I still have nightmares about exceptions in Java