So what makes your tr (or even better, grep) "parallel and distributed" is not that it somehow runs N greps per location to utilize multiple cores and whatnot, but just the fact there are as many individual grep processes running as there are objects to be parsed (307 in this case)? But what if your log file _is_ that mythical 700GB haproxy access log that is slow to copy back to you? Are such large files chunked into numerous "objects" residing on on many different ZFS servers when they are initially being sent to the object store?
I think what I’m confused about here is where the compute is running. Like, are you not spinning up a jail on the same OS that is hosting your ZFS volume? Isn’t your storage server going to have different hardware characteristics that lend itself better to storage than to compute?
10 years later, still loving how comfortable Bryan feels being so inappropriate 🤣all the while delivering such amazing technological solutions!
I love Bryan Cantrill dearly
Brilliant speaker. Very compelling. Great talk!
great talk, loved it.
This guy is a wizard
Great talk -- and funny too
So what makes your tr (or even better, grep) "parallel and distributed" is not that it somehow runs N greps per location to utilize multiple cores and whatnot, but just the fact there are as many individual grep processes running as there are objects to be parsed (307 in this case)? But what if your log file _is_ that mythical 700GB haproxy access log that is slow to copy back to you? Are such large files chunked into numerous "objects" residing on on many different ZFS servers when they are initially being sent to the object store?
yes.. ! bryan is a brilliant speaker
I think what I’m confused about here is where the compute is running. Like, are you not spinning up a jail on the same OS that is hosting your ZFS volume? Isn’t your storage server going to have different hardware characteristics that lend itself better to storage than to compute?
Damn hilarious :)
"Goes 19th Century on my ass" :D :D :D