Everyone's Racing To Replace Redis - Who Will Win?
ฝัง
- เผยแพร่เมื่อ 3 เม.ย. 2024
- Valkey, KeyDB, and more are quickly gaining traction after the Redis license swap. Excited to see where things end up!
THANK YOU LWN FOR LETTING ME USE YOUR BLOG POST. I subscribed right after and it's been awesome, highly recommend subscribing (it's cheap as hell too!)
SOURCES
lwn.net/Articles/966631/
www.linuxfoundation.org/press...
news.ycombinator.com/item?id=...
Check out my Twitch, Twitter, Discord more at t3.gg
S/O Ph4se0n3 for the awesome edit 🙏 - วิทยาศาสตร์และเทคโนโลยี
HUGE shoutout to Joe Brockmeier and LWN for letting me use their article, highly recommend checking them out lwn.net/
Glad to see you were able to get it cleared without issue - was a bit of a scare on the livestream. Sad to see that they needed to expand the validation on account creation after recent attacks.
@@Kane0123 sorry for my ignorance but what are you talking about?
LWN is top-notch
Back to raw dogging my ConcurrentHashMap singleton
We all need to have a personal library with all our "temporary" solutions
@@marcosdlyI just made one, with the help of Chat Gippity, in Golang while watching this video. I know I'm oversimplifying it, and maybe it's the Dunning-Kruger Effect, but how difficult is it to make a simple in-memory key-value store? I know for more complexity and fine tweaking you'd want something like Redis, but come on
@@marcosdlyI have an npm package called `@samual/lib`
@@redpillsatori3020 That is a slippery slope my friend lol, I feel you though. Most small scale use cases you'll be fine.
@@redpillsatori3020Redis is not just a in memory KV store. Add durability, reliability, a lot of other operations redis does (sets, maps, queues, etc) and it gets really complex really fast. 😢
If you want to know why those "weird Github alternatives" exist. There is a whole story behind that too. Gogs maintainer does something that pisses people off, that causes a fork called Gitea. That goes well for a while until they piss people off, which causes another fork called Forgejo. Forgejo is a fork maintained by Codeberg.
Soon after that it became Codeberg which instead used Gitea, and in 2022 it forked Gitea into Forgejo as a response to the maintainer of Gitea forming a for-profit company behind it without much (if any?) communication with the community, transferring all trademarks to it and going against former promises made.
Codeberg is funded by donations and the primary goal is FLOSS and free/libre content. Codeberg the organization is more than "just a fork of Gitea". I encourage anybody with an interest in these topics to consider Codeberg over GitHub.
Codeberg is an ngo and a non-profit collaborative organization driven by individuals, which came to be as a response to the Microsoft acquisition of GitHub back in 2018. In the beginning it started as a project named TeaHub which was a fork of Gitea.
Genuinely interesting
@@lh8228 orrrr we could just use github, and use those donations to fund devs instead of going "grrr Microsoft" lmao
-Sent from my Microsoft Edge
@32:52 _"What's with all the weird GitHub alternatives in the Linux community, Jesus Christ?"_
No, that's totally bass-ackwards. The correct question would be: What's with the weird idea that everything needs to be on GitHub?!?
Amen!
Yup, hard L take.
@@DanWalshTV : "L take"?
To say that people that don't host their code on github don't want to be part of the open source eco system is such a bad take :/ (32:50)
How dare they not obey the Microsoft overlords
It would be a bad take, but... He literally didn't make that take. All he said was that the alternatives are "weird", not that using them means one doesn't want to participate in the OSS ecosystem. How the hell did you make such a leap in logic?
@@G-u-z-i-o"they just don't wanna be in the open-source ecosystem"
"Oh no, Redis is no longer open source, I must switch to a fork" - said a developer probably using Mac OS, deploying on AWS/Azure/"the cloud", using Visual Studio Code and Chrome.
...fair point
I don't get this comment
How did you know!
Smart developer tbf. He understands that redis in "the cloud" will cost more in the future
It's Ark instead of Chrome, but yea hahahah
Redis team not being good enough to scare Amazon off Redis but Vercel being good enough to scare Amazon off NextJS is an absolutely horrific take.
The linux foundation is NOT the linux kernel. The linux kernel doesn't use Redis....
Yeah no shit Sherlock
Did you also know that 2 + 2 is 4?
@@vaayuo Theo _heavily_ implied that the kernel, or at least core GNU software, relies on Redis.
"They're very strict about what the do and don't allow to be supported on a kernel level [...] Linux cannot have something like Redis as a core dependency [...] as permissive as the Linux open-source license"
@ Did you also know that there are 60 seconds in 1 minute??
@@Debianz your communication skills are impeccable 🙄
Did you just say that Redis is a core component of Linux? I don't think that's true.
2:00 - What does "Linux has Redis as a core dependency" mean? I haven't used one single thing in Linux that required me to have Redis installed. I've used Redis myself for server backends, but I've never had an application require it. The Linux Foundation probably created this fork for the benefit of the open-source community.
I'm pretty sure the Linux kernel does not depend on redis and never has…
Also: you couldn’t fork Linux and put your own proprietary license on top, the GPL's copyleft clause prevents that.
The kernel does not depend on it but the license will prevent Redis from being included in the default repos of most distros, requiring users to add a 3rd-party repo. Would you go through the hurdle of adding a Redis PPA when you could just apt install valkey?
I was referencing 1:53 onwards. He just makes a lot of… strange statements there. Like how „Linux is very strict about what is supported at the kernel level“ and wants to remove redis as a „core dependency“.
Also, the Linux foundation has no say in what dependencies do and don’t include by default.
@@MustNotContainSpaces I get that. I went back to relisten, it was indeed strange. Apple fanboy that is allergic to self-hosting confusing everything about Linux hosting - what, I would have never guessed?!
@@spicynoodle7419 I think he's just getting caught up in his words, confusing Linux kernel with Linux distributions. It is amateurish for sure and surprising.
@@MustNotContainSpaces very, very nonsensical
The sad thing is, this strengthens companies like AWS in some ways. Like OpenTOFU where it's forked into a charitable organisation. This kind of preemptive licensing I think only works when you first start your project.
It's usually companies like AWS that hugely contribute back to projects that they use as well, since they need to squeeze every ounce of performance out of it. Small-mid sized businesses don't have the money to let their developers contribute back to OS repos on company time.
they only help themselves, never the people/company eho invented the tech @@rdvansloten
EDIT: oh lmao the cut-in. good guy theo
correction at 2:50:
BitKeeper, not Mercurial. Mercurial emerged around the same time as Git in 2005. Linux never used it.
Oh man, LOTS to correct about that statement; there's enough that it's probably best to pretend Theo didn't mention it. Here's the key details of what actually happened:
Bitmover ALWAYS charged for bitkeeper from day one. Torvalds was happy to pay and didn't care that it was proprietary. But not everyone liked it and it was a constant source of disagreement on the mailing list. Tridge reverse engineered the client to make an open source alternative and Bitmover pulled the plug themselves, kicking Linux off their platform because they didn't want Tridge's client to be a thing. Some people said they should switch to Subversion but nobody took the suggestion seriously because it wasn't even close to good enough. Mercurial (alongside a few other alternatives) was created specifically to fit the need by people who understood the problem, with the explicit intention for it to be picked up by the Linux kernel. But Torvalds had opinions about what he wanted and made his own system himself. It wasn't really "better", it's just that Linus wanted something specific and he decided to do it himself. A few of the other hopeful alternatives abandoned their project after Git was picked by the kernel, but Mercurial stayed around, saying their stuff was good enough to stand on its own as an alternative to Git for anyone who wanted it.
C# here we go, bro haven't yet realized how fast is C# lol
Shh let him live in Javascript land.
@@MrC0MPUT3R ok 💀
Not faster than deez nuts
I discovered your channel just recently - really good content, love the in-depth stories about very niche topics like this one :) Keep on the good work!
1 week after Redis license changed announcement, C gigachad already making their own version of Redis.
there's also Redict by Drew DeVault
~ 2:50 - "Mecurial is git before git" - Actually, both git and mecurial were initially started within days of each other.
Why would AWS do anything just because of pressure from a twitter thread? Not saying I agree with the licence change but the idea that any of these corporations are benevolent creatures that will see the error of their ways when they are pointed out is an absurd thought.
Thanks for all the details in your video. Good the see a healthy alternative (Keyval) coming soon with most Linux distributions.
Instead of just saying _"We're ending open-source"_ I feel like it would make more sense to do a funding drive, and say _"This is how much we need to raise in order to remain sustainably open-source."_
Yeah, a lot of FOSS need to start following in the steps of Thunderbird.
It was BitKeeper before Git, not Mercurial. Both Mercurial and Git have been developed as alternatives to BitKeeper.
He corrected himself
Wish valkey will add multi-master and active replica feature like in keydb. We're moving from redis too, and actually looking into keydb as it allows simpler cluster configuration for us. But if Linux Foundation will create their own, we might favor them just because who they are 😂
Yep the call button is just such a game breaker. Even if I would like to pay for the solution. If there is a "get a quote" or call button I am instantly looking for something else.
Yeah.. you went off with that comment on VCSs, even after the correction. Apache subversion (SVN) is released under the Apache open source license. SVN is very much not dead (in a similar way PHP or COBOL is far from dead), but does not fit into the modern era of OSS projects (in particular the repo is centralised, not distributed). Bitkeeper is the one that died. Mercurial is just not popular.
I agree with a comment I read somewhere that it was really a missed opportunity to name Valkey as Valkeyrie.
I wished more people knew linux exists. Since they decided not to host their project on github, no one in the open source ecosystem know they exist.
technically there is an official mirror
@@phygs well, true. But I did not count it on the count of it being a very very very limited one way mirror of a very specific tree. it does not show or host every other trees and discussions that shows every development.
I find it hard to believe that someone who knows what "open source" is does not know "Linux" exists 😂
@@tomshieff well, based on Theo, since they don't use github for hosting their development, they are not participating in the opensource ecosystem.
@thephoenix-bf5qd but how can that be since Linux is not participating the open source ecosystem? I mean they are not in github! Isn't it impossible to participate in open source without being in closed source code forge, owned by Microsoft? Anything other than that sound inherently ridicoulous!
I'll also say in terms of open source vs proprietary is that proprietary software, especially the really enterprise ones, usually have a worse user experience in order to protect buisiness secrets. The licencing, installation, and artificial limitations imposed on you are really insulting because you're actually paying for it.
Thing is, the Linux Foundation is not the Linux community nor even Linux developers. Yes, the Linux Foundation helps with some things around Linux (like organizing events, employing certain developers and I believe funding the infrastructure). But its primary objective is serving its constituents - Intel, Microsoft, IBM, Oracle, Facebook and so on.
A lot of Linux projects host their own repos, I'm not sure why exactly, but it's probably a mix of GitHub not being OSS, privacy, and having more control over their own projects (no surprise bans/suspensions e.g. xz's maintainer and repo).
The ZFS thing does not really comply because that's a kernel module that can't be included in the Linux kernel because of a licensing conflict. Redis doesn't need a kernel module AFAIK. Linux distros don't have to stick to one license, it's just that they don't like to include non-free packages by default.
Note sure what's wrong with Garnet being written in C#/.Net (especially for Azure). It can be a really fast language/framework
Theo knows next to nothing about the current state of .NET, take his comment with a grain of salt.
It is, in fact, very fast... Faster than Redis and Dragonfly in a ton of benchmarks; lower latencies and etc.
It's written in C# but it uses A LOT of unsafe pointers and pinned memory buffers. It is however MUCH more approachable to many more people than a project written in C or C++. Source: I have PR merged on the project.
I'll probably keep an eye on Valkey but currently deciding between Dragonfly and Keydb.
I really wanted to use Dragonfly, but had too many issues trying to get it to work in Docker as a Redis replacement, where Keydb just worked immediately.
I made the switch as soon as the Redis news came out & I'm back to in-memory-db happiness 😅
@@everyhandletaken Hmm it looks straightforward in the docs but of course things aren't always as easy as docs make it seem. I'll probably just try to put both into testcontainers and see which gives me the most headache, as a starting point.
@@ChrysusTV yeah, I was really keen on Dragonfly & not even thought of Keydb, but in the end I am happy with what I ended up with & hopefully you will be too!
Really interesting video, especially the part about Redis history. This is even worse than I thought! I hope the Linux foundation Valkey alternative will pick up speed.
Check redict out. It has been more actively worked on and by a much more renowned developer as well.
This made me laugh so hard, thanks for sharing!
Thank you for clarifying the whole BSD sublicensing being OK. I was more than a bit concerned that your last video gave a platform for worries this wasn't the case. Though INAL MIT should allow the same. Otherwise anyone using shadcn ui in a commercial project is screwed.
you can store kv pairs on postgres with some minor mods looks into it
With snap onboard, this means keydb's performance enhancements suddenly feel likely to be folded into valkey. The king is dead, long live the king
Redict seems more in line with Foss than Valkey, and from what I understand it has a head start in development and compatibility. But I have my doubts it will maintain its momentum. Not sure where KeeDB stands. Guess we'll just wait and see how it shakes out.
We pay $1200 per month to MongoDB
We do not pay as much but all in all, we are happy customers because managing the clusters is pretty much hassle-free (backups, user management, monitoring etc.)
I actually agree with the underlying claim that Ofer Bengal was making. I do like how it ended up happening though - with all these cloud providers funding the Linux Foundation to work on the project 🤣
I love the OpenSource world. It always feels like a rebellion against capitalism
As one falls, many more in it's place arise.
What about redis search and redis json?
Good thing we never migrated away from memcached
To be fair memcached is slow as shit, redis was a bit faster but it certainly wasnt my favorite. Have you heard of tarantool
When memcached is considered slow. Most of the time it's the poorly implemented client
@@iuse9646 we benchmarked it, in our usecase redis wasn't faster, only easier to use and a bit more modern
@@daylen577 one thing I recently noticed is that when benchmarking redis vs memcached the perf difference is not that large.
But in production memcached nodes always feel much more stable both in latency, reliability and memory usage.
Especially when lru turned on and memory reaches its limit.
That is why many big corps still use memcached or memcache-like service for their main cache.
@@quangtung2912 The only thing Redis does significantly better is the management side, it essentially acts as a key-value database and you can access it as such from various tools (like JetBrains' DataGrip) and that can be super useful at times. Memcached is just a lot more simple, and you often end up clearing the cache entirely to resolve an issue instead of trying to figure out what's going wrong.
Out of context, I want to buy those hair. Where can I find 'em?
Garnet, the C# alternative, is actually faster than Redis and all of the alternative by a GOOD margin :D
Yeah but they don't pay Theo, so it's trash
@@bpx2798 shots fired 😅
Glad to see you asked for permission before reading out someone else's post
4:25 Windows has WSL so it does not really need native redis.
More github alternatives, please.
Gitea
@@DreamingCodes thanks.
Gitea actually went proprietary it seems. Quite ironic under this video lol
[1/3]
Besides that, you can go with something else than git entirely. SourceHut offers Mercurial hosting too, but there's also Pijul and Fossil.
[3/3]
There's a fork called Forgejo maintained by Codeberg (which used to be a Gitea instance). Besides that, you can look for some GL instance and there's also SrHt.
[2/3]
FWIW, the MongoDB Atlas serverless offering seems quite reasonably priced for SMB and startups.
I assume you meant documentdb and not dynamodb? If you meant document db it's definitely much less powerful than MongoDB from our companies experience. I also like the atlas platform, very nice for things like profiling queries etc.
Is that Arc browser?
Do you work at Netflix btw?
Can you do a video on the Anaconda/Conda license updates, please.
Didnt linux kernel use BitKeeper before linus wrote Git
yup, he mixed up BitKeeper and Mercurial
Vercel is built on top of AWS, why would they compete?
I’m not really sold on your mongo take. The mongo license change happened like 6 years ago and since then the company has continuously had major customer growth and seems to be doing well. I mean time will still tell of course if they eventually fade. But they seem to be kind of like Vercel in the fact they are constantly innovating and providing a far superior platform than AWS is with their documentdb solution. And their Atlas solution integrates very nicely with your AWS environment (or azure or gcp if you use them). But when the license change happened, I remember the same takes were being made about them and how that AWS will eat their lunch. And, as I said, we are at 6 years and counting now,
That said, I still think they’re an anomaly with all of this. To successfully do what they did they must provide a compelling story to stay on them and they’ve at least been trying to do that.
None of this applies to redis I think and they are still probably dead.
Did you miss the part where he showed that Mongo was losing millions of dollars? Or where it gained a mere 1000 paid customers in the past year (or was it quarter)?
I went to redis lab looking for managed redis, their pricing was like a middle finger no wonder cloud providers take their money.
what's next AWS Valkey
@t3dotgg is MongoDB next? The Linux Foundation could destroy NASDAQ: MDB
Perhaps Valkey could grow to capture Mongo use-cases as the project grows
edit: LOL, i commented well before the entire mongo segment
Mercurial and git are actually quite contemporary, I believe Mercurial started development first but released later? I don't think it is fair to say Mercurial came "after git".
Dagonfly is a great replacement!
Microsoft Garnet is interesting. Redis replacement written in C#? Quite unusual.
We making it out of Redis with this one 🗣️💥🗣️🗣️💥
I can't get my head around the fact Redis, the low latency in memory key value store designed to handle as many requests as possible, is not multi threaded and the maintainers were against implementing it.
OK it's built in C and doing safe multithreading in C is not an easy task - but still, given the project goals it's still a bit nuts to me.
complexity is the devil
@@beowulf_of_wall_stOr maybe don't use C
Is it worth discussing why Redis felt cloud providers weren’t contributing fairly despite Amazon be happy to throw engineering and Money at the project under the Linux foundation?
Self interest sure, but are essentially co-opting open source tech by going this route?
Because any given tech is nice, but if all companies who want to do cloud-something with it have to choose between Pepsi and AWSCola, there will be a fracture in the ecosystem.
i just noticed theo doesn't look at the camera for the entire video
Rest in Peace Redis
Let me just this redis alternative will pop up fast asf
Sleeping on Garnet would be a mistake.
4:30 Bun 1.1 already came out, with windows support lmao
Who uses Windows?
Redis is dead! Long live Redis!
the erlang BEAM replaced redis over a decade before redis was created
Corporations will just use a Redis cloud equivalent anyways. It might even be better for the devops for not having to manage Redis instances.
Only $9/month.. That's at least $8 too many if not 9
It's funny how whenever a license change, or corporate change happens, everyone forks it and the project dies. Nobody ever seems to pay attention.
That only shows that is is impossible to make money off open-source as a small business. You will lose money and your work will be taken away from you. Open source is good for hobby projects and for ultra-big corporations.
open source is good if its not your only product
You're not wrong. Your product needs to be super popular for making money but then the big players will make a good-enough copy that has the main features.
@@vladsaveluc2659 That only if your product is something like a nosql database. For more niche software, selling licences is the only way to make any earning
I didn't remember mongoDB existed till now.
What about dragonfly?
I will never get on a phone call to somebody's tool.
"what's with all the github alternatives in the linux community" this comment makes me feel old since i remember when Balmer called Linux communism. Yeah they now contribute and even ship it but some of the old heads still sleep with one eye open.
just swapped to linux somewhat recently, from windows10. the amount of times ive encountered issues caused from weird linux people being weird and stubborn is too dang hi. thats why im loving popOS so much so far. realistic consistency between everything instead of "ITS FREE AND OPEN SOURCE ONLY AND IT WORKS SOMETIMES BUT YOU GOTTA FIX IT MOST OFTHE TIME... STILL BETTER THAN WINDOWS THO FOR SURE".
so much penguinCope.
Don't underestimate the power of corporate idiocy, w.r.t. ridiculous subscriptions like MongoDB. Spending on commercial software among the Fortune 100 is truly insane. Throw in AI's outlandish promises, and the C-suite is pissing money in every direction.
valkey is a banger name
You don't talk about dragonfly
You forgot Garnet by microsoft
The most mature and fastest alternative currently is Microsoft Garnet.
Nix
bun is on windows now
Are we about to enter The Redis Wars?
"Everyone" in the title is a clickbait, I'm pretty sure that majority of Redis users are not going to do anything because of the license change.
Those companies and amazon only care about open source when they rely on it for their business. Come on.
tbh Redis (Redis Labs) will win because all forks will be 3-BSD and they can just absorb any contributions to any of them, too bad it wasn't GPL lol.
4:25 Redis does not have support for windows
I love how theo loves BUN but the zig people find bun to be an illegible mess
As for redis, I think it’s safe to say it’s jover 😂
Redis screwed itself with the way it made this change, but that doesn't mean the monopoly cloud providers Amazon, Google, and Microsoft, who make a lot of money using open-source software, are in the right. Good to see the hardcore open-source groups taking up development.
OK, TH-cam still really doesn't like me posting links. Garnet was mentioned in the comments for your first vid on Redis' self-sabotaging.
Looking up garnet, cache-store, Microsoft and Badrish Chandramouli should hopefully bring up the announcement blog post.
I think the problem isn't performance, it's that it's not the most well supported language on Linux and if MS decide to drop it one day (or stop supporting C# on Linux) it'll just wither away in a way that just doesn't happen with established projects in common languages.
It's fully supported on linux and completely open source - I'd guess you meant that it's not the most popular OSS language, which is fair enough. If MS were to stop supporting C# on linux (which would be absolutely bonkers, they use C# on Linux for azure stuff), there'd immediately be an active community-driven set of binaries/etc.
@@OnHalo Yeah, I mostly don't see a lot of traction for C# in the broader linux world but maybe I'm not looking in the right places.
But anything can happen and just 10 years ago MS was still the enemy of open source so, we'll see.
It's very well supported on Linux.
Theo, why don’t you open source uploadthing? Then you can spend your time and money innovating fast enough that I don’t run your OSS version in a Docker container.
A bit hypocritical.
yeah i want to host his thing and resell it without paying him anything, if he doesn't do that then he is literally destroying open source software