If you want to get some serious views, do a Kubernetes gateway mega benchmark. Nginx, APISIX, Kong, Istio, etc. There are so many gateways and basically no one benchmarks. I suspect it's because these gateways are popular in enterprise, and enterprises care about features and not performance, but regardless.
Dude I just want to say your channel is a literal goldmine. Real world use cases, relevant technology, explanation of the implementations, great metrics, always responding to good feedback positively. I watch one of your videos and then see another one I’m interested in watching and it’s always uploaded like within the last month. The quality and frequency you upload these at is astonishing. Easily one of my favorite channels right now! This is my first time donating on YT, I just have to after appreciating this channel so much. Thank you for all the hard work and please keep it up!
WTF I could bet that Redis will be a lot faster than memcache. Last time I was using memcache was like 10 years ago. I remember that I switched to Redis because everyone was saying that it is faster and it started to being recommended in frameworks, while Memcache become obsolete. I think I will go back to Memcache for most projects where I do not need extra Redis features (just simple key->value cache). Thank you for that.
Redis won I think mostly because of simplicity, marketing and overhyped. Memcached always have been better for caching for many reasons: - Multi threading, so much better scalability - More stable latency - Much more stable memory usage with slab allocator - More precise and more performance LRU implementation Many many big companies still use memcached (or memcache like) for their caching: Netflix, Wikipedia, Reddit, Quora, Pinterest, Facebook, Instagram.
hahah u look like me, i always use memcache and it work fine but i change to Redis because of word of other people saying it better and fast. i don't hesitate to change without test it to see the performance at all.
You definitely deserve more views, your channel is a goldmine for software engineers. I love how you use real world technologies and basic configuration. I would also love to see videos comparing these technologies but with configuration tunning to see where they can go.
I agree, just remember that for *most* applications, performance should be like a #4 or #5 priority, after (not in order) security, maintainability, developer/devops velocity, etc. Of course if you expect a million users and there are no other trade-offs, pick the faster one.
@ thanks Anton for the answe, im familiar with aws and also azure but i was mostly deploying web apps, i don’t have experience with monitoring. But i’ve seen that you have some playlists for garfana and prometheus, i’ll check them out. Your channel is the best im recommending it to everyone interested in this topics
Hi Anton, thanks for your terraform and kube tutorials made a production cluster at my current job using them truly valuable. Thanks for these benchmarks too!!
Great content as always! Just a quick question: have you thought about using YCSB or other benchmarking tools to simulate scenarios that are closer to real-world use cases? For example, to create workloads with a get/set ratio of 90% or higher, reflecting the read-heavy patterns typical in caching systems like Memcached, which are widely used by platforms like TH-cam, Twitter, and Facebook. It seems like for now the radio is 50% as shown in the main.go file.
Dragonfly was sooo unstable for me and not fully a drop-in Redis replacement. Trust me.. I know.. You can better try to migrate to either Valkey or KeyDB. But avoid Dragonfly.
Great benchmarks! Could you add a test comparing PHP with Laravel Octane (Swoole, RoadRunner, FrankenPHP) and raw PHP with Open Swoole? For fairness, compare framework to framework and language to language. Thanks!
depend on the context. however for a mpa kind of style. open swoole and franken is slower than swoole. now road runner some does win and the other are loaing to swoole. eg iaong swoole table
Hmm I thougth long and came to the conclusion that you definitely should test selfhosted Auth-Providers like: - Keycloak - Authelia - Authentik And compare them performance wise :) I definitely would like to see this!
Thank you very much for another amazing test, Anton! A tinny feedback: could you please add a legend with the information on application instances count and resource limits. For example: "2 PODs each 1 vCPU 1 Gb". You mentioned VMs were `m7a.medium` but amazon product names tend to be confusing for those who mostly uses other cloud providers. Resource metrics is more universal to grasp. Also sometimes you run tests using two pods at the same time (sorry if I'm wrong), having this legend would improve the way we consume percentage units on the graphs.
thank you for the feedback! i'll try to improve. m7a.medium has 1 vCPU and 4 gig of ram, and i use 20 pods per application. here's the source code: github.com/antonputra/tutorials/blob/main/lessons/225/test/2-redis.yaml#L8
I've always disliked Redis with a passion. No service seems to randomly break as often in prod as Redis. Thank you for the last push I needed to move to Memcached!
I want to point out that you don't neccesarrily have to choose one or the other, if you need the advanced features Redis has for a specific set of operations in your application you can use it for only those, and you can just choose to use memcached with all other operations that do not need those features. In all cases good test.
thank you! from the devops perspective, any additional component requires additional playbooks, procedures, etc. from the platform perspective, less is much better! your team is familiar with how to manage and recover if something happens in prod
@@AntonPutra Absolutely, it is always a trade-off, so every architect or whatever person is designing the system will have to think about the technical implications both in regards to development but also operations. The comment was not so much meant for you as it was for people who might be stuck thinking that they couldn't just use both of them. You just happen to present this in a way that might trap people in that mindset. I think most experienced people will understand though.
Try to use lager chunks of data e.g. 10kb per record. You will notice that the Redis performance will be drastically go down compare to small records (drop will be not linear as expected). As well as usage of expire method call after the set operation.
Reasons why FastAPI fails so hard: 1 - in first test, you comparing more JSON implementation in Golang and Python. By default, FastAPI uses slow standard library implementation. 2 - in second test, in Python uses slow SQLAlchemy ORM, while in Golang there is straightforward query 3 - in second test, your code in Python uses session.refresh(), that produces another SELECT statement (that gets all the rows). In Goland implementation, you using RETURN SQL statement only to get inserted ID. To compare Web frameworks, we need to implement some changes in python code. I can make PR for it.
Technically you do still save database resources if you off-load the caching part to a different application (on a different machine, or on the same machine as the DB if the DB could not utilize those resources anyway).
I think these benchmarks could benefit from some sort of "control". Like for example in this SQL could be one. After all that what you are trying to cache often. Without control the graph might be very impressive looking, but then shows just minor different at the end that will not make any difference. Like say A could be twice as fast B, but if the difference is a nanosecond, does it matter. So with control it would scale it to show "no difference". But if there still is a massive gap, maybe something to consider. Not saying it's not valid result without control but it lacks the real world perspective of its actual impact.
Something feels wrong to me about the sharp uptick in memcache latency around 80% cpu. malloc fragmentation? I feel like a Linux subsystem is tuned wrong. Looks like NUMA crossing I've also seen in a long ago life.
p100 can be very large and volatile and usually it does not describe very well performance characteristics of backends.This is why it's common to measure p50, p99 or p99.9 latencies of backends
@@poet9638 redis is not only for caching, I used redis lists and it can be used as queue system. So, maybe it's easier than using a lot of different systems in one project.
@@dmitrii_cl Hey, I think there was a bit of a misunderstanding. When I mentioned Redis' concurrency, I wasn't talking about its caching speed. I meant its ability to handle concurrency with tools like Redlock for distributed locks, which helps manage access to shared resources across multiple processes.
🔴 To support my channel, I'd like to offer Mentorship/On-the-Job Support/Consulting (me@antonputra.com)
I would be interested to see stuff from NATS ecosystem
Please test etcd.
If you want to get some serious views, do a Kubernetes gateway mega benchmark. Nginx, APISIX, Kong, Istio, etc. There are so many gateways and basically no one benchmarks. I suspect it's because these gateways are popular in enterprise, and enterprises care about features and not performance, but regardless.
would you compare redis vs dragonflydb
Maybe include valkey also? With its better multi-threading
Bro is picking up performance benchmark like infinity stones.
he's picking up youtube content meta for programmer section
🤣🤣🤣
picking up performance benchmark and infinity stones.
Dude I just want to say your channel is a literal goldmine. Real world use cases, relevant technology, explanation of the implementations, great metrics, always responding to good feedback positively.
I watch one of your videos and then see another one I’m interested in watching and it’s always uploaded like within the last month.
The quality and frequency you upload these at is astonishing. Easily one of my favorite channels right now!
This is my first time donating on YT, I just have to after appreciating this channel so much. Thank you for all the hard work and please keep it up!
I like the fact that you don’t just throw the benchmark but also explain how you’ve made it and how it is relevant
thanks!
Memcached vs KeyDB (Redis fork focused on performance)
and vs Dragonfly and Skytable
+ ValKey, another Redis fork backed by AWS with improvements. Actively being developed
this comment should be pinned , thank you for the information i always use valkey , this is new for me
vs DiceDB
thanks! i'll do KeyDB as well as Dragonfly in a week or so
⚠If you can improve Redis or Memcached, please send me a PR, and I'll rerun the test! ⚠
sure. done!
WTF I could bet that Redis will be a lot faster than memcache. Last time I was using memcache was like 10 years ago. I remember that I switched to Redis because everyone was saying that it is faster and it started to being recommended in frameworks, while Memcache become obsolete. I think I will go back to Memcache for most projects where I do not need extra Redis features (just simple key->value cache). Thank you for that.
They wont allow you to use memcached in enterprise company
as soon as anyone suggests how to improve Redis, memcached looks like a better alternative for a simple use case like SQL caching
@@twitchizle why?
Redis won I think mostly because of simplicity, marketing and overhyped.
Memcached always have been better for caching for many reasons:
- Multi threading, so much better scalability
- More stable latency
- Much more stable memory usage with slab allocator
- More precise and more performance LRU implementation
Many many big companies still use memcached (or memcache like) for their caching:
Netflix, Wikipedia, Reddit, Quora, Pinterest, Facebook, Instagram.
hahah u look like me, i always use memcache and it work fine but i change to Redis because of word of other people saying it better and fast. i don't hesitate to change without test it to see the performance at all.
this channel is so good, keep up the great work!
thank you!!
You definitely deserve more views, your channel is a goldmine for software engineers. I love how you use real world technologies and basic configuration.
I would also love to see videos comparing these technologies but with configuration tunning to see where they can go.
I watch those benchmarks because they are like therapy :D
😊
Your benchmarks are what people like us need so badly. Now we can have the source for future tech decisions.
I agree, just remember that for *most* applications, performance should be like a #4 or #5 priority, after (not in order) security, maintainability, developer/devops velocity, etc.
Of course if you expect a million users and there are no other trade-offs, pick the faster one.
You’re really helping me choose technologies that I want to work with in th future, Thankyou.
my pleasure!
Splendid overview of the problem before the benchmark, Anton!
Love the systematic approach you use for all these tests. Very good!
Thank you Anton. One request can you do an extra video like behind the scenes to explain to us how do you setup those instances in aws
it's simple if you're familiar with AWS
@ thanks Anton for the answe, im familiar with aws and also azure but i was mostly deploying web apps, i don’t have experience with monitoring. But i’ve seen that you have some playlists for garfana and prometheus, i’ll check them out. Your channel is the best im recommending it to everyone interested in this topics
I just want to say `thank you` for all those awesome benchmarks! :-)
my pleasure! 😊
Keep up the good work, your benchmarks provide necessary information to take decisions based on multiple factors.❤
Hi Anton, thanks for your terraform and kube tutorials made a production cluster at my current job using them truly valuable. Thanks for these benchmarks too!!
my pleasure!
That is wild. Honestly didn't think the difference between these two would be this large.
Thank you for the great video! It would be interesting to see comparsion between Redis and DragonflyDB
yes coming in a week or so!
Thank you for doing these tests!
Redis vs DragonflyDB
yes in about a week
some quality work🙌🙌
Hi Anton Putra, your video is amazing! By the way, your name sounds like an Indonesian name.
Вот это я удачно зашёл на youtube, класс!
haha
Great content as always! Just a quick question: have you thought about using YCSB or other benchmarking tools to simulate scenarios that are closer to real-world use cases? For example, to create workloads with a get/set ratio of 90% or higher, reflecting the read-heavy patterns typical in caching systems like Memcached, which are widely used by platforms like TH-cam, Twitter, and Facebook. It seems like for now the radio is 50% as shown in the main.go file.
I loved it. It's awesome. Would be very nice to see how Redis and Memcached behave with evict strategy.
you mean when they reach memory limit?
@@AntonPutrayes, i'd like to see the latency when redis and memcached reach memory limit using the available alghoritms such as TTL, LRU and LFU
very interesting result! thank you!
Redis vs NATS next.
Very interesting! Also adding ValKey (Redis fork) could be helpful. Benchmarking streams as a second test is very interesting as well.
something like NATS vs kafka vs RSMQ?
yes nats in a a week or two
@@AntonPutra great job, Anton
I wished you've added Hazelcast and Valkey for this caching tests. Thanks for your efforts.
thanks, i'll add them in the near future, especially valkey
Have you tried dragonfly? It has been advertised as performamt redis with same redis like api
no, but i keep getting that ad on x all the time 😊
Dragonfly was sooo unstable for me and not fully a drop-in Redis replacement. Trust me.. I know.. You can better try to migrate to either Valkey or KeyDB. But avoid Dragonfly.
Great benchmarks! Could you add a test comparing PHP with Laravel Octane (Swoole, RoadRunner, FrankenPHP) and raw PHP with Open Swoole? For fairness, compare framework to framework and language to language. Thanks!
depend on the context. however for a mpa kind of style. open swoole and franken is slower than swoole. now road runner some does win and the other are loaing to swoole. eg iaong swoole table
will do!
Awesome work! I'd like to see Swift (Hummingbird framework) vs. Go and Rust
Hmm I thougth long and came to the conclusion that you definitely should test selfhosted Auth-Providers like:
- Keycloak
- Authelia
- Authentik
And compare them performance wise :)
I definitely would like to see this!
Very informative content
thank you!
awesome! if you are searching for test scenarios and like to: a test of memcached vs varnish (vs redis) would be interesting too
Amazing as always. It would be amazing if you could do a Docker Swarm VS Nomad by Hashicorp
Thank you very much for another amazing test, Anton! A tinny feedback: could you please add a legend with the information on application instances count and resource limits.
For example: "2 PODs each 1 vCPU 1 Gb". You mentioned VMs were `m7a.medium` but amazon product names tend to be confusing for those who mostly uses other cloud providers. Resource metrics is more universal to grasp. Also sometimes you run tests using two pods at the same time (sorry if I'm wrong), having this legend would improve the way we consume percentage units on the graphs.
thank you for the feedback! i'll try to improve. m7a.medium has 1 vCPU and 4 gig of ram, and i use 20 pods per application. here's the source code: github.com/antonputra/tutorials/blob/main/lessons/225/test/2-redis.yaml#L8
I've always disliked Redis with a passion. No service seems to randomly break as often in prod as Redis. Thank you for the last push I needed to move to Memcached!
i've been dealing with redis for the last 7-8 years, and the most common production issue is always redis 😂
Note: memcache is not the same as memcached
your videos are tryly helpful. i am using postgres for my project and memcached for my cache(i only need the fast key-value storage)
I want to point out that you don't neccesarrily have to choose one or the other, if you need the advanced features Redis has for a specific set of operations in your application you can use it for only those, and you can just choose to use memcached with all other operations that do not need those features. In all cases good test.
thank you! from the devops perspective, any additional component requires additional playbooks, procedures, etc. from the platform perspective, less is much better! your team is familiar with how to manage and recover if something happens in prod
@@AntonPutra Absolutely, it is always a trade-off, so every architect or whatever person is designing the system will have to think about the technical implications both in regards to development but also operations.
The comment was not so much meant for you as it was for people who might be stuck thinking that they couldn't just use both of them. You just happen to present this in a way that might trap people in that mindset. I think most experienced people will understand though.
how about using SQLite in-memory as a cache?
sqlite in memory? that's new to me
@@AntonPutra yes, you can use SQLite in RAM. Interesting stuff :)
@@d3stinYwOw ok interesting
Yes I really wanna see how SQLite in memory compares to Memcache
zmq vs kafka va redis streams?
ok interesting, never heard of redis streams before though 😊
Try to use lager chunks of data e.g. 10kb per record. You will notice that the Redis performance will be drastically go down compare to small records (drop will be not linear as expected). As well as usage of expire method call after the set operation.
KeyDB vs Redis vs Memcached would be nice
thanks! added!
I've heard that Redis v8 going to have huge performance improvements, would be interesting to see if it is true
yes, waiting for the ga version and will test it as well!
After some time a tutorial on Prometheus?
sure, prom 3.0 was just released a few days ago
♥️
Can we see some benchmark for java spring boot (against node/go) which is used a lot in company software
Redis vs Valkey, the fork recently ibteoduce multi-threading, so really keen to see how much of a difference does it make
Great video, can you try dragonfly in place of redis which is a drop in replacement for redis. Thanks
Now try keydb, dragonfly. Those are also alternatives of redis
ok added to my list
@ thank you I appreciate that 🤌
Valkey.. nobody?
postgresql vs mongodb when?
Exactly what I want to see
but that's a different data paradigm altogher right
? why do you want to see it?
Apples and pears right?
@@mickolesmana5899 data paradigm doesn't matter. Both databases are "general-purpose" hence it's valid to compare their performance.
@@zuzelstein wrong
What about valkey or dragonfly, keydb?
Whats the opensource fork of redis now? Perhaps you should consider that instead if redis
dragonfly vs keydb vs redis?
yes dragonfly coming in a week or so
vs Valkey
Would it make sense to ask for Memcache vs SQLite?
May I ask Anton...
What do you use for your slides/presentation? (Or is it a draw like program for aws architecture?)
sure, i use adobe suite
@@AntonPutra Ahh .. Thank you!
Reasons why FastAPI fails so hard:
1 - in first test, you comparing more JSON implementation in Golang and Python. By default, FastAPI uses slow standard library implementation.
2 - in second test, in Python uses slow SQLAlchemy ORM, while in Golang there is straightforward query
3 - in second test, your code in Python uses session.refresh(), that produces another SELECT statement (that gets all the rows). In Goland implementation, you using RETURN SQL statement only to get inserted ID.
To compare Web frameworks, we need to implement some changes in python code. I can make PR for it.
@AntonPutra, can you do a comparative benchmark between Microsoft garnet and redis?
Also Garnet and memcached
can we have redis vs sqlite vs memcache?
What about Valkey, one of the Redis forks?
yes, it's on my list, i'll do it soon
Do we still get advantages for caching DB queries if all popular databases are equipped with their own caching layer?
Technically you do still save database resources if you off-load the caching part to a different application (on a different machine, or on the same machine as the DB if the DB could not utilize those resources anyway).
yes, unless that layer can be independently scaled horizontally
I think these benchmarks could benefit from some sort of "control". Like for example in this SQL could be one. After all that what you are trying to cache often. Without control the graph might be very impressive looking, but then shows just minor different at the end that will not make any difference. Like say A could be twice as fast B, but if the difference is a nanosecond, does it matter. So with control it would scale it to show "no difference". But if there still is a massive gap, maybe something to consider. Not saying it's not valid result without control but it lacks the real world perspective of its actual impact.
Redis or keyDB vs SQLite in-memory, please.
Thank you! Could you make benchmark on redis vs dragonfly db. Dragonfly position itself like 5x less resource consuming, while have redis like api
yes i'll do Dragonfly in a week or so
Can you make a redis vs dragonflyDB vs KeyDB vs Mongo for caching? Or any combination of those
Can you compare Redis vs DiceDB? It's a new KV database which claims to be faster than redis.
thanks! will do!
When redis quit open source, there were many lists of alternatives, some of which are many times faster. How about some of them for next benchmark?
Bro please compare key db vs memcached vs redis
can you do postgres json or jsonb vs mongodb
performance and also data size
yes, in the very near future!
Redis vs Keydb vs Valkey
ok added to the list
Redis has a cool drop-in alternative - it's called Dragonfly. It would be interesting to compare both
yeah, keep getting those ads on x. will do it maybe in a week!
You should do Scylladb vs Cassandra
Wouldn't it be more fair to compare Memcached with the Redis multicore forks? For example DragonFly etc.
Hi interesting test.
Can you bench Sōzu reverse proxy vs Nginx.
never heard about Sōzu before but i'll add it to my list!
I haven't used redis as a SQL cache. I've only used it as a key-value store for data I can afford to lose (ie, backing store for HTTP sessions).
Prisma vs sequelize?
How about YugabyteDB vs CockroachDB vs Cassandra vs ScyllaDB?
Tigerbeetle the Transaction DB, please 🚀
compared to what since tb is the only of its kind
never heard before, but added to my list!
Also maybe dragonfly tests 👀
yeah, i keep getting ads on x all the time
Something feels wrong to me about the sharp uptick in memcache latency around 80% cpu. malloc fragmentation? I feel like a Linux subsystem is tuned wrong. Looks like NUMA crossing I've also seen in a long ago life.
Redis vs KeyDB!
ok added!
maybe can added sqlite as cache on ruby on rails 8
maybe in the future
Valkey vs Redis vs DiceDB
ok added
bro you need to take the ring to mordor
well the most reason ppl choose Redis over Memcached is for distributed in-memory storage purpose not performance
Please do Kafka vs Pulsar
What does p99 percentile mean?
it means 99% of all requests completed under that time range, p95 means 95%, etc.
@ So why not p100 (if there is such a thing)
p100 can be very large and volatile and usually it does not describe very well performance characteristics of backends.This is why it's common to measure p50, p99 or p99.9 latencies of backends
Waiting for,
Redis vs ScyllaDB
NGINX vs OpenCyperPanel
Citus data vs vitess with 8 workers pleaseee :D
specifically 8? 😊 i'll see when i can do it
@@AntonPutra just a random number.. But more workers better showcase
@@djordje1999 got it
LavinMQ vs RabbitMQ would be interesting.
ZeroMQ, baby !! Faster then you can look.
ok added to my list!
🍿 Benchmarks: th-cam.com/play/PLiMWaCMwGJXmcDLvMQeORJ-j_jayKaLVn.html&si=p-UOaVM_6_SFx52H
Keydb vs Redis
ok added
When elixir 😊😊😊😢?
Next… Redis vs SQLite 🎉
questionable 😊
Redis is not open source and then it should be replaced with ValKey - Open source fork of redis my linux foundation
agree!
openlitespeed vs nginx serving php applications
I think Redis is used because it has excellent concurrency processing capabilities, but...;;
@@poet9638 redis is not only for caching, I used redis lists and it can be used as queue system. So, maybe it's easier than using a lot of different systems in one project.
i think it's mostly because it's hyped. i even regularly see billboards on the highway in the bay area with redis 😊
@@dmitrii_cl Hey, I think there was a bit of a misunderstanding. When I mentioned Redis' concurrency, I wasn't talking about its caching speed. I meant its ability to handle concurrency with tools like Redlock for distributed locks, which helps manage access to shared resources across multiple processes.
im working in a tech company which my job is keep redis cluster alive, you offended me bro
i'm responsible for redis as well
Memcached vs KeyDB
added!