Publish-Subscribe Architecture (Explained by Example)

In this video, I want to discuss the Pub sub architecture or publish subsribe architecture and talk about the pros and cons and differences over Request Response.
One of you guys asked me which is better REST or Message queue. I struggled to answer for a while but After thinking about it I discovered that both REST and Message queue does boil down to basic elements. REST being HTTP which is Request-Response and Message Queue being Publish subscribe.
Timecodes
0:00 Intro
3:00 Where does request response pattern breaks
7:00 Response request pros & cons
11:12 Pub/sub pattern
17:50 Pub/sub pros/cons
- [ ] Request/ Response
- [ ] Pros and Cons
- [ ] Publish-Subscribe Definition
- [ ] Pros and Cons
Req/res
Browsing
Fetch or XMLHttpRequest
SQL query
Pros
- Simple to implement (REST)
- Stateless (HTTP)
- Sync or Async (service and client)
Uploading a youtube video
- Compress video
- format video
- Notify subscribers
Upload breaks down the video into small chunks and uploads each one
Once all successfully uploaded send a whole video to the compress service
One video compressed, the compress service sends to the format service and notifications service
If you are only using pure request-response, The client will be waiting until all of that stuff is done and if the connection is broken or any failure happens you lose.
Plus let's assume You want to also add a feature where the video will need to be checked for copyrighted material.. so the compress service now need to make a request to the copyright service and make sure all of that requests return successfully
Cons
- Doesn’t scale with multiple receivers
- Creates extreme coupling between multiple services
- The client and servers have to be running to communicate.
- chaining, retries, circuit breaking
- Chain effect and circuit breaking (microservices)
Meet the publish-subscribe model
Ok, this is starting to sound like a commercial.
Publish subscribe or pubsub for short is an architecture model where a publisher posts. a message to a topic (usually in the form of a queue) and clients optionally can subscribe to the topic and get notified when a new message is posted.
Upload breaks down the video into small chunks and uploads each one
Once all successfully uploaded send a whole video to message queue or topic
The compress service subscribes to the topic. The new video will get received by the compress service, compresses the video and publish it back to the topic or queue as a compressed.
The notification and copyright service subscribes to the topic.
Redis/kafka/RabbitMQ
Pub sub
Pros
- Scales with many receivers
- Loose coupling(publishers don’t know the subs, subs don't know the pubs)
- Caching
- Work while publishers or subscribers are offline
- Great with microservices avoid chaining, retries, circuit breaking
- Pairs with event-driven architecture
Is the pub/sub perfect though?
Decoupling side effect is a serious problem. You just decoupled the sub from the publisher, how do you guarantee to deliver a message?
Cons
- Message delivery issues (exactly once delivery ar least once) • The Two Generals’ Problem
- Network saturation- (depends on Pubsubs system) in push model clients might get overwhelmed, in pull model false attempts
- scaling is still an issue
🐘 Postgres Videos
• PostgresSQL
🧮 Programming Pattern Videos
• Programming Patterns
🛡 Web Security Videos
• Web Security
🦠 HTTP Videos
• HTTP
🏰 Load Balancing and Proxies Videos
• Proxies
🏭 Software Architecture Videos
• Software Architecture
🛰 Network Engineering Videos
• Network Engineering
💾 Database Engineering Videos
• Database Engineering
🐍 Python Videos
• Python by Example
🔆 Javascript Videos
• Javascript by Example
Support me on PayPal
bit.ly/33ENps4
Stay Awesome,
Hussein