How I Would Build a Software System for Scale
ฝัง
- เผยแพร่เมื่อ 28 มิ.ย. 2024
- 🌎 Follow me here:
Discord: / discord
Twitter: / tomdoes_tech
Facebook: / tomdoestech
Instagram: / tomdoestech
TikTok: / tomdoes_tech
☕ Buy me a coffee: www.buymeacoffee.com/tomn - บันเทิง
![ช่วยพาฉันผ่านเวลา - วสันต์17 [Official MV]](http://i.ytimg.com/vi/HFnntWoqc54/mqdefault.jpg)
Some tips based on my experience in a large project (30+ microservices, 70+ devs):
DX - In cases where you have many microservices, running them all locally for development would rarely be possible. Devs would need beefy laptops to be able to run all the services, and you would also run into the issue of having different setups (eg. docker-compose) for local development and production (eg. kubernetes), which will get exponentially harder to keep in sync as you introduce more and more services and infrastructure pieces. Best approach for this (but pricy) would be to have on-demand remote environments, ideally running inside of kubernetes (or your platform of choice), in which case you eliminate most of the "works on my machine" problems, by having the devs work on the setup that's closest to production. As a bonus with this approach, you also get the benefit of being able to provide on-demand environments for QA/Compliance teams, or for any other purpose, without much effort, since it uses the same logic.
Microservices - Please use kafka/rabbitmq/nats (or alternative tech) for communication between services, as opposed to having direct communication. This adds decoupling to your architecture, and will let each service run at it's own pace. This is especially useful when APIs change (like in the example from the video) so instead of having to now update all services that depend on the one that changed the api, you would only change that one, and the way it communicates with the "message bus".
Great explanation 👌🏽.
I would like to know if a startup developping a multi-tenant SaaS (around 100k users predicted) should begin with microservices architecture or just start with a multi-layer monolith architecture (frontend, monolith backend, single DB) ?
@@prosperzegue6735 It really depends on your case. For example, if your team never worked on microservices, choosing to do so from the start will add unnecessary complexity and slow you down.
You will inevitably have to go the microservice route with that scale, but starting with the monolithic approach and then slowly breaking apart the monolith into different services is still a valid strategy.
You can make this process easier by not tightly coupling your code. This can be specific to the tech/language/framework you choose, but you can still take measures to make moving away from the monolith less painful.
On the other hand, if you're confident in your team and have prior experience with microservices, I don't see a reason to start with the monolith except for development velocity (only in case where your team doesn't have experience with the microservice architecture). Just be careful of the premature optimization pitfall, and don't over engineer your setup.
@@wizz0056 Thanks so much for your reply. Your answer is perpect 👌🏽.
very nice to see how a senior dev would approach a project architecture-wise. thank you! very useful 👏🏻👏🏻👏🏻
Please make a minimal project using this architecture if you have the time.
Great job explaining this architecture. One crucial piece that I see missing is your observability/ monitoring layer. I’ll add prometheus, grafana to collect app and infrastructure metrics. You can even spice it up a bit with jaeger for traceability. This will equip you with a feedback loop to extend your pipeline to do canary releases later on.
Ahh yeah good call. This was the second time I recorded the video and I forgot the observability part. Probably should have written it down instead of doing it off the top of my head.
4:40 im using bun and my microservices can consume shared packages from the package.json at the root of the project. So if i need to upgrade a package shared across all microservices i can just update the root package json
That sounds awesome! I haven't used bun much
how about the communication between micro-services? gRPC?
Yeah gRPC is a great way to communicate between them, could also use a message bus like Kafka
I'm not a big fan of gRPC, it requires more setup and the difference doesn't take much, I think message bus and json is enough 😂
could you make one more video with this architecture for minimum project
Very nice! Thanks for sharing 🙏
Very nice video Tom!. But I am worried about running all the stack in a developer machine. From my experience, even a resourceful a machine can run out of resources pretty quick running everything at once.
Also, as you are targeting a micro services architecture for the backend, how about using micro frontends for the web? It can enhance the work of several teams working separately in the same frontend. I also understand that micro frontends also facilitates to ship often.
I don't have any experience with micro frontends, from my understanding they sound good in theory but are a nightmare to work with.
I think there's a way to get the whole thing running on Dev's machines, or at least the parts they need
Are there any youtube tutorials that do this in practice which are explained in a clear manner?
Not that I know of sorry
NextJS Layer will be using Pages or App Router? Is there any need for App Router in this architecture?
Here, there is no need to go for app router. Choose based on the team and their experience.
@@cannotthinkofanybetterhandle Thanks
Hey there. If we are using any other backend then do you think we could leverage proper type safety?? Like at that point it just becomes manually typing it out...
Shouldn't we make bff common for both mobile and web (instead of doing it twice in the next js web app and a separate mobile bff)?
No, BFF layers should be for a specific UI, they're 1-1
Tom! Why not ROR ?
What is RoR?
How do you do BFF?
The bff is just another service
@@TomDoesTech how do you communicate them, with gRPC?
Or you can just deploy LOCALLY if your target audience requires strict data privacy, has unreliable internet connectivity, or needs low-latency access to critical systems, such as in healthcare facilities or remote locations.
Like an intranet?