Yeah. Really good video. And you showed all code from start to finish. I love nick chapsas channel too, but so many of his videos say 'i just did this or that, and wired up this beforehand', so we cant see all the code and follow along. Excellent touch to add the section at the end where you mention the possible improvements. 10/10 video.
Thanks a lot, I'm really glad you found this video valuable. After releasing my course, my main objective going forward is upping the quality of my YT videos, and providing even more value. So I'm happy this video delivered :)
Hello again, I'm unfamiliar with Docker in general. Do you have a video on something similar to what is in this video? I don't understand the part at 11:13. Then magically the DbContext is connected to the correct SQL dB. (The dockerfile and appsettings might reveal more, but they weren't shown). Maybe it's a basic thing, but I'm a noob so 😂
I have been working with C# since past 1.5 years and I am not sure if I will be able to build something like this in near future 😭😭. Thanks for the video, this is quality content.
@@MilanJovanovicTech Other random real life apps similar size to this Url Shortener.... but I can't think of any! :p I did think about a web crawler/scraper but maybe that's a bit too big or complicated.
@MilanJovanovicTech Hi, Milan. You can show different patterns like command, composite, chain of responsibility or async/await pattern. I think these are very interesting patterns that would be very useful.
Thanks for another great video Milan! Additionally, I implemented a background service which generates a batch of codes and stores them in the database. When the amount of "available" codes (I use a flag) reaches 10, it starts generating a batch again. Works like a charm!
You forgot to mention that strings in SQL Server are case-insensitive. In this case, the "xyz" string would be equal to "XYZ". You can fix this by using different collations.
Best part are suggestions for performance optimizations. That is making huge distinction among million of similar videos/articles. Most less experienced developers can kind about complex topics like performance optimization, scaling in/out and also when to actually do it in whole development and production life cycle.
_random.Next(Alphabet.Length - 1) is wrong. To quote from the docs, Next(Int32) returns a non-negative random integer that is less than the specified maximum. You're existing code will always omit the last character
Loved it! Finished your pragmatic clean architecture, will your new tutorial include these tips you are sharing in TH-cam. It would be better to have them to wrap up.
Hey, that's awesome. What did you think of the course? 😁 The next course I'm planning will be about Modular Monoliths. I'm currently working on a high-level curriculum. But it won't be ready until halfway through 2024.
@@MilanJovanovicTech It was well structured and explained. Both theory and real life sides helped me to easily grasp the concept. The course really shows how deep experience you have ✌🏻
Another optimisation you could do is to use a Bloom Filter to avoid querying to the database to check if the code is already existing or not, with minimal memory usage. Redis also supports this Data Structure. 🙌
Since the chance of collision is very low, we can optimistically store the generated url and let a unique index fail upon the duplicate. Then, simply generate the new url and try again.
Great video Milan! Maybe this could become a series, system design practices etc. Maybe you can implement a simple image optimaztion service that transforms .png files to .webp? I dont know the complexity of the idea or it is applicable.
You have already done few videos about different types of architeture: clean architecture, vertical slice. Maybe it's time to do a video about onion architecture and compare to clean, and a layered architecture that is done in a correct way. Also you can make about EF Core Extensions library, it has as free as payed features. I have personally used this library for bulk operations with a great success. Maybe this library can sponsore your video 😊
@@MilanJovanovicTech they are but still maybe you can find something to talk about. And thanks for answering! I really appreciate that you take your time and engage so much with a community!
Nice work, another optimization that could be done is before hitting the database to check if string matches the required length and has only valid characters
Apologies, I don't understand this. Where is the optimization here? Isn't it redundant since he is the one generating the string from a list of characters that he chose?
@@daraghodonoghue_ That would be for checking when doing the GET request to redirect to the proper URL, of course this is useless for the generation of the URL
@@daraghodonoghue_ He defined the length as a constant 7, so anything different than that can be ignored to don't even waste time and resources making request to database
Decent video, but the only right answer to "How would you build a URL shortener with .NET and SQL Server" should be "Why would you want to build a URL Shortener .NET and SQL Server in the first place?"😄
10:00 Is that an add-on you have installed to have the "Initialize field from constructor" or have you made it yourself. If so do you have a video on how to do this?
What about if we want to add a statistic per every link? Also, the end user want to view a charts with detailed info. The design of the system would be very interesting.
Hello @MilanJovanovicTech, thank you for your response. In the context of software architect interviews, discussions usually center around low-level system design. However, I'm keen to learn more about both high-level design principles and the intricacies of low-level design, particularly in the context of a specific domain. Your insights in both areas would be invaluable. Could you please share your expertise?"
hmm... why dont use hash creation tool + salt per user? the same approach we use for passwords, but with less caution regarding length and salting iterations. Libs are already there, they are more random and stable)
No particular reason other than I wanted to... I'm querying by a unique value, so First/Single would behave identically in terms of the generated SQL. Single would never throw an exception because the value is unique.
Hi. Thanks for the great video. I'm not 100% surge, but actually I think that using FirstOrDefault is better because in a large set of data, the query stops on the first hit, and with SingleOrDefault has to query the full table to check if has more than one record. I'm not sure that the index invalidate that. Keep the good work and this segment of real life app implementations. 😊
Create more short codes than your current throughput, basically. If you need to fetch more than you have, you increase the batch size you generate temporarily.
Can some hash function can be used for code generation, instead of custom code generator implementation? It is less posability to have hash collisions but trade off is that code will have at least 16 characters.🙂
I have an error with migration. Are you supposed to have pre created DB? I configured the connection string in appsettings.json file with db name but it gives an error
@@MilanJovanovicTech10:17 if(!dbcontext.shortenedUrl.Any(somecondition) return code else return await GenerateUniqueCode(), i meant it like this, looks cleaner but maybe not better in preformance but also it's probably not gonna happen often
this is a non-related question about one earlier video (default interface member implementation) do you think it violates or might lead to violation of LSP?
@@MilanJovanovicTechbut oh well, speaking of SOLID, maybe you consider a series talking about those principles? uncle Bob really paid some attention to SOLID in the Clean Architecture book, it would be awesome to see your take on it, both in micro- (classes) and macro- (modules) scales
One more performance item to keep in mind: what if there are multiple requests to shorten the same URL? You then start adding technically unnecessary records in the database. It could also be worth while checking if there is already a short URL for the requested long URL and if so then rather return the existing short URL instead of creating a new one.
@@MilanJovanovicTech It could still be useful to check (even maybe in memory cache) even if context bound to a user. Use case: You have a UI which does the POST to the API to generate the short URL, there is a network or UI bug which prevents the UI from showing the short code. The user resubmits the same request mutliple times, the server then creates short codes for the same long URL multiple times. Or teh user simply loses the shortened URL and then makes another one down the line. ** We created a URL shortener at a previous company and we found that +-12% of the requests where for URLS that had already been shortened before when we ran metrics after the 1st 6 months.
@@RicusNortje If different users are requesting for the same url and you have dashboard to dispaly the link clicks for each user how will you handle this sort of condition? I think it is better to generate unique url for each request.
@@bilalmehrban It depends on your use case. If this is a public facing commercial product where click analytics are requried per user then yes you need it unique per user. If it is an internal tool to shorten for example marking URLs with lots of UTM tags then it is the URL that is important and not who made it.
@@MilanJovanovicTech I think it's a good approach and the chance of generating duplicate "codes" is very low. However, it would have been better to put a limit on the number of iterations in the your while loop and use CryptoRandom class instead of Random. Or in general, you could have used the NanoId library to generate such unique keys
- not sure what your short URL generation method can generate is unique data. - not sure what implementing BL + accessing to Data layer in controller is the best idea. Please put your code in service. IMHO
@MilanJovanovicTech Why not just insert data and get the GUID from the database and return the first 8 hexadecimal group as the code? It will probably be better performance than looping to generate the code and you will avoid the race condition.
Mister please when you release such kind of videos please do not omit anything , mention all what we need in the project because not all your subscribes are good
Want to master Clean Architecture? Go here: bit.ly/3PupkOJ
Want to unlock Modular Monoliths? Go here: bit.ly/3SXlzSt
Yes, more system design videos like this would be great, because you not only discussed the ideas, but implemented it! Thanks!
Any suggestions? I'm always open to hear what you (and others) want to see on the channel
@@MilanJovanovicTechhow about a limited online sale of tickets? Scared resource but large demandings.
Yeah. Really good video. And you showed all code from start to finish. I love nick chapsas channel too, but so many of his videos say 'i just did this or that, and wired up this beforehand', so we cant see all the code and follow along.
Excellent touch to add the section at the end where you mention the possible improvements. 10/10 video.
Thanks a lot, I'm really glad you found this video valuable. After releasing my course, my main objective going forward is upping the quality of my YT videos, and providing even more value. So I'm happy this video delivered :)
Hello again, I'm unfamiliar with Docker in general. Do you have a video on something similar to what is in this video? I don't understand the part at 11:13. Then magically the DbContext is connected to the correct SQL dB. (The dockerfile and appsettings might reveal more, but they weren't shown). Maybe it's a basic thing, but I'm a noob so 😂
@@MilanJovanovicTech can you reveal appsettings json please
Teaching software systems design by implementing them is a great idea.
Glad you think so! I'll do more of these videos, but with more details since this one was a little scarce on the "design" side
I have been working with C# since past 1.5 years and I am not sure if I will be able to build something like this in near future 😭😭. Thanks for the video, this is quality content.
You're welcome! :) And don't worry, you'll get better. 💪
Thanks for making this video. Please make more similar complete tutorial videos. ❤
Awesome! I'm glad you found it valuable 💪 Any suggestion about possible future videos? What would you want to see?
@@MilanJovanovicTech Other random real life apps similar size to this Url Shortener.... but I can't think of any! :p I did think about a web crawler/scraper but maybe that's a bit too big or complicated.
@MilanJovanovicTech Hi, Milan. You can show different patterns like command, composite, chain of responsibility or async/await pattern. I think these are very interesting patterns that would be very useful.
I enjoyed the system design aspect of this video. I definitely think a series of system design videos would be great!
Awesome, thank you!
Yes these videos are very clear and concise. Defo more sys design videos please! Great work!
Thank you! Will do!
Thanks for another great video Milan!
Additionally, I implemented a background service which generates a batch of codes and stores them in the database. When the amount of "available" codes (I use a flag) reaches 10, it starts generating a batch again. Works like a charm!
That's smart!
Can you share GitHub link for that code please ?
Great job! Thank you.
Thanks a lot :)
Really nice showcase video. These are really fun to watch, keep them coming!
Thanks, will do!
You forgot to mention that strings in SQL Server are case-insensitive. In this case, the "xyz" string would be equal to "XYZ". You can fix this by using different collations.
Now I wonder how the unique constraint will behave, have to test that
@@MilanJovanovicTechunfortunately the same way as without a unique constraint...
Best part are suggestions for performance optimizations. That is making huge distinction among million of similar videos/articles. Most less experienced developers can kind about complex topics like performance optimization, scaling in/out and also when to actually do it in whole development and production life cycle.
I think I'll do a part 2 about scaling this service, and do some benchmarking 🤔
Your videos are very clear and to the point. I way prefer your videos to the others out there (eg Amichai, Nick).
Thanks a lot, I'm glad you found this video valuable! :)
Excelent content! Looking forward to more content like this.
Awesome, thank you!
Awesome as always! Definately would love such more system design videos. How about a video on Single Signon with openiddict or ID4?
Thanks, glad you enjoyed this one! Those are great suggestions 👌
Very enjoyable watch Milan! More of these
Nice, glad you liked it!
_random.Next(Alphabet.Length - 1) is wrong. To quote from the docs, Next(Int32) returns a non-negative random integer that is less than the specified maximum. You're existing code will always omit the last character
Fair enough, because it's exclusive on the high end. Minor thing to fix.
Hi mister pls can you
Write fixed version of the exact part you have mentioned
@@NagechaO you just have to remove the -1
Yes More system design videos, like Twitter, Parking Lot & Elevator please!
Great ideas!
Great video. We need more!
Working on it!
Thank you so much for this great content. You're the best Milan.
You've almost watched all the videos 😁
Wow, that's a nice video! Strongly liked it! I'm looking to implement it by myself using Dapper, for example. It would be an interesting experiment
That'll be interesting!
Another great video!
Glad you enjoyed it!
Loved it! Finished your pragmatic clean architecture, will your new tutorial include these tips you are sharing in TH-cam. It would be better to have them to wrap up.
Hey, that's awesome. What did you think of the course? 😁
The next course I'm planning will be about Modular Monoliths. I'm currently working on a high-level curriculum. But it won't be ready until halfway through 2024.
@@MilanJovanovicTech It was well structured and explained. Both theory and real life sides helped me to easily grasp the concept. The course really shows how deep experience you have ✌🏻
Another optimisation you could do is to use a Bloom Filter to avoid querying to the database to check if the code is already existing or not, with minimal memory usage. Redis also supports this Data Structure. 🙌
What if it gives us a false positive?
Since the chance of collision is very low, we can optimistically store the generated url and let a unique index fail upon the duplicate. Then, simply generate the new url and try again.
Great video Milan! Maybe this could become a series, system design practices etc. Maybe you can implement a simple image optimaztion service that transforms .png files to .webp? I dont know the complexity of the idea or it is applicable.
Definitely, and that's a pretty cool idea
You have already done few videos about different types of architeture: clean architecture, vertical slice. Maybe it's time to do a video about onion architecture and compare to clean, and a layered architecture that is done in a correct way. Also you can make about EF Core Extensions library, it has as free as payed features. I have personally used this library for bulk operations with a great success. Maybe this library can sponsore your video 😊
EF Core extensions sounds like an awesome topic. But how do I compare Clean vs Onion when I consider them practically identical 😅
@@MilanJovanovicTech they are but still maybe you can find something to talk about.
And thanks for answering! I really appreciate that you take your time and engage so much with a community!
Nice work, another optimization that could be done is before hitting the database to check if string matches the required length and has only valid characters
Apologies, I don't understand this. Where is the optimization here? Isn't it redundant since he is the one generating the string from a list of characters that he chose?
@@daraghodonoghue_ That would be for checking when doing the GET request to redirect to the proper URL, of course this is useless for the generation of the URL
@@gusgyn I understand. Thanks!
Edit: actually I don't understand. What required length are you talking about?
@@daraghodonoghue_ He defined the length as a constant 7, so anything different than that can be ignored to don't even waste time and resources making request to database
@@gusgyn Oh sorry I completely understand now. Brain fart on my end. 😂
Good video. First think come to my mind this can be done with Hashids. Keep making cool videos.
Interesting library, I will look into it!
Please create separate series for "system design"
Will do
Decent video, but the only right answer to "How would you build a URL shortener with .NET and SQL Server" should be "Why would you want to build a URL Shortener .NET and SQL Server in the first place?"😄
For the fun of it 😁
10:00 Is that an add-on you have installed to have the "Initialize field from constructor" or have you made it yourself. If so do you have a video on how to do this?
ReSharper
Great ..nice work 🎉
Thanks a lot 😊
Do you have an idea how Google does it for TH-cam, across the world between multiple data centers that the video id is unique?
Good question. I'll take a look and see what I can find about YT architecture.
Thanks bro.
Any time
What about if we want to add a statistic per every link? Also, the end user want to view a charts with detailed info. The design of the system would be very interesting.
We'd need to expand the links with a UserId, and track the stats in the redirect endpoint
Nice tutorial
Thanks! Glad you liked it.
Hi Milan.. this is a great video can you please provide any video on low-level design.
Something specific?
Hello @MilanJovanovicTech, thank you for your response. In the context of software architect interviews, discussions usually center around low-level system design. However, I'm keen to learn more about both high-level design principles and the intricacies of low-level design, particularly in the context of a specific domain. Your insights in both areas would be invaluable. Could you please share your expertise?"
hmm... why dont use hash creation tool + salt per user? the same approach we use for passwords, but with less caution regarding length and salting iterations. Libs are already there, they are more random and stable)
Length is super important for a URL shortener. 6-7 characters is ideal.
hi, great video, but I have one question. Why did you choose FirstOrDefault method over SingleOrDefault in the redirect endpoint?
No particular reason other than I wanted to... I'm querying by a unique value, so First/Single would behave identically in terms of the generated SQL. Single would never throw an exception because the value is unique.
Hi. Thanks for the great video. I'm not 100% surge, but actually I think that using FirstOrDefault is better because in a large set of data, the query stops on the first hit, and with SingleOrDefault has to query the full table to check if has more than one record. I'm not sure that the index invalidate that.
Keep the good work and this segment of real life app implementations. 😊
How would you go about the creating short codes in advance? If you get 1000 request at once, but only have i.e. 100 in advance.
Create more short codes than your current throughput, basically. If you need to fetch more than you have, you increase the batch size you generate temporarily.
@@MilanJovanovicTechwill you use any sort of background service for the code generation in advance?
great video
Thank you!
Can some hash function can be used for code generation, instead of custom code generator implementation? It is less posability to have hash collisions but trade off is that code will have at least 16 characters.🙂
Yeah, 16 characters is pretty bad for a URL shortener 😁
I have an error with migration. Are you supposed to have pre created DB? I configured the connection string in appsettings.json file with db name but it gives an error
Try calling DbContext.Migrate, it'll take care of creating the DB (or create it yourself)
Hi Milan, why have you changed all the int to var on the for loop? There are any advantage to use var?
Just a personal preference. No impact on performance.
Hi Milan, Can you do a comparison video of minimal API's & controller-based API's?
That's a nice suggestion, adding that to the list!
Database.Migrate is not creating the table for me. Anyone know what could be the problem? It creates the DB and __EFMigrationsHistory table only.
It is migration
Is the migration created?
It's the same for me. It does not create the ShortenedUrls table for me.
Would you consider posting code for these on github? I'd like to check your docker file and a few others that don't get shown in the video.
No, I share the code on Patreon... But I also typed out everything in the video, so shouldn't take you too long to copy it.
@@MilanJovanovicTech Thanks I should have spotted that - I'll sign up.
Would returning await GeneratreUniqueCode if the code already exists be cleaner option for this? would it impact performance toomuch?
Then you override someone else's URL?
@@MilanJovanovicTech10:17 if(!dbcontext.shortenedUrl.Any(somecondition) return code else return await GenerateUniqueCode(), i meant it like this, looks cleaner but maybe not better in preformance but also it's probably not gonna happen often
this is a non-related question about one earlier video (default interface member implementation)
do you think it violates or might lead to violation of LSP?
Why not ask under that video? 😂
@@MilanJovanovicTechcuzz ahh, well... guess I didn't want ya to miss it? 😅
@@MilanJovanovicTechbut oh well, speaking of SOLID, maybe you consider a series talking about those principles? uncle Bob really paid some attention to SOLID in the Clean Architecture book, it would be awesome to see your take on it, both in micro- (classes) and macro- (modules) scales
Hello. Is there a github repo where we could use the code as example? Thanks😊
I share the source code on Patreon: www.patreon.com/milanjovanovic
One more performance item to keep in mind: what if there are multiple requests to shorten the same URL? You then start adding technically unnecessary records in the database. It could also be worth while checking if there is already a short URL for the requested long URL and if so then rather return the existing short URL instead of creating a new one.
What if they are coming from different users? I don't think they should be sharing the short code in that case.
@@MilanJovanovicTech It could still be useful to check (even maybe in memory cache) even if context bound to a user. Use case: You have a UI which does the POST to the API to generate the short URL, there is a network or UI bug which prevents the UI from showing the short code. The user resubmits the same request mutliple times, the server then creates short codes for the same long URL multiple times. Or teh user simply loses the shortened URL and then makes another one down the line.
** We created a URL shortener at a previous company and we found that +-12% of the requests where for URLS that had already been shortened before when we ran metrics after the 1st 6 months.
@@RicusNortje If different users are requesting for the same url and you have dashboard to dispaly the link clicks for each user how will you handle this sort of condition? I think it is better to generate unique url for each request.
@@bilalmehrban It depends on your use case. If this is a public facing commercial product where click analytics are requried per user then yes you need it unique per user. If it is an internal tool to shorten for example marking URLs with lots of UTM tags then it is the URL that is important and not who made it.
@@RicusNortje Totally agree on this it depends on the use case.
is there a reason you don't just use guids for the code of the url?
Yes - they're too long
@@MilanJovanovicTech makes sense :)
Can we use first 6 character of guid to make it unique?
how to add cache distributed redis in code?
th-cam.com/video/V_vdMQ5dpMw/w-d-xo.html
Why not to make the code the primary key, instead of only unique? That way it would be a clustered index and then more performant
Would be a random distribution then, versus int/long that is monotonically increasing
@@MilanJovanovicTech yeah, but would not cluster indexing it make the query faster?
Is you Docker Compose Linux or Windows?
I use Linux
I think it would be better if we create a static class of shorturlLengthNumber and Alphabet fields and use them inside it
What do we get other than moving it to a different place?
I will use Hashids package for this, So I won't need the while loop to generate the "code"
What are the disadvantages of this approach?
@@MilanJovanovicTech I think it's a good approach and the chance of generating duplicate "codes" is very low.
However, it would have been better to put a limit on the number of iterations in the your while loop and use CryptoRandom class instead of Random.
Or in general, you could have used the NanoId library to generate such unique keys
- not sure what your short URL generation method can generate is unique data.
- not sure what implementing BL + accessing to Data layer in controller is the best idea. Please put your code in service.
IMHO
- why not?
- this is a demo sample, in that context it makes more sense
Please do a rate limiter video from scratch, without using any packages.
Nice idea 👌
@MilanJovanovicTech Why not just insert data and get the GUID from the database and return the first 8 hexadecimal group as the code? It will probably be better performance than looping to generate the code and you will avoid the race condition.
Limited alphabet, comes out to ~4 billion codes
@@MilanJovanovicTech Thank you for the reply. That makes sense.
We called this permalink😊
Ah yes!
why not use the guid as the unique code
It's too long
Probably, the length of the GUID is long.
I would have personally made the method recursive until we find a non existing code.
Iterative > recursive, imho
Guys please can someone show how implement caching please ?
Check out any of my caching videos
Mister please when you release such kind of videos please do not omit anything , mention all what we need in the project because not all your subscribes are good
Don't say that, all of my subscribers are awesome 😁
👋👋👋👋👋