Optimizing Communication and Networking in Database Systems - The Backend Engineering Show

แชร์
ฝัง
  • เผยแพร่เมื่อ 22 ธ.ค. 2024

ความคิดเห็น • 44

  • @codedoctor3265
    @codedoctor3265 3 ปีที่แล้ว +6

    Wow , I'm listening to your videos while working !! No more listening to songs while working nowdays !! Your videos are really helpful for senior engineers :) . Hard Core Stuff !!

  • @TheRDB46
    @TheRDB46 3 ปีที่แล้ว +4

    I dont want to tell you what to do, but would love to see you build out a system from linux distros to networking entire backends, im sure you can throw in some curve balls or ask the community, appreciate everything you do mate

  • @AbinashPattajoshi
    @AbinashPattajoshi 3 ปีที่แล้ว +1

    Super video! I applauded for ₹1,000.00 👏👏👏👏

    • @hnasr
      @hnasr  3 ปีที่แล้ว +1

      thanks Abinash for the support

  • @ofek73340
    @ofek73340 3 ปีที่แล้ว +1

    your channel is a gold mine. thank you so much!!!!

  • @batman4797
    @batman4797 3 ปีที่แล้ว +18

    "Once you add Django, mango, fango and flask and react" HAHAHAAHAHAHAHAH

    • @ritwizsinha1261
      @ritwizsinha1261 3 ปีที่แล้ว

      Yeah I also laughed hard at this ; )

  • @bitethepain1
    @bitethepain1 2 ปีที่แล้ว

    Important video with a lot of value. Thanks 🙏

  • @autohmae
    @autohmae 3 ปีที่แล้ว +2

    21:32 you assume it's encrypted. Their are no guarantees for that as far as I know. It's just 'separated from other customers'. Take for example VXLAN, not encrypted.

  • @farooqmss
    @farooqmss 3 ปีที่แล้ว

    @9:55 do you deliberately exclude java based servers?

    • @hnasr
      @hnasr  3 ปีที่แล้ว

      No, Java works too

    • @farooqmss
      @farooqmss 3 ปีที่แล้ว

      @@hnasr was just wondering, but I see you talk rather little on java and MySql, but you seem to know these deeply
      just my observation, might be wrong!
      cheers!

  • @sivasaianem382
    @sivasaianem382 3 ปีที่แล้ว

    I read portgres won't allow multiple operations on a single connection. Is it that we can’t extend this idea to applications using postgres ?

  • @julienviet
    @julienviet 3 ปีที่แล้ว +1

    On the concern of databases being able to support more concurrency, I think it is not related to the actual database concurrency like you explain, e.g you say that row level locks would be an issue.
    I believe if we are concerned by pure scalability, the important point is how the database is actually implemented. Some database can be implemented with blocking model and therefore the scalability limitation will be related to system resources. Some other databases might be implemented with a non blocking model (i.e event driven) and in this case multiplexing would make lot of sense because the database would be able to perform more work.

    • @hnasr
      @hnasr  3 ปีที่แล้ว

      lets discuss this more because its interesting.
      you either do pessimistic locking or optimistic locking, you can either do row level locking or lock escalation to table / page lock. Optimistic locking will consume less resources but in case of conflicts the db has to fail the transaction which will cause a retry that puts more load on the database.
      row-level locking consume more resources (the smartest implementation was postgres where they don't keep track of row locks in memory but in the tuple) but in case of sqlserver or other dbs its just sucks in memory very quickly.
      I think it will come back to the user's data modeling technique to do as little locking as possible to make this scale better. given that the database doesn't lock in itself ..
      we will have to wait and see how more transactions and concurrency can be handled by databases.. if you have examples of certain implementations do share. thanks

  • @MrSunil9999
    @MrSunil9999 3 ปีที่แล้ว

    This channel is like comedy show for computer nerds

  • @darshaniaudhish8240
    @darshaniaudhish8240 3 ปีที่แล้ว

    Hey Hussein, unable to view parts of your database engineering course on udemy. Section1 video doesn't play.

  • @VivekYadav-ds8oz
    @VivekYadav-ds8oz 3 ปีที่แล้ว +1

    Is it necessary to use networking to access DBMS, even if the application and database are running on the same computer? Wouldn't the overhead of ACK packets and handshakes be unnecessary in that case? Also in real world, how often is it that database and applications are hosted on different computers? (If you couldn't already tell, I'm a complete n00b).

    • @shravandhar6169
      @shravandhar6169 3 ปีที่แล้ว +1

      Almost all the time. The DB's IP should not be known to the end user. It can be configured in a VPS and isolated to a decent level.

    • @khalid5d3
      @khalid5d3 3 ปีที่แล้ว +1

      Even if your application is just a webapp or a webservice, you might think you can just host both the app + db on the same host machine. But the thing is, if your app needs to scale to serve more requests you then start to upgrade the specs of your server host, this is called vertical scaling.
      But at some point vertical scaling will not be enough, if you reach your max performance on a single machine. You will need to horizontally scale, that is to run the same app binary on multiple host machines.
      And here you have multiple app hosts connecting to a single dbms hosts. How to do that? Well, TCP or whatever.
      This is just one aspect (the performance). Not to mention other problems which, in order to solve them, you need to have multiple hosts running your app. Example: uptime SLA, resilience, etc....

  • @HarshKapadia
    @HarshKapadia 3 ปีที่แล้ว

    Really nice video, Hussein! Thank you!

  • @nitishbhatia25
    @nitishbhatia25 3 ปีที่แล้ว

    Wishing i can give more than 1 like to this video.
    1 doubt.
    UDP is layer 4 and they built quic over udp which is also layer 4? Can u please clarify this?
    And how do they get away with the problem of ossification when introducing quic at layer 4. Any good resource to understanding all this more clearly would be awesome.
    Please keep making such videos where you think out loud. It’s beautiful.

    • @hnasr
      @hnasr  3 ปีที่แล้ว

      TCP and UDP are layer
      I think this video helps th-cam.com/video/7IS7gigunyI/w-d-xo.html

  • @sumitrawat2037
    @sumitrawat2037 3 ปีที่แล้ว

    27:24 that's not a silly example. I remember in junior school our maths teacher would encourage us to learn multiplication tables of upto 20 numbers BY HEART. And in any case a student cleverly regurgitate it using logic in his viva, they would deduct the marks. NOW THATS SILLY !!

  • @Taklsdjw093
    @Taklsdjw093 3 ปีที่แล้ว

    If it is not Already exist, we need to have hardware acceleration for streams/channels. Just as we have fireworks celebration for AES which allowed encryption to be more ubiquitous and cheaper Computer resources.
    Multiplexing support at the water level would solve a lot of this

  • @MrEtronic
    @MrEtronic 3 ปีที่แล้ว

    Hi nasser i think the problem has been solved only one level lower in the NVMe protocol and NVMe over fabric , although i dont know how they do it . I tried reading the spec and they dont go into MVCC. but it would seem they were successfull in implementing concurrent writes over a network.

    • @dvskr1
      @dvskr1 3 ปีที่แล้ว +1

      NVMe is on top of PCIe ..It is for peripheral connecting right and not a network I/o protocol..I am not sure if things have changed recently,I have worked on NVMe storage connectivity few years back

    • @MrEtronic
      @MrEtronic 3 ปีที่แล้ว

      @@dvskr1 nvme over fabric is a thing . So I believe they might have something like iscsi for nvme.

  • @utsabbanerjee9672
    @utsabbanerjee9672 3 ปีที่แล้ว

    Food for thought

  • @autohmae
    @autohmae 3 ปีที่แล้ว

    7:38 euh... SQL Server supports SAML ? Yes, it support SSO with Windows Authentication (SSPI/Kerberos usually). But SAML is a web protocol (yes, used for SSO, but not for SQL Server as far as I'm aware).
    18:15 My guess has always been, it's also to avoid TCP slow start

    • @hnasr
      @hnasr  3 ปีที่แล้ว

      Ohh I misspoke thanks for correcting me,

    • @autohmae
      @autohmae 3 ปีที่แล้ว

      @@hnasr Yeah, I assume you did. But it does show how much 'web-centric' we really are :-)

    • @hnasr
      @hnasr  3 ปีที่แล้ว

      So much webby stuff

    • @autohmae
      @autohmae 3 ปีที่แล้ว +1

      @@hnasr If anyone doubts where the future is going: Microsoft is planning and working on making Outlook a Progressive Web App (PWA) and deprecating regular Outlook.

  • @gradientO
    @gradientO 3 ปีที่แล้ว +3

    Pls do a video on that Instagram story hack by @pgtalal, MrWhoseTheBoss did a video saying he did HTTP proxy interception or something. But I think you can cover more on that. Pls look into that

  • @ikonxTech
    @ikonxTech 3 ปีที่แล้ว

    thanks man you the best

  • @Marek13nagy
    @Marek13nagy 3 ปีที่แล้ว

    Hello, have you considered doing a podcast from these kind of longer videos?
    Maybe some people would like that idea. What do you think?

    • @hnasr
      @hnasr  3 ปีที่แล้ว +2

      I do actually have a podcast, check it out husseinnasser.com/podcast
      Maybe I should start announcing it more thanks for the reminder

  • @chandeeparora.7165
    @chandeeparora.7165 3 ปีที่แล้ว

    Hey Hussein, Can you please do a video on system design for a stock ticker application(like the one you did on twitter design)? Impromptu works :)

  • @siya.abc123
    @siya.abc123 3 ปีที่แล้ว

    Lol yesterday I had a weird error connecting to sql server from my api. "The server does not support the requested protocol."

  • @Serpher1
    @Serpher1 3 ปีที่แล้ว

    So HTTP/2 for something like Tomcat/postgresql combo is bad.

  • @lambdamax
    @lambdamax 3 ปีที่แล้ว

    I hope we also learn about the TH-cam outage some day.

  • @Euquila
    @Euquila 3 ปีที่แล้ว

    I just want to hug your mind lmao

  • @hitmusicworldwide
    @hitmusicworldwide 3 ปีที่แล้ว

    What does Netflix have to say about this