number 0
number 0
  • 17
  • 60 685
Introducing iroh
All these papers we've talking about? We put that research into open practice with iroh.computer.
Come hang! iroh.computer/discord
มุมมอง: 22 026

วีดีโอ

40x less overhead! Rateless Invertible Bloom Filters - Part 3 of 3
มุมมอง 1.7K7 หลายเดือนก่อน
Practical Rateless Set Reconciliation by Lei Yang, Yossi Gilad, Mohammad Alizadeh: arxiv.org/abs/2402.02668 Part 3 in a 3 part series on bloom filters Chapters: 00:00 Nerd Shade 01:36 Set Reconciliation Overview 03:08 vs Invertible Bloom Filters 05:04 Design Goals 06:40 Ratelessness 08:42 Probability Mapping 12:20 Building a RIBLT 15:18 Reconciliation using RIBLT 19:30 Conclusion
Invertible Bloom Filters - Part 2 of 3
มุมมอง 1.4K7 หลายเดือนก่อน
Part 2 in a 3 part series working up to a new set reconciliation algorithm. "Invertible Bloom Lookup Tables" by Michael T. Goodrich, Michael Mitzenmacher arxiv.org/abs/1101.2245 Chapters: 0:00 Intro 01:31 IBLT Operations 02:42 Data Structure 03:44 Data an IBLT can Hold 04:45 Inserting Key-Value Pairs 06:36 Listing Filter Contents 11:51 Yo Dawg, I heard you liked IBLTS... 15:37 The catch 17:11 l...
Bloom Filters - Part 1 of 3
มุมมอง 3.8K7 หลายเดือนก่อน
Part 1 in a 3 part series working up to a new set reconciliation algorithm. "Space/time trade-offs in hash coding with allowable errors" by Burton H. Bloom dl.acm.org/doi/10.1145/362686.362692 Chapters: 00:00 Intro 00:31 What's a Bloom Filter 02:40 Building a Bloom Filter 04:02 Querying a Bloom Filter 05:28 Parameter Tuning 07:47 In Distributed Systems
BLAKE3 and bao deep dive
มุมมอง 2.8Kปีที่แล้ว
00:00 BLAKE3 hash function 01:55 how BLAKE3 works 04:52 BLAKE3 chunking examples 08:28 bao verified streaming 11:13 bao inline & outboard 11:52 bao encoding examples 15:56 outboard encoding 17:47 slice encoding 20:48 chunk groups 23:58 size proofs 27:01 size proofs and chunk groups 28:23 append only data & keeping post-order traversal
Move The Bytes Working Group meeting 8
มุมมอง 393ปีที่แล้ว
Move The Bytes Working Group meeting 8
Move the Bytes Working Group Meeting 7
มุมมอง 130ปีที่แล้ว
Move the Bytes Working Group Meeting 7
Kademlia, Explained
มุมมอง 21Kปีที่แล้ว
Let's tackle "Kademlia: A Peer-to-peer Information System Based on the XOR Metric" by Petar Maymounkov and David Mazières: www.scs.stanford.edu/~dm/home/papers/kpos.pdf Timestamps: 00:00 Kademlia DHT 00:37 DHT, why? 02:09 keyspaces 03:17 really big hash maps 04:41 a 4 bit DHT 05:39 computers in space 07:03 keyspace colocality & distance 08:04 The XOR operator 09:46 XOR for distance 12:35 routin...
Move the Bytes Working Group Meeting 6
มุมมอง 265ปีที่แล้ว
Move the Bytes Working Group Meeting 6
mapping peer to peer locality with latency & fancy geometry
มุมมอง 892ปีที่แล้ว
Papers: • OASIS: Anycast for any service: www.scs.stanford.edu/~dm/home/papers/freedman:oasis.pdf • Meridian: A Lightweight Network Location Service without Virtual Coordinates www.cs.cornell.edu/people/egs/meridian/meridian-talk-sigcomm05.pdf 00:00 Coral DSHT, maybe not 01:06 Mapping the entire internet as a service 02:41 Meridian: Network Location Service 04:02 Latency Rings 06:20 Distance Ca...
The other Kademlia
มุมมอง 3.6Kปีที่แล้ว
A look at "Sloppy hashing and self-organizing clusters" by Michael J. Freedman and David Mazières (2002). These are the same authors of the Kademlia DHT paper 00:00 Sloppy hashing and self-organizing clusters 01:22 DHTs are the wrong Abstraction 02:31 DHTs have poor locality 04:30 Le Network 06:01 Kademlia Example 08:47 Clustering 13:50 Sloppiness 16:17 Self-organizing clusters 17:04 Cluster Di...
Move The Bytes Working Group Meeting 5
มุมมอง 194ปีที่แล้ว
Move The Bytes Working Group Meeting 5
let's replace bitswap with a two week old paper?
มุมมอง 1.9Kปีที่แล้ว
Aljoscha Meyer. 2022. Range-Based Set Reconciliation and Authenticated Set Representations: arxiv.org/abs/2212.13567 Thanks octodon.social/@expede for recommending! 0:00 - intro 0:46 - binary search over the wire 1:56 - alice & bob want to sync sets 3:16 - protocol message types 4:39 - sync kickoff, defining the set to sync 5:50 - range fingerprints 8:00 - recursion response 8:30 - data respons...
Move The Bytes Working Group Meeting 4
มุมมอง 175ปีที่แล้ว
Move The Bytes Working Group Meeting 4
Move The Bytes Working Group Meeting 3
มุมมอง 932 ปีที่แล้ว
Move The Bytes Working Group Meeting 3
Move The Bytes Working Group Meeting 2
มุมมอง 1032 ปีที่แล้ว
Move The Bytes Working Group Meeting 2
Move the Bytes Working Group Meeting 1
มุมมอง 4552 ปีที่แล้ว
Move the Bytes Working Group Meeting 1

ความคิดเห็น

  • @thebutlah
    @thebutlah 2 วันที่ผ่านมา

    What an incredible explanation. Thank you so much!

  • @Cottarino
    @Cottarino 4 วันที่ผ่านมา

    I don't think you should say that hashing the computer name, is the same as generating a random number, hashing it again will result in the same number, the hash isn't random, it just looks random.

  • @StefanoBub
    @StefanoBub 19 วันที่ผ่านมา

    Incredibly well explained. The algorithm reminds me of a theory spanwned some time ago, about the 5 grades of friendships. You are 5 acquaitances away from anyone on this planet

  • @xKaihatsu
    @xKaihatsu 24 วันที่ผ่านมา

    You did an amazing job of explaining the algorithm. What an ingenious system.

  • @X90Chris
    @X90Chris 27 วันที่ผ่านมา

    You are amazing! Thank you so much, this is really helpful!!!!

  • @mahendars1417
    @mahendars1417 หลายเดือนก่อน

    Damnnn

  • @ronakshah725
    @ronakshah725 2 หลายเดือนก่อน

    top-notch explanation, subd!

  • @chrraz
    @chrraz 2 หลายเดือนก่อน

    The stuff you mention at 11:30 about being non-Euclidean doesn't really make sense. Being symmetric is a requirement to be a metric and definitely doesn't make it non-Euclidean

  • @arushk
    @arushk 2 หลายเดือนก่อน

    this was soo coool !!!!

  • @kebman
    @kebman 3 หลายเดือนก่อน

    Now, what if you duplicate parts k-buckets over distributed systems, and suddenly you add in redundancy. If you can also add in "life expectancy probability" (for servers, i.e. for server up time, but also for server creation) into the calculation, then you can have a truly dynamic system, that can have servers pop up and "die" everywhere, without it mattering in the slightest for either up-time or service delivery times (outside of time units that are noticeable for human beings anyway). In other words, it's a super stable system that's very, very hard to break by any outside force; The kind of system that might even survive a nuclear attack, just like how the Internet was intended. ;)

  • @kebman
    @kebman 3 หลายเดือนก่อน

    I want to use this process to create a distributed and secure (as in non-censorable) messaging system. It's a direct reply to EU's murmuring about illegalizing all cryptography in the region, which is just an insane take on "security".

  • @kebman
    @kebman 3 หลายเดือนก่อน

    Ah so you just use XOR to count occurrences of "1s" and if there are too many 1s than some threshold, then the space is too close? Edit: When you find distances you basically find probability spaces at the same time. And you can check this with the probability space of other dictionary words, to make room for enough probable possibilities in every category? Incidentally, when you do polling, you'll immediately know when or if you're about to run out of addresses, so you can load-balance efficiently and start shovelling data over to other nodes/servers.

  • @jmms49
    @jmms49 3 หลายเดือนก่อน

    excellent explanation, thank you for making this video!

  • @DaxSudo
    @DaxSudo 3 หลายเดือนก่อน

    Honestly, the malware developer in me instantly goes; This would be an amazing way to check to see if a hashed password exists in my password list for brute forcing and cracking passwords.

  • @japanijoota6190
    @japanijoota6190 3 หลายเดือนก่อน

    thank you, this was a great explanation

  • @manjurhassan9145
    @manjurhassan9145 3 หลายเดือนก่อน

    This was such a clear explanation. Thank you.

  • @spaceprior
    @spaceprior 3 หลายเดือนก่อน

    I didn't understand discovery. Are you sure it shouldn't be the address of D as the hash key with routerD as a hash value? I'm not sure how it'd be used this way around but I'm probably just missing background.

  • @rickjames4727
    @rickjames4727 4 หลายเดือนก่อน

    congrats

  • @sandromartins3579
    @sandromartins3579 5 หลายเดือนก่อน

    I am shocked how simple and fascinating you made it! First time I see a video from you, but already your fan.

  • @kennys1881
    @kennys1881 5 หลายเดือนก่อน

    Really extremely insightful explanation <3 And the algorithm itself too..

  • @linusgasser5019
    @linusgasser5019 5 หลายเดือนก่อน

    Thanks for the video, I needed a refresher, and it worked grat :) There are some wrong details: - 2m55s: 4/8/16 bits are 16/256/65536 values - 5m16s: why have Dijkstra to start from 0? And why is the 'v6' stored twice? - 10m15s: the way you explain the xor distance, and the examples you chose, makes it look like it's |a-b| (see one of the comments), when in fact the xor measure gives a new definition of the distance which has nothing to do with the distance on the linear scale: 1000 and 0111 are 0001 away on the linear scale, but 0111 on an xor scale - 12m39s: 0101 xor 1101 is 1000 - 22m28s: the 'log' here is in fact 'ln' or 'ln2', so for 1e6 nodes, you need something like 20 hops. Still not too many, and still very impressive. And as you said, there are many details in the Kademlia protocol...

    • @rhylow2733
      @rhylow2733 5 หลายเดือนก่อน

      Isnt n11 placed wrong aswell, for the routing table of n5? Maybe im confusing smth here but 0101 Xor 1011 is 1110 not 1011 (18m23s)

    • @linusgasser5019
      @linusgasser5019 5 หลายเดือนก่อน

      @@rhylow2733 Yes, absolutely correct. And why the 'v6' is twice there, once at position '6', and once at position '12' is also a mystery to me.

    • @dagmarski4133
      @dagmarski4133 4 หลายเดือนก่อน

      I think everyone misunderstand what O(log_n)-hop means. It does not mean that for n nodes it takes log(n) or ln(n) hops. Big O notations only refer to how much the process scales. The true expected number of hops could really be log_n multiplied by any constant.

  • @asdfghyter
    @asdfghyter 5 หลายเดือนก่อน

    really cool algorithm! do you have any practical uses for it in your distributed systems work?

  • @MekeninzoUG
    @MekeninzoUG 6 หลายเดือนก่อน

    I stopped at 4:00 and now I'm currently working out the trivial details so I can come back to the next step of the video.

  • @deformercr6680
    @deformercr6680 6 หลายเดือนก่อน

    Dude you're my new favourite tech educator. Great energy, and very nice presentation

  • @WhoEls
    @WhoEls 6 หลายเดือนก่อน

    I don't really understand what the benefits of Kademlias K-Buckets are over Chord's Finger Tables. Seems like the same concept to me.

  • @geopopos
    @geopopos 6 หลายเดือนก่อน

    The dragon of the west? Sign me up

  • @daniel2color
    @daniel2color 6 หลายเดือนก่อน

    Just had my mind blown at 8:20 🤯

  • @daniel2color
    @daniel2color 6 หลายเดือนก่อน

    Bloom filters for the win! Thanks for sharing. This is great

  • @ROBERTOGONZALEZNAVAS
    @ROBERTOGONZALEZNAVAS 6 หลายเดือนก่อน

    I´m studing computer ingeneering and this is the best explanation i found about Kademlia. Great video, simple explanation, new subscriber. Thanks form Salamanca, Spain

  • @genxindia1154
    @genxindia1154 6 หลายเดือนก่อน

    Looks good, how does it compare to Holepunch?

  • @IgorGuerrero
    @IgorGuerrero 6 หลายเดือนก่อน

    Immediately subscribed because of the synth in the background.

  • @dude2093
    @dude2093 6 หลายเดือนก่อน

    Still no idea what this does lol

  • @darshankabariya5474
    @darshankabariya5474 6 หลายเดือนก่อน

    why xor is not working for 0001 xor 1000 = 9 but real distance is only 7

    • @linusgasser5019
      @linusgasser5019 5 หลายเดือนก่อน

      The idea is that xor is the new distance measurement. So you cannot compare xor(a,b) with a |a-b|. But you have three things which are very important for the algorithm to converge to the desired destination: 1. distance to self is 0: xor(a, a) = 0 2. the order of the nodes is not important: xor(a, b) = xor(b, a) 3. triangle inequality: xor(a, c) <= xor(a, b) + xor(b, c) 1. means that if we find the destination, the algorithm will stop 2. allows to go both ways: from the source to the destination, and from the destination to the source 3. lets the algorithm converge: without this inequality, the algorithm might jump around without closing in to the destination

  • @AtomicPixels
    @AtomicPixels 6 หลายเดือนก่อน

    What is it though? Is it an OS? Web os? I’ll check the repo

  • @GodstimeIsrael-m4n
    @GodstimeIsrael-m4n 6 หลายเดือนก่อน

    Thanks. Best explanation of Kademlia ever

  • @sharathmohan8316
    @sharathmohan8316 6 หลายเดือนก่อน

    Feels like pied Piper

  • @marinaevdokimova4681
    @marinaevdokimova4681 6 หลายเดือนก่อน

    the lesson was very useful, but his green eyes.......💔💔💔

  • @Rundik
    @Rundik 6 หลายเดือนก่อน

    "No central server required" - that is a lie. I don't know how your stuff works, but you will need at least one server for signaling and at least one stun server and a turn server if you want to traverse any NAT.

    • @remchen1582
      @remchen1582 หลายเดือนก่อน

      those are not centralized

  • @xpynim
    @xpynim 6 หลายเดือนก่อน

    wdym

  • @NoOne-jg3jx
    @NoOne-jg3jx 6 หลายเดือนก่อน

    can we integrate Iroh in Web ? building web apps using iroh and web frameworks ? like React, Svelte ?

  • @LeoDaLionEdits
    @LeoDaLionEdits 6 หลายเดือนก่อน

    I never even heard of a bloom filter before watching this series, but after I feel enlighted. Thank you for being so enthusiastic and teaching this in an easy way to understand

    • @n0computer
      @n0computer 6 หลายเดือนก่อน

      Hey thanks!

  • @afjelidfjssaf
    @afjelidfjssaf 6 หลายเดือนก่อน

    So basically reinvented p2p so traffic usage is offloaded to the users to reduce server cost?

    • @edism
      @edism 6 หลายเดือนก่อน

      Look up what p2p is, you're just saying reinvented and not making sense.

  • @Metruzanca
    @Metruzanca 6 หลายเดือนก่อน

    Wait, I was just watching something relating to Bevy and saw the FishFolk Game as a bevy example of multiplayer. Funny to see it appear here as well. It take it Iroh is built with rust? Also based named. Uncle Iroh is based.

    • @n0computer
      @n0computer 6 หลายเดือนก่อน

      iroh is indeed written in rust: github.com/n0-computer/iroh

  • @evanbarnes9984
    @evanbarnes9984 6 หลายเดือนก่อน

    How's your middle-out compression?

    • @n0computer
      @n0computer 6 หลายเดือนก่อน

      so good. we have the best middle-out compression.

  • @pedrows498
    @pedrows498 6 หลายเดือนก่อน

    How does it compare to Veilid?

  • @JanLunge
    @JanLunge 6 หลายเดือนก่อน

    Cool to see a tool for this, the devices of users aren't all equipped to store the entire state of their data so for some of my apps this would not be the final solution but having more interconnections that don't rely on a central server is always welcome. maybe this could be extended for federated software approaches as the activitypub protocol is still having a lot of issues

  • @theblckbird
    @theblckbird 6 หลายเดือนก่อน

    When I saw the channel name I immediatly thought that this was just another useless AI tool... But this is way better!

  • @maxwibert
    @maxwibert 6 หลายเดือนก่อน

    A little late to the party, but this is an awesome video and I have learned a lot from it. I have a small piece of feedback, though I’m not sure if implementing it would be helpful for your intended audience. I feel that your explanation of the triangle property leaves a little to be desired. At a surface level, the triangle property doesn’t have all that much to do with “triangulation” in the sense most people are familiar with, i.e. taking measurements from multiple places to narrow down a point’s location. **Way** down under the hood, the triangle property is important for triangulation, but only because it’s important for basically any process that involves geometry and metrics, and as far as I can tell the use of the word triangle in both of these concepts is more or less coincidental. Having not read the Kademlia paper, I have a guess as to how the triangle property may be relevant to the hopping protocol: basically a big hop plus a small hop lands not much farther away from the starting point than just the big hop. This assures us that we will never hop into the wrong distant k-bucket from our current node.

  • @cken27
    @cken27 6 หลายเดือนก่อน

    I probably need to watch one more time to understand the nuances, but I like the way you explain. Thank you

  • @Omesh7
    @Omesh7 6 หลายเดือนก่อน

    So what is it exactly?

    • @eduardosabino4874
      @eduardosabino4874 6 หลายเดือนก่อน

      what i though "serverless" meant when i first heard the word

    • @MrBarneyDinosaur
      @MrBarneyDinosaur 6 หลายเดือนก่อน

      A poorly explained p2p system. Nothing new.

    • @Omesh7
      @Omesh7 6 หลายเดือนก่อน

      @@MrBarneyDinosaur that's it?, something like pied Piper?

    • @RIP212
      @RIP212 6 หลายเดือนก่อน

      ​@@Omesh7yes. Just P2P. It's not the first project like that.

    • @Omesh7
      @Omesh7 6 หลายเดือนก่อน

      @@RIP212 ya so basically torrent??