TCP Meltdown - Computerphile
ฝัง
- เผยแพร่เมื่อ 14 เม.ย. 2020
- Why it's a bad idea to build a Virtual Private Network using TCP. Dr Steve Bagley on TCP over TCP...
/ computerphile
/ computer_phile
This video was filmed and edited by Sean Riley.
Computer Science at the University of Nottingham: bit.ly/nottscomputer
Computerphile is a sister project to Brady Haran's Numberphile. More at www.bradyharan.com
I think a discussion on the torrent protocol would be a pretty natural follow-on from this video
I would also like to see a video on this!
He should actually do more TCP attacks like TCP Reset or SYN Flood also
Did they do Tor already? If not that too, maybe figure out why torrents and Tor don't play well together.
@@deoxal7947 As far as I'm aware, there isn't anything stopping Tor and torrents working together as long as you configure the settings right, but it puts undue strain on the network so people tell you not to do it.
@@deoxal7947 torrent resolves peers by ip. if dht peers and seeders could be resolved as hidden services it would be ok. But it ain't.
So, this is how you can shart your ip over tor.
Aren't we supposed to switch everything to UDP because we can't do handshakes anymore?
The function is being renamed to ElbowBump
TCP has to be 6 feet away, and have gloves on to do the handshake now
We should be able to switch to UDP because connection are more reliable and computation is improving.
@@RussellRiker These wild and reckless RFC proposals that encourage contact with strangers. Its 1149 all over again ;-) /s
@@chrisspencer6502 Obviously you're not a Comcast customer. Or AT&T.
They provide the best network 2004 can offer.
"TCP Meltdown" sounds like a fatal unfixable remote code execution vulnerability on every single network device on the internet. I mean that was my first reaction when I saw the title.
Yeah, halfway through I was expecting some sort of buffer overflow vulnerability when the network that tcp is operating on is too reliable. I guess this term is probably older than the exploit, though.
I'm already conditioned to ignore all security vulnerabilities that have fancy names, logos or websites. ;)
Don't think of Spectre/Meltdown ;)
@@zvpunry1971 Why tho
@Deoxal I believe it was a joke/sarcasm since ";)" was used.
Even fancy Spectre/Meltdown have a real and more scientific names, so it doesn't matter either way.
I would tell you a joke about UDP but I’m afraid you wouldn’t get it.
That joke is probably older than you
I'm afraid you might not hear my laughter, either.
I want to tell you a joke about TCP
@@sutirk i got it
Matheus Kirstus I want you to tell me a joke about TCP
A TCP packet walks into a bar and says, "I'd like a beer."
The bartender replies, "You'd like a beer?"
The TCP packet replies, "Yes, I'd like a beer."
The bartender says, "here's your beer"
The TCP packet replies, "I got my beer"
The bartender replies, "I acknowledge you got your beer"
So that's your 3 way handshake.
I had a joke a about UDP, but you might not get it
@@Rickety3263 I don't acknowledge that.
@@Rickety3263 😂
7:10 I don't think the receiver sends and ACK for packet 5 until it's received packet 4. Any acknowledgement for a sequence no implies all previous packets have been received. That way if an ACK didn't arrive back for packet sequence N but did for (say) N+1 then the sender knows that all packets up to and including N+1 have arrived
Yes, when the receiver gets packet 5 they will send ACK 3 again, signaling that the packet 4 is missing. Assuming that packet 6 hasn't been received yet, when the receiver gets packet 4 it will send an ACK for packet 5, signaling it is all okay until that packet number
I was wondering this too, but I guess he forgot to mention that the two hosts may be using Selective Acknowledgement (SACK - RFC 2018, I think).
That's the optimisation he mentioned. The details are a bit too fussy to spend more time on, he already had to rush that actual problem!
Well thats the thing, countless nerds have set their fractal brains upon the method by which math can be done to determine the most efficient way to do ack packets, and ultimately since it wasnt the topic of the video, and we basically already use close to the best method possible, he decided the exact method of ack ack was unimportant and instead let it be taken care of by the compiler.
Yeah, but this is a (pretty recent) optimization over the original protocol. There have been a lot of optimizations from the inception of TCP. But the idea is the same, just tries to use less bandwith.
I'm kind of digging the electro-printer paper thingy you're doing.
Ryan N Yeah! Pro-create has lots of "paper" backgrounds. It's a fabulous app!
But that means we're missing out on the lovely felt tipped writing utensils scratching on the paper. It never really bothered me much, but I do remember comments of people who truly seemed to hate it.
Procreate for taking notes, no thank you. Goodnotes is a much better app for this purpose, in my opinion
Yes, the EPPT
I suspect that man could plug his finger into an RJ45 socket and just talk to the internet
Like R2D2 in a nice shirt
It's not as far away as expected. For instance, I'm fluent in SMTP - the protocol that transport emails.
@@yottaforce that's the most boring protocol to be an expert in
@@boogalewie7093 it's not exactly a pickup line. Years ago I had an A0 plotter in my bedroom. Didn't impress women either.
@@yottaforce hahaha okay
He could probably do it over rj11 mate lol
I've been really enjoying the channel content. I'm a software engineer where I do have programming knowledge, which does help here and there to understand quite a big amount for most video's, but it also helps me to built knowledge I don't really put time into in how it actually works under the hood, since I never really needed the information. It's just nice-to-have knowledge. This video is one of those video's where, personally for me, it's extra nice-to-have knowledge but I'm happy the channel is out there with this great, brief and informative amount of knowledge. I learned a lot from computerphile. Thanks to the channel for that! :)
There is another reason you don't want to force everything into TCP. It's called latency.
Sometimes a missing bit of data is a lesser problem then a slightly higher latency. Voice over IP is a great example, where a dropped packed results in a barely noticeable click but half a second latency causes us to constantly interrupt each other.
When a TCP-VPN has lost a single packet, all the UDP-Packets that are transported through it, have to wait until the outer TCP has rebuilt the original sequence, which is a disaster for VoIP and other real-time applications.
Ye, totally. Same goes for things like online gaming. Connection latency matters muuuuch more than perfect stability in these cases. You can make assumptions and interpolate to smooth out lost data, but getting the message in time is the difference between landing a headshot and being headshotted. ...from around a corner, where you were a while ago and the other guy's machine still shows you to be.
@@WhompingWalrus Also depends on the type of game. In other games, having the correct game state is more important. Some games use TCP to ensure this. Otherwise you can run into desync issues. And those may be a complicated mess to solve. Or take a lot of bandwidth to re-sync completely.
You gotta talk about fragmentation and MTU sizes, too. This problem gets even hairier when packets are split over multiple VPN frames.
Due to the coronavirus, all TCP applications have to be converted to UDP in order to avoid handshakes.
Also those packets should be spaced at least 10ns apart to keep the 6 ft rule.
Is that TCP window and timeout adjustment the reason why file transfers start off slow and increase the transfer rate gradually until it reaches the max bandwidth after a few seconds?
Yes!
TCP doesn't know how fast it may send data, and tries to not completely overrun the network connection. As long as no packet is lost, the speed will be increased. "Slow start" is actually the technical term to describe this behaviour.
wow. after 7 years being an IT admin I finally understood basics of TCP. Thank you for that :D
So a relaliable connection over a relaliable connection becomes less relatiable than over an unreliable one. Which means unreliable != bad or not performant. Outstanding.
The first 11 minutes were a nice and excellent introduction, but when he actually got to explain the problem, it felt as if he rushed through it. The animated graphics were way too fast and didn’t help either.
I respond to you to say that I agree.
I had been thinking the same thing. I am a system administrator who often has to do complex networking tasks, so I understand the topic pretty well. I really like this guy and this channel, but when he got to the actual explanation that was the point of the entire video, it started getting kind of muddled, and I felt like I could have explained it better. Which is rare because these guys usually do a great job. 😔
Yeah seriously the first 11 minutes should have been 2 minutes and the last few minutes should have been 10.
Im new to networking but you managed to explain this in a way that even I understood. Thank you so much for this!
Really? I thought the explanation was terrible, lost in a sea of rambling pronouns.
Really love your work guys, especially in these difficult times. Keep up the great work!
Greetings from Germany
your illustration gives more clear explanation how TCP works compare than other videos I've seen. many videos just explain 1 case 1 packet, but this video explanation many TCP packets is sent simultaneously.
"It's a bit more complicated than that, but you get the general gist of what's going on"
You assume a lot of me
The part with "TCP 1 and 2" was a little bit unclear and i think they've been mixed up at some point but i got the gist of it.
Agreed, maybe they should have said cargo and the carrier. Maybe use tunnel analogies?
Well the main point to remember is that there are two sets of tcp controllers fighting for packet sending, and the lower down one is being managed by the top level one and they dont communicate, so you essentially get dozens upon dozens of duplicates because for every lost packet on the lower level (which the top level already knows about) it tries to resend it, but the top level one was taking care of it, but now the software is making even more network traffic, because it doesnt know it hasnt been received, so dozens of duplicate packets get sent that also get *relabelled* under the top tcp header because it thinks they are new packets.
Its sort of an IP storm, but across one connection. This can be solved by making only one tcp controller, or making the tcp controllers talk to each other, but the point of vpn is it isnt supposed to know or care about whats inside of it, since its encrypted, but when it comes out the other side of the tunnel it acts like a regular packet and if its tcp, ack/nack the same way that a regular tcp would, agnostic of the higher level tunnel it used to get there, because thats the point, the data inside the tunnel is supposed to be agnostic to the tunnel itself.
If you made the two tcp controllers talk to each other they would essentially have to know critical pieces of your private information, size, order, etc, as well as not be as private any more, so instead, they can just set all the lower level packets and headers to be assumed to be unreliable, and let the *topmost* header take care of receive/ack/nack protocols.
Hypothetically you could do it the other way around too, let the bottom most one be tcp and the top most be udp, but part of the point of a vpn tunnel is that its specifically encrypted and routed so that it gets where its going and ensures its security. Im sure some nerd somewhere has tried that and figured out why it doesnt work.
Other people in the comments are talking about other forms of vpns and vpns that you can set to use specific ports etc, and the main point of this video was just that it creates a ton of duplicate packets and slows down your network because the majority of your bandwidth gets taken up by ack/nack of the two tcp controllers fighting for dominance.
You are right, he could have just said "VPN TCP" and "normal TCP".
@@RockWolfHD what is the normal TCP? Is it the one within the private network?
@@user-cj7px8ub6y Not really. With "normal TCP" I mean the protocol used for packets on the transport layer of the tcp/ip protocol stack. Instead of TCP you could also use UDP.
Because of how VPN is functioning it's also checking if the packets were successfully transmitted and is telling this the sender.
That's why TCP and VPN is a bad idea, because you would have to layers of TCP which is slowing everything down.
Listening to Dr Bagley : "I know this. Oh... I guess I didn't"
TCP is one of those protocols that always surprises you. I learned from Dr Eric Cole that the sequence numbers increment by the size of the payload in bytes; so you can find out how much data was sent in the stream by subtracting the original sequence number from the final sender's sequence number. I don't think that accounts for retransmissions but its pretty accurate.
I found this one really informative and well broken down to digest. Thank you !
This is magic. I have been thinking about this question all day after my game networking exam
12:30 till we get to the real issue, only to get a "you get the general gist of what's going on"; come on I wanted more on the protocols fighting each others, moar drama, arena battles, something... ; and that packet that went missing at 12:30, did they call a CSI brigade, did they ever find it? The suspense is intense...
I spy a Michael Abrash book on the shelf back there. That takes me back 😅
Yeah, I did notice that, too
I never occurred to me that VPM (edit: VPN) might transmit on an unreliable protocol (UDP) but this video shows how it makes so much sense.
VPM?!??
@@Belioyt I don't think it takes a Reed-Solomon codes or an explanation of keyboard topology to "decode" that very minor typo. (VPM) -> "VPN"
One small correction, tcp encapsulated in a packet is called 'segment' (layer 4 if you will). Packet is layer 3
Great explanation. I literally had to troubleshoot this problem at work and discovered exactly what you said.
Th really important thing I picked up was the VHS tapes behind the classic DR Who Blu-ray sets
The clearest explanation I could've asked for
am i the only one who was focusing more on the red machine thing on the left
Not sure what it's been turned into - may just be a decorative pattern of blinkenlights - but it started off life as a panel in a PDP 11/70; the red and purple combination is unmistakable.
I was scanning his bookcase and giving him Props for OG Star Trek DVD Collection and some kinda Doctor Who thingy collector item.
Dr Bagley, always a pleasure.
Does the acknowledgement include a hash of the packet received or something comparable? Does the receiving machine just say "yep, I received something in package no. 3" or does it say "yep, I received package no. 3 with the fingerprint 3d5f3..."?
This video is right. "TCP over TCP" is a problem. Or even "TCP over anything that does resends on its own".
In the early days of WiFi, they had the same problem: The WiFi connection is inherently unreliable and may lose packets *even if it is not overloaded*. But WiFi has its own ACK mechanism, and resends lost packets. On the other hand, the delay caused by that resending makes TCP think the connection is overloaded (the technical term is "congested"), and drop its speed. I actually don't know how they fixed it, and on what layer, but TCP over WiFi is working fine now.
The VPN problem could technically be solved in a different way, though. Running the VPN over UDP instead of TCP removes the VPN-layer TCP. It would also work to remove the application-layer TCP. You do know that the packets at the VPN server arrive all and in the right order, you just don't know when. So there is no need to fix lost or reordered packet for the part of the connection that is inside the VPN. This means: The alternative is to not just tunnel all IP packets to the VPN, but to tunnel the TCP connections using a different protocol that is better fit for the transport - and there already is, although it is not as convenient to use as a standard VPN: SSH does encryption and authentication quite fine, but it can run several streams of user data over the one secure SSH connection. You can use it to do TCP from a the software that needs the connection to the SSH client, then the SSH-client forwards the *application data*, not the *IP packets* over the SSH connection, and finally the server uses TCP again to transport the data to the final destination. This can either be statically by forwarding single ports, or it can be dynamic forwarding of TCP connections to everywhere by using the standard protocol SOCKS between the originator of the TCP connection and the SSH client.
Microsoft Press is/was still printing the Petzold book in the newer orange style? I guess that makes my old original copy "vintage".
I've learned something there. Thanks for this nice and understandable explanation!
Please make a video explaining how VoIP softwares work at the transport layer or the application layer. What are the protocols used, and how does the information get split up into packets and reassembled on the other end, and is it someone to intercept the packets?
Can you talk about some other TCP/IP quirks and mechanisms like selective ACKs, cookies, taking connections over using sequence number collissions and IP spoofing, etc?
I love his 'book'shelf...
Why not keep a list of received packets on the receiving computer as well but it then it has the time-out if it receives say packet n+1 but not n and then requests that packet again instead of constantly acknowledging packets?
I'm sure there is a reason for this for which I am too flat headed to comprehend/think of.
I wonder... What if the second TCP would use delayed ACK? And do VPNs use QinQ (802.1Q/802.1ad)?
What's that blinking box in the background
He really know how to explain the videos, he is very good as teacher.
I love his bookshelves. "Programming Graphics", "Programming Windows", a big book about GCHQ, and of course, the inevitable boxset of Star Trek TOS.
They are the two volumes of the GCHQ quiz book -- well worth getting :)
Also "OSPF: Anatomy of a Routing Protocol". Classic book. It's outdated, of course, but you get stories from one of the guys who actually invented the protocol on why they made the decisions they did.
The animations at 8:10 are a bit confusing since they suggest TCP packets are wrapped around IP packets when it's the other way around.
Is the slow down because of the double exponential back off i.e TCP2 backs off and TCP1 backs off the back off leading into exponent of exponent time slower sending by the source ?
Please do more TCP attacks!! Such as TCP SYN flood, TCP reset attack, and TCP session hijacking
Great video. Thanks. Would be nice to go into a detailed example of how the actual packets are building up. Maybe a drawing of some sort
Is this TCP/IP "throttling" defined by protocol itself or it depends on TCP/IP stack implementation (Windows, Linux etc.) - simply put, is it behaving same way on every connected device? Interesting video, thanks!
So why is udp encapsulation off by default on most systems in favor of Protocol 50?
Excuse my ignorance but what’s the flashy lights box on the bookshelf behind?
Rob Craig see my answer to V above
@@robinturner2300 Sorry don't follow you. Too many months in lockdown numbs the brain...
Rob Craig go look at the newest comments...👍
What when packet number gets corrupt? For example, packet 4 is lost but packet 5 arrives as packet 4 because of header seq. number corruption, so R receives packet 5 as packet 4.
The same applies to confirmation packet corruption (when confirmation of packet 5 arrives at S as confirmation of packet 4 due to data corruption). What in such cases?
Very fascinating and educational video
Is the led pattern on the pdp a program or rolling shutter effect?
It's a test program (it's a PiDP) >Sean
@@Computerphile okay, it looks almost too smooth. I thought leds should be either solid on or off, but there's fade. Thanks! (Great video btw.)
I love this Channel!
Very interesting. To me it seems like two simple systems interacting cause an emergent property that is more complicated. Like bots dueling on price on Ebay.
what is one the left on the book shelf?
I want to know about those flashing lights in the 1970s looking panel. Is that a strobe effect with the camera, or something seen with naked eye... what’s it for?
The PDP? I don't believe original PDPs of any model were capable of doing that. I think it's a modern replica running a demo. The modern replica PDPs are amusingly empty inside: The electronics that once took up that entire enclosure are now a tiny circuit board, so it's almost entirely empty space.
@@vylbird8014 Yeah, it's a replica. It's a PiDP-11, a 6:10 scale replica of a PDP11 that's designed to work with the simh PDP emulator running on a Raspberry Pi.
There's another computerphile video about it. You should find it by searching for PiDP.
But what happens if some part of imformation in TCP packets changes on the way? How it fix this?
There is a checksum with each packet, calculated from the contents. If it doesn't match, re-transmit is requested.
@@ignoramus3736 And in TCP, not only the content of the packet, but also the header and even some parts of the lP header (TCP Pseudo Header - RFC 793, 3.1, Checksum)
What is the box with the blinkenlights on the shelf? Mini PDP-11?
Eighties Seeker see my answer to V above
@@robinturner2300 There is currently no "above" (this is the fist comment). Too bad YT doesn't allow searching through comments. Am not reading all 300 comments here to find the answer.
Eighties Seeker sort comments newest first and look for my post
Can you also explain sctp and how it differs from tcp? Pros and cons...
Please explain how WireGuard works and how id differ from other VPNs. What make is better?
I can answer that. IPSec and OpenVPN are monsters of code, very complicated and with a ton of options because they try to adapt to every possible circumstance and future cypher that may exist while being compatible with everything.
WireGuard is very simple and easy to configure, just by selecting the hardest current cyphers and using them by default.
(In the core, at network level, they are the same.)
Hey, I put all this in a series of comments on the VPN video! 🙂 OK, you put in a lot more detail. Though 'sequence number' is a slight simplification, though not a very significant one for the purposes of this explanation.
I'm just happy to see the PiDP-11 running today 😁
This is basically the TCP waits issues that build up and causing a resource overload on the server, correct?
great lecture!
In your 6 packet example, how does the receiver know when it has received all the packets? Lets say it receives packet 1 to 5, but 6 keeps getting perpetually lost. How does it know that 1 to 5 isn't the complete message?
its a stream, you dont know that
At the TCP layer, it doesn't actually know that, because there's nothing that says what the complete set is in advance. What TCP does have is connections, something Dr. Bagley glossed over. The hosts negotiate a connection at the beginning, and they tear it down when they're done with it, using the FIN (finish) flag. This basically says, "I'm closing this connection, because I have nothing more to send." If part of the data keeps getting lost, the sender is unlikely to close the connection that way. If communication breaks down too badly, the hosts will eventually time out the connection, and if one party tries to use it after the other no longer considers it open, the latter will respond with the RST (reset) flag, which means, "I'm aborting or refusing this connection." The applications using the connection are told how the connection closed and can take action as they see fit.
UDP is also apparently used for video, where there is more speed and less need for integrity.
I couldn't help but notice all that epic Doctor Who stuff in the background.
Great video!
I really thought it would be some new exploit, but I learned something new about network infrastructure instead.
What can I say...
... Acknowledged?
Glad to see I'm not the only person with a copy of Michael Abrash's Graphics Programming Black Book. It's the largest book I own
How many times are the unacknowledged packets resent ?
What is the red screen in the background?
Great explanation!, 10 out of 10.
I think this explanation would have been more clear if you had not used TCP1/TCP2. I knew exactly what you were trying to explain and found the explanation muddy. Others may have found it incomprehensible. Instead, you should have used ITCP/VTCP for the internal and virtual layers. It is the VTCP(TCP2) that becomes UDP. The problem is in having two "reliable" networks one inside the other. There should be just one layer that provides the reliability, and it makes sense for that to be the ITCP layer because it is always going to be there. Cheers, Russ
While I do agree that it got a little bit muddy, I don't see how I and V as identifiers would be significantly better than 1 and 2.
Ah, but what if the outer layer was “aware” of the inner layer? No reason to pipe this through that without sprinkling in a bit of traffic-management brains.
Very interesting, thanks!
Start at 10:52 if you already know about TCP, UDP, and IP... and just want to know why not to run a VPN over TCP.
I thought I knew all there was to know about TCP/UDP/IP but this video proved me wrong :-)
p.s. With regard to the PDP-11/70 front panel seen in the upper left on the book shelf, I worked on the real machine back in the day; believe it or not, it sported magnetic-core memory
Good explanation.
where does he get those shirts?
What's the device on the shelf?
You can build a reliable connection on top of an unreliable connection, but not vice versa. PS the "infiniband contract" is that if you can guarantee packet ordering, you get rid of most of the overhead of a TCP/IP protocol, because the protocol does not need to wait for further packets to arrive before processing them. If you get a packet out of order, its an error, plain and simple. This can't be done on the internet, but it can be done on a local net.
Did they run out or green bar paper? It's just not the same without it.
I'd love to use UDP but can't get it to work with OpenVPN. No matter how many ports I open and how many firewalls I set up, it won't work.
Doesn't Wi-Fi have an ACKed protocol that runs underneath the internet protocol? Does that have the same problem?
never heard of acked. wifi uses tcp or udp just like any other network.
perhaps you are thinking of DHCP which handles address allocation, it uses udp.
@@judgeomega No, not DHCP, AFAIK Wi-Fi has a link layer protocol in the MAC that has retries and so forth orchestrated by the LLC.
@@BooBaddyBig so you are thinking of CSMA/CA?
@@judgeomega Nope.
@@BooBaddyBig It's possible it's wrong, but from the LLC wikipedia page:
"In wireless communications, bit errors are very common. In wireless networks such as IEEE 802.11, flow control and error management is part of the CSMA/CA MAC protocol, and not part of the LLC layer. The LLC sublayer follows the IEEE 802.2 standard."
From the Wi-Fi wikipedia page:
"Because of this, for Wi-Fi, the Logical Link Control (LLC) specified by IEEE 802.2 employs Wi-Fi's media access control (MAC) protocols to manage retries without relying on higher levels of the protocol stack."
and what about VPN encryption, if there is an error in the packet wouldn't the decryption be impossible?
The point is that the tcp in the vpn takes care of that. You will not have errors, or if it detects them not send an acknowledge packet so it will get the package again. So tcp takes care of packet loss, the underlying protocol takes care of errors within packages via checksums.
For full VPN encryption, you just need to negotiate the encryption keys and then encrypt every UDP packet. It doesn't matter that the receiver might miss some or get them out of order because it can decrypt them in any order.
What’s that machine in the back?
Did you solve those GCHQ puzzles then?
Is that shirt a penfield tiling?
Online games are transmitted over UDP as well.
i see ...the tractor-feed paper is finally emtpy ...
but wait ... we can use it as background ... nice done
Had to be used as bog roll
What's the drawing program he's using?
looks like Procreate
If you want to see an interesting experimental network, you should take a look at an cjdns on GitHub.
were I missing packets or why he repeating some parts like 3 times o.o
Whats the worst case that could happen in a tcp meltdown?
The deeper issue is that using TCP breaks streaming by frustrating the better never-than-late design point for applications like streaming.
Bagley ... Bagpuss, I see what you've done there :)
what is that pdp1170 thing there?
is it a replica?