As an engineer, it's hard to keep up with all tech all the time, so I like your direct no thrills and frills, to-the-point approach. I appreciate it. Additionally, no stupid music or other background effects that only serve to annoy and distract. Refreshing. A very useful and informative video. Thank you.
i dont come near as engineer, but, those things really can make the video quality worser (in my opinion) i think for almost any video. like that, that explains, maybe tutorials you along something.
From 320x240 to HD to FHD to 4k to 8K, we 10000 fold or 1 000 000 fold increrased data rates, but NO INFORMATION added to the 320x240 small few MB video. So out society do not goes to future with some new information, that can change something. Look at Russia, they do things effectively, with 1000 - 10000x less budget can get 10x more powerfull wepaons and more precise weapons, that can penetrate Patriot or IRIS or MANTIS sytstems. So a lot of traffic a lot of GB pre sec. does not mean we have more info.
Thank you for breaking it down like this. Almost nowhere on TH-cam people go into this kind of detail because the video maker will just assume their audience knows these things.
a couple of things to bear in mind: -10gbe copper SFPs generate a LOT of heat and use a lot more power per port than DAC or fiber so avoid them where possible -singlemode and multimode are about the same price now anyway, and singlemode is futureproof while multimode is not, so if you're going to put in fiber may as well go singlemode, future you will thank you
THIS-^ Ditching copper SPF+ virtually removed all my problems with cheaper gear like the Mikrotik..problems like it going from not working in a full 10gig environment to it never dropping a packet. well, almost never
Are you not worried about overmodulation by using singlemode on a short distance? Optics for singlemode on higher speeds are way more expensive too. You can get up to 40 Gbps over 150 meter of 2 strands of OM4 using 40GBase-SR-BD and up to 400 meters if you use 8 strands with an MPO/MTP connector.
The older adage was that you would use Multimode for very bendy applications (it works when twisted around a pencil) and singlemode for long, straight runs. However, modern Singlemode fiber has a minimal bend radius of 1cm (slightly under half an inch), so that is also not a reason to choose multimode.
I just put together my own personal Rack and homelab at home. I've been working and building them for work for years now but finally got around to building my own. I went with a mix of 2.5Gbps and 10Gbps. 2.5Gbps is for all the everyday PCs, access points, etc., 10Gbps is for my NAS and Workstation as well as my custom built Pfsense Firewall (my ISP Internet is 10Gbps Fiber). So far so good. Came across your video here as I was curious about 25G networking (already thinking about upgrading and I finished the setup yesterday). Subscribed. Good info.
It's cheap enough to go 40Gbps/100Gbps by buying an out of warranty enterprise switch now. I run Single Mode fiber as transceivers now are better at not getting burnt out over short runs. Cabling is not the expensive part anymore....
When I was working with IT, 10 Mb/s was all the rage, after years of RG58 cabling the common common copper wire made all the difference. At the time, embedded controllers weren't a thing yet, so the ethernet hubs were repeating the same packets on all ports, and the machines had to run a collision detection on their own. Some gear used token ring on ethernet but we didn't. Only when microcontrollers became powerful enough to check what was in some layers of the packets that we finally got the "switch" - where the packet was addresses to the port with the associated address. From that point on, on a network without collisions, the race for higher speeds made sense, and we had Cat5 (100 Mb/s) in the blink of an eye. Thank you X the explainer Dave. It is happening all over again!
The first home networks I ran used 'thin ethernet' where if there was an issue _somewhere_ in the big daisy-chain, the whole network went down with no indication of where the fault lay. 10Mb RJ45 was _such_ an improvement. (A few friends and I installed it when, as students, we moved into a 6-person house. We had a spare 486, so that got put in the closet with a modem so that all of us could share a single 56K internet connection -- amazing for the time.)
I built a 10base2 network when I was a first year at university. The student accommodation didn't have networking so I built our own. Network switches were very expensive back then so we used the cable only 10base2 system. Didn't have any trouble with malfunctions, but I made all the cables myself very carefully.
Ah, the horrors of troubleshooting standing waves on 10base2 networks. Every time someone plugged or unplugged any PC, some PCs would end up in a dead zone and drop off the network, so I'd spend all morning crawling around under desks and work benches, adding a meter here, removing a meter there, trying to find a combination where every PC could se a signal. The first network upgrade I ever did was to connect our 10base2 network to a cheap 8 port 10baseT hub, along with my PC and the company 'file server' under my desk. People weren't convinced that swapping one 10Mbit network for another would make any difference, but once I'd proven how much more reliable my connection was, I got the budget to buy 6x move 10baseT cards for everyone in the development office. Within a year, I'd convinced the director to fit structured cabling throughout the unit, and I was able to upgrade us to switched 100Mbit throughout, so the admin office could get the benefits too. I even got them to stump up the cash for, a *real* server (with a DAT drive for backups), a UPS, and replacement PCs for all of the seriously outdated 486 PC's that designers were still using. *8')
Great to hear! I'm by no means an expert, just summarizing what I learned in doing the process for my own place! I'm sure true fiber experts found mistakes, but hopefully it's pretty accurate!
Another amazing trip down memory lane. I worked in a major corporation who still had token ring in 1999. Thanks for a couple of triggers, they highlighted what it means to be on the spectrum. Appreciate all you do!
I worked for that that major corporation as well (probably). Joined as a tech illustrator (software manuals) using DrawMaster, and ended up building computers with Token Ring and OS/2. Had the first luggable 486 with the red screen.. It just fitted a short T/R card !
All this stuff seemed complex as hell and this one video made everything clear, I already spent a lot of time trying to find information about eg. DACs, fiber types, transceivers etc. Thank you a lot, this saves lot of browsing.
Very useful, educational, and lean video. I watched it twice and enjoyed viewing it each time and Dave makes excellent presentation with no fluff. ... Excellent
Fast home networks are fun 😁I made the 10G jump in 2016, and in 2017 I upgraded to a 32 port 40G Arista switch grabbed off ebay for very cheap. Since then I also scored a few smaller mellanox 12 port "56G" switches for dirt cheap. Naturally a large chunk of my home network is running @ 40Gbps - including the gaming/home theatre PC because why not, got the ports right :) Uh, you can say things have gotten a bit out of hand. Now .. its time to start a 100G upgrade... I might have a problem. Unfortunately the crazy good deals are less common these days due to the still ongoing supply shortage on new enterprise grade networking equipment, making data centers and enterprises hold on to their network gear and more companies buying off the used market, driving prices up. Also many really good deals comes with some pretty severe caveats that takes some .. I mean a lot of work to overcome. Def not for everyone.
I had issues with fan noise acceptance in the house for anything enterprise. I went to 10GbE in 2020 when the early QNAP consumer level 10GbE switches came out... then got a second one. I was in a condo at the time, so I didn't have a good place to hide loud equipment. I'm in a house, now, so I can pretty easily dedicate a closet and isolate the noise pretty well, so I might do something faster... or just get a bigger, dedicated central 10GbE switch and keep the QNAP switches for edge switches. Though they don't do 2.5GbE at all, but small switches with a single or pair of 10GbE SFP+ ports and a few 2.5GbE UTP ports have gotten super cheap... Well, QNAP now has a 16 port 25GbE consumer level switch that's not particularly cheap (currently $1200), but supposedly quiet. So... I guess I'm running fiber everywhere. (Yes, I realize I'm commenting on a 10 month old comment, but a lot is still about the same.)
THANK YOU! I really would have needed a video like this before I started with 10G... This is a great introduction and I'm sure will help many understand how it all works!
I just stumbled across your channel and I'm so happy I did. I'm in the process of upgrading my home network to 10 GB and your easy to digest explanation style is perfect. You definitely gained a new subscriber.
Dave, thank you! You saved our startup thousands of dollars with this video. We were under the impression that we would have to purchase a costly (Mellanox) network switch to connect our server to our SAN. On a side note, I remember being 11 years old setting up my first token ring coaxial network in my house. Those were the days!
I ran a Farallon Phone Net using AppleTalk down the hallway between my Mac SE and my sister's Mac (forget model). It worked, but I never had anything to send to my sister.
yes, most programmers I've worked with are dumb AF when it comes to hardware and some even didn't master basic OS functionality (but he was a great SAP-consultant). I always teach my younger peers that every IT'er is also a user. You can be adept in one field, but a total noob in another, and then there's Dave 😆
I remember the good old days using Coax in ring topology. One loose cable on a single computer that got bumped loose and the whole network on the ring goes down! I'd be crawling from PC to PC to figure out which one was loose. Changing to star topology was a pure dream.
According to Cisco. 100m cable is made of these 3 cables. 4 m from switch to patch panel. 90 m from patch panel to connecting point going inside a wall and shielded cable if preferred. And then 6 m from connecting point and to user. Indoors, the recommended max is 100m (330 ft): 90m solid plus 10m patchcord allowance.
In the multi-mode vs single-mode discussion, I was under the impression that multi-mode's limitations aren't from signal loss per se, but the fact that the signal gets "smeared" across time from light taking more or less time to reach the other side depending on how much it bounces. If you try to hear someone speak in a room with lots of echoes, it's very difficult -- not because you can't hear the sound of their voice, but the fact you're hearing the end of one word at the same time as the beginning of the next.
Modal Dispersion becomes an issue with longer runs with multi mode. It also has to do with the type of light source is used, the grade of multimode cable. Single mode on the other hand doesn't experience modal dispersion. Multimode cable is more expensive, but the optics are cheaper. Singlemode cable is dirt cheap, but the optics are more money, and depending on the required bandwidth at the time of an upgrade, it may be necessary to replace the multimode with a higher grade (especially if one has 62.5um multimode installed and the upgrade path is 10G or higher. The 2 benefits with single mode is the fact that with single mode, the maximum bandwidth has not been reached yet, and in the case of service providers you can multplex many "colors" or wavelengths on a single strand, in the enterprise it is usually used in the form of bidi optics that use a single strand for both transmit and receive.
Yes, different wavelengths have slightly different path lengths. Over longer lengths you get a complete screwing up of the signal. Transceivers don't produce just the wavelength of light they're rated to, but bands at either side. So as the path differences build up, as a shorter wavelength bit is going low it'll be swamped by a longer wavelength from a different point in the signal being high. The receiver just sees this as a "high". In multi-mode the more gradual change between core and jacket means this smudging is more pronounced. With single-mode the path lengths are far more uniform (there's a much sharper transition from core to jacket, so all of the wavelengths reflect at once), so that smearing you pictured takes a lot longer to happen- tens of kilometers. Multiple close-but-distinct wavelengths can be crammed down the same fibre with no ill effect using WDM/CWDM/DWDM. In 18 years of working with industrial fibre systems, multimode has been rare. The cost savings just aren't worth the lack of flexibility, range, and multiple-wavelength functionality. Multimode seems to be common in more pure-IT industries where things like FORJ passes and distance aren't a concern. My house is also single-mode so it'll work fine on smaller scales too!
The discussion will not be important soon. The production volume of single mode is so big in China that it drives down the price of the whole thing to multi-mode level. Thus there is no incentive to use multi-mode. The effect will spread to the whole world several years later. It will be all single mode just because of price.
Multi-mode is easier to work with, but once cables are cheap enough that you just replace them if they fail, rather than trying to repair them, that doesn't matter so much.
Just for those who are new to this: SFP+ connectors are made for 100 to 500 mating cycles (the range of numbers you will find in different datasheets).
Back in the late 90s i didnt own a switch or a hub. I had a cheap rj45 1 to 2 splitter. And we shared our dial up from the host computer. And played quake 2 and various other games with 3 computers this way, as long as host was the pc on the side of the adapter that didnt split. And some years prior to that I vaguely remember playing a bit of doom over coax cables with bnc connectors with my dad. Talking about networking brang back some memories. Thanks for the nostalgia trip Dave :D
If you're running a machine with PCI slots modern enough for those NICs, MDIX will probably already be in the stack. Regardless, I love the crossover callback. It's important for the younger generation to still understand these fundamentals I think. I love both the tech and videos about autism you put out, equally! Thanks!
MikroTik Routers have a fair price tag, but are not very easy to administer. Nevertheless a good solution for a homemade lab. Unfortunately, it's not very easy to get rid of all bottlenecks, e.g. I can't find a 10g card for my notebook...
@@VraccasVII I'm running a couple of cloud instances for free, so today, I removed my server rack completely from my home office to have a bit more space available. That's probably not forever...
Wow . Once years ago, had to setup trade fair, with fiber optic cable runs , it was fast for sure, but disassembling the runs without damaging the miles of cable well was tedious at best. Our in house ran over just two fiber feeds , speed was impressive. Well thanks again for teaching me something new. Take care.
Re: multimode vs singlemode - the cost difference has really come down in recent years, to the point where the cost difference for a home setup is probably only a few dollars. A lot of folks have started recommended that any greenfield deployments standardize on single mode for everything for simplicity. Since singlemode can do everything multimode can, and then some, you don't have to worry about where to use which type of cable.
I am autistic with ADHD I haven’t been diagnosed but both of my children are I appreciate your videos . I suffer from short term memory. I have difficulty reading and writing so it’s hard for me to express my feelings. I love your videos and I always look forward to your videos. Thank you. You mean the world to me. I always look forward to your videos.
What drivers are you using and what operating system do you use with your PCIE 10gtek card? Windows 11 doesn’t support those cards anymore as far as I can figure out. Only win 10?
Wonderful video! I am a new subscriber here, but I work for the nations largest telecommunication provider. I have 5 Gbps symmetrical Fiber Internet at my house and I find it difficult to fully utilize that connection speed. For work I install & Repair FTTP or Fiber To The Premasis utilizing SFP+. We use standard, alternate, and XSGPON SFP modules. The difference between the three are optical frequency and XSGPON is the SFP+ Module that is capable of 5Gbps Single Mode Fiber. I also install & repair FTTN Single & Bonded pair DSL connections as well as IP-RT IP-CO internet connections. I truly love your videos! The way you explain everything is very unique!
Still the best 'second' video for introducing people to home networking. If they get interested in having a home network, better to get them introduced to everything and the future. More speed is better.. (Secret-note; first video is 'Warriors of the net'. If they survive that, they will have the right determination to actually use good network advice).
Thank you your explanation has made my life as an almost networking lay person who’s trying to upgrade slow cat 4/5 domestic network to fibre so I can improve the data speeds for all my devices TVs etc
This was very insightful with perfect timing. I'm developing a 25G fiber multiplexer at work (three 25G fiber ports over 1 fiber), and was looking for software to use to test throughput. Iperf looks like a good option. Good work as always.
Loved this video, took me back to my days working in the 90s when we migrated my office building in London from cat3 to cat 5 with fibre between the floors. That project taught me a lot. It was running 16mb/s token ring with in a year I had moved the net builder 2 to the edge of the lans and replaced it with a pair of token ring switches with spanning tree. So each floor had a redundant link.
Been working at a small ISP for some time now. We use SM BiDi (one fiber for send and receive) for Fiber to the Building and there are cases, where the signal blinds the receiver, if the distance is too short and/or SFPs get mismatched. Also, as the beam is much more focused and usually has a higher frequency, which translates to "way more 'impact' energy", the risk of permanent eye injury is much higher with SM when mishandled. SM can reach as far as 200km (120mi) with carrier grade SFPs and you might not even be able to look at it twice before running out of functional eyes. Therefore, MM and TP are still valid options for short distances and/or hobbyists and unless you want to go faster than 10G, upgrading an existing inhouse copper network to fiber gains you nothing but trouble. (Exceptions apply.) When I have to talk to a customer about that topic, it's mostly about the widespread misconception that fiber would be faster, better, stronger in any case. In reality each SM, MM and TP have their pros and cons. 10G are 10G no matter the physical medium, pings are the same, too - even with a media converter in between. Btw, broken fibers still DO happen, especially if you are a proverbial tinkerer. At Consumer Grade, hardware with 10G support is rare and so are the use-cases for the average Joe. I see on a daily basis what people have connected to their home/soho routers and more than 80% - small businesses included - won't be able to make use of even those 10G. Consider yourself well equipped if you have devices that are capable of 2.5G without bottlenecks apart from your LAN. Now, if you belong to the other 20% and you know what you are doing, your mileage my vary. In this case, I won't have to send you a technician (and then forward your ticket to Billing) and you can simply ignore me.
I have 2 GB fiber optic internet to the house, that's what I stream you on. Just like you I have that weird fascination that "The future is actually here" since I'm using fiber optic for my internet connection. Directly to my house. Fiber optic internet. Blows my mind. I still remember hearing about Netflix, You know, the new DVD mail order place, talking about streaming video to everyone's houses and laughing because there's no way everybody is going to get on cable internet. And DSL can't handle it either.... And here I am with a freaking fiber optic connection directly to the backbone. It blows my mind.
Dave did I hear you say you had a screw loose? Lol somebody had to say it. As you the single mode, fiber optic, installing a connector on a fiber run , is a job for experts, the alignment critical, tooling used expensive. I was allowed to try it and got lucky ... had little interest in doing it again, the tech was impressed, I of course acted unsurprised at my success . Since such a rig is well beyond my resources , I shall go buy a lottery ticket. The two dollar dream ... will try to get a copy of your book, had a friend that suffered, and passed young. Before his last trip in, he stopped by and wanted to give me his 53 pink Lincoln, I was shocked it meant so much to him. I told him I'd store it till he came back. No he said I want you to keep it. . He didn't come back, I gave it to his mother. Why ...it hurt to drive it, my friend was gone . Natural causes ? Drug experiment ? I'll never know. The program was experimental . Have not thought about this for almost 50 years ....take care keep teaching
Man your videos are awesome. I have not done networking in years but ready to make my own network since I am into video editing and also clothing data files for my mother business. I guess time to need out. Baby steps but you are a wealth of knowledge thanks.
100G and 40G ports in switches can often be broken out into 4x25G or 4x10G links (twinax or fiber, not twisted pair). Computer NICs typically won’t break out that way (some Mellanox is the exception)
Afair Intel 710 can do bifurcation too. Do you know any interface, that would provide hardware LAG support? Mellanox can do hardware LAG offloading for VFs but I just need card that would be LAG aware. Simplifies network a little.
It’s typically the OS that does it rather than the card. You can create a team in Windows server, or a LAG in Linux. In VMWare you need a distributed switch to do an LACP LAG
@@ausnetting most of the time it is OS. However, some interfaces do have internal switch fabric. In Intel 710 this fabric is only there to allow connectivity between VFs (as access switch typically would not reply on same interface, as packet was originated from). Mellanox ASAP2 engine can do LAG for VFs without exposing that fact to guest or host machine. It surely works in Linux (aka Proxmox) but have not found (yet) info about Windows support for configuration interface
I've been wanting to upgrade my 1GB ethernet for 3 years now. I've been told 2.5G is cheaper than 5GB & way cheaper than 10GB. I haven't look into it for at least a year and now after watching this video I've decided to take the plunge and finally upgrade. To what I have yet to choose. Every time I see and read instructional videos like this one I learn a bit more of what's needed. Brand names is what's keeping me from purchasing. I've read reviews of NIC cards that were name brand but had many complaints of burning up.
One thing you appear to have almost mentioned, is that fibre patch cords are always crossover. In fact even the in the wall structured cables are crossover too. When you connect them, either directly, with a single cable or between rooms, with the in the wall cable & 2 patch cords, you will always wind up with a crossover. You can see in this video, one fibre is yellow and when compared to the other end, it's the opposite side of the connector. Often, when you get a patch cable, the two connectors are loose, with a plastic clip to position them. With that, you have to make sure to cross them, if needed. That 15 pin connector you said looks like a game port is for the old "ThickNet" 10base5 cable, which was used with a transceiver. That connector is for connecting the transceiver. Even some 100M NICs didn't need a crossover. The earliest LAN I worked on used triaxial cable. That's a cable with 2 separate shields. It was a proprietary Rockwell Collins network on the Air Canada reservation system in early 1978. It used Time Division Multiplexing (TDM), instead of packets or frames. It was also a ring and ran at 8 Mb. BTW, some vendors will resell returned equipment and will shrink wrap it again before selling it.
Worth mentioning that there are also AOCs, which can be used similar to DACs but for longer distances. AOCs are still usually cheaper than buying two optics. The downsides here are that if the cable gets damaged, you have to replace the entire cable, where as if you used two optics and a cable, you only need to replace the cable. Additionally, for 100gig+ over multimode, MPO cables have started becoming the norm, but these are just bundles of normal multimode fibre pairs. This means they can also be used to run multiple fibers between locations, for example running fibres between floors in a building. I've also used these with MPO cassettes as breakouts. You install a 100gig switch, with QSFP ports on one side, plug in a MPO cable, run the cable to a different room, connect to a cassette, which splits that MPO into normal multimode pairs, which can be used for up to 4x25G or 4x10G devices.
I started a job in WAN last januari. We're using OS2 single mode for distances up to 80km, at 10, 40 or 100 Gb/s per fiber. Some SFP's will reach out to 120km, but we're not using those. Never work with live fibers! The lasers will cut and burn you, seriously! There's a lot of energy going down a fiber that needs to reach for kilometers. And you can't see them, they're outside of the visible spectrum. Get the distance wrong, and you'll burn out the SFP on the receiving end. Also, always have a ferrule cleaner handy. It's good practice to clean the ferrules every time you make a connection.
I would love to see a follow up video where you enable RDMA on you NICs. This should give you the same performance but without the CPU overhead of processing each packet when using SMB Direct (file copy). Also you should try out enabling SR-IOV on your VMs. This will give your VMs direct access to the NIC.
That is if the NIC support those features. I bought several 10/25 gig Mellanox Connect 4 dual NIC cards from E-bay on the cheap a couple of years ago. I am not sure if he bought the 4 or 5 series so it will vary on model.
RDMA is huge offload. However, you have to be cautious with SR-IOV. It gives some advantage in demanding applications on VMs and there is no denial for that. However, I have experienced issues with handover in HA Hyper-V cluster. Network connectivity was interrupted whenever handover happened and we were never exceeding used VF count on particular interface. Also, windows guests were unable to utilize failover networking. I have ended up with plain, old LBFO teams for VMs. I do prefer machines to be 1% slower in high-performance networking (not a big deal, when half of people are working remotely anyways) but without unnecesary downtimes.
@@LampJustin If your Nic supports iWARP then you don't need PFC. Also Mellanox has some newer Technolgies (ZTR) that lets you get away without configuring PFC.
@@brylozketrzyn Agree about SR-IOV with HA clusters. If you only have a limited number of VMs that use SR-IOV and ensure that there are enough VFs available on any given node then you should be OK. I would also avoid LBFO as it is end of life. I don't think SR-IOV makes your network any faster, but it reduces CPU overhead somewhat.
Just wrestled with 96 25gb Twinax cables for 24 servers across 6 racks. Of course, it was just plug and play as the cards and OS support 10gb and 25gb without drama :) Just another datacenter day!
for future proofing, single mode is the same it’s been for 30+ years and still handles terrabits per second (with DWDM transceivers). Go single mode if you can.
First time viewer, absolutely loves the content so far. I'm glad I got to see this. You've explained every little thing and definitely can tell you've put a ton of research beforehand. Absolutely worth subscribing. Thank you.
This is not the era of hubs anymore. However some devices have "hidden" switch inside to increase number of ports, so you may have your bandwidth divided between downstream devices (yes, I am pointing at you, Mikrotik)
Fellow old guy, that's wanting to start learning networking, here! Just stumbled across your wonderful jem! You got my sub and like! Looking forward to looking and learning more! Thank you for sharing your experience and knowledge! 😮
Another old guy here who was old when I went back to school a technical college at age 47. The networking aka Cisco class was my favourite difficult enough to push me but not so much that I didn't learn. I'd suggest in-person classes if you can if you're like me you need a push. At the very least Cisco Packet Tracer if you can get it to work without an account and the labs many of which are out in the wild to follow along on.
@@dg-hughes I've installed it and poked around on a couple of times. Certainly looks complex and stocked pretty good! Looking into those labs, pretty cool that we have so many cool "modeling" options that can get so close to real world practicality.
You should check out the 400G-ZR optics. They can do 400G, up to 120km. Combine this with the fact you can use different frequencies of light for DWDM, and an optical combiner/splitter, with up to 96 channels, you can get 38Tb/s down a single fibre. The QSFP optics do cost around £35K each though
There's some amazing fiber tech out there that's way above MY paygrade! I believe they can shoot multiple lasers in and extract them at the same bounce angle and this run multiple streams of light in a single tube, all kinds of crazy things! And it's not on/off, they can use multiple level of brightness and so on...
And now even 800G optics are being introduced around one of my work sites. That requires PAM4 modulation on 8 lanes of 106Gbps with QSFP-Double Density transceivers. From here on with 200G lanes it's going to have to be all optical, bypassing the electrical interface and piping fibre directly into the switch chip on the board using silicon photonics. If you have to ask - like me you probably can't afford them - well just yet anyway :).
I wish I were working with you,learning from you, Helping you in all those networking things,😢 I Like devices,computers and all those stuffs. You’re amazing Sir.
Back in the earlier days there was an electrical difference on the tx and rx sides of the transceiver, and the Ethernet isolation transformers had different winding ratios for each meaning that you had to use a crossover cable to line up tx-rx. These days, electronics are better and more capable, and Ethernet isolation transformers use the same winding ratio for "tx" and "rx" meaning that the transceiver can use tricks to automagically rearrange them.
Also beware of SFP+ FC2, these are 16/32GB Fiber Channel or iSCSI and look identical to a typical SFP+. You will get traffic over them, but they will usually default to max 1GB Ethernet, especially if they are HP or Dell branded (they don't make them just slap their label on them).
To make it even more confusing, some modern cards even support both SFP types, but that doesn't mean it works well, or at all. (FC and Ethernet are 2 completely different "languages").
I highly recommend you use single-mode fiber even in your home. You will never need to run a new fiber again vs multi-mode which may at some point need to be replaced to get higher speeds. Costs between multi-mode and single-mode are very similar now a days. I did a write up about my setup but I can't post the link here however my name and wiring a home for fiber should help you find it.
For SMB (the network file system protocol) the basic rule of thumb is approximately 1.1GB/s (plus or minus 5%) of real world throughput per 10Gbps of bandwidth. Assuming there are no resource constraints (storage, CPU, memory, PCIe bus, etc.). So a 1Gbps link can push an average of ~110 MB/s (+/-5%), and 25 Gbps can push ~2.75GB/s (+/-5%). Converting 2864.59 MB/s to GB/s (2864.59 / 1024) is 2.79 GB/s, which is within the expected real world range for 25Gbps. These are not official numbers, just observed behavior after working with SMB for years.
University of Alaska Anchorage was using BNC connected coax into the 1990's; when working registration in 90-92, most of the registration devices were IBM 5150's, running TN3250 terminals over ethernet. They also (mostly) still had the matching monitors. That ended about '94... when all class enrollment went to phone or dial-up for continuing students. '95 saw 10-base over cat 5 TP into offices, replacing dialup via PBX lines not-connected to outside telephony.
Now a days, many colleges, including 2 year state colleges are running full 2G internet, at minimum, with internal switches capable of the same using RJ 45 connections with smart ports that shut off if you swap PC's that have a different IP address so the network guy(s) needed to be aware and reset the port. All phones are VOIP based phones so if you first note no dial tone, chances are your PC is not getting the internet either. Your port has shut down. The 2 year state college I worked at was upgrading to 2G internet internally and externally and have wireless nodes for personal computers and/or phones. The internet and networking has come a long way since the 90's and the college also runs modern Dell computers, a combination of desktops and laptops.
My nerd credentials goes back to 10Base2 thinnet with terminators and BNC-T connectors, 10Base5 Thicknet with vampire taps, and token ring networks. There was actually no TCP/IP protocols on these networks originally. Used multiplexors to connect multiple sites across ISDN lines, etc. Then 10BaseT came along with IP addressing instead of the old IPX addressing in Netware networks. Life changed dramatically. I can hardly wait for terabit networking standards, because you know they will come.
My home network has gone from a mix of Thicknet and Thinnet, to Token-Ring, to normal twisted pair copper which was a godsend because as with USB devices, none of my ethernet connected devices ever became obsolete again. I absolutely love that I can mix my old 10Mb stuff with 10Gb on the same network. Same experience with USB-1.1 and USB-3 (4?) although it can really choke if you have a crappy chipset in your USB switch as was often the case.
Cat 5e is actually rated for 5Gb up to 40m, and 2.5Gb up to 100m. Cat 6 is rated for 5Gb up to 100m and 10Gb up to 55m, and typically works up to 100m in most installations. In short, you only need Cat6A if you need connections at 10Gb beyond 55m, and then fiber optic is probably a better option. Dave is right (as usual), 6A is considerably more troublesome to hand terminate. I’ve done quite a bit of it, and you want to avoid that if possible. 5e and 6 are notably easier to hand terminate.
Technically if you crimp 2 phone lines together as a ghetto cat5e to could push 10gbps for a very short range. It’s mostly copper wires at the end of the day
As an engineer, it's hard to keep up with all tech all the time, so I like your direct no thrills and frills, to-the-point approach. I appreciate it. Additionally, no stupid music or other background effects that only serve to annoy and distract. Refreshing. A very useful and informative video. Thank you.
Summed up my thoughts exactly .
agree
i dont come near as engineer, but, those things really can make the video quality worser (in my opinion) i think for almost any video. like that, that explains, maybe tutorials you along something.
From 320x240 to HD to FHD to 4k to 8K, we 10000 fold or 1 000 000 fold increrased data rates, but NO INFORMATION added to the 320x240 small few MB video. So out society do not goes to future with some new information, that can change something. Look at Russia, they do things effectively, with 1000 - 10000x less budget can get 10x more powerfull wepaons and more precise weapons, that can penetrate Patriot or IRIS or MANTIS sytstems. So a lot of traffic a lot of GB pre sec. does not mean we have more info.
Amen
This guy can explain with no inflection how exciting and amazing and great something is without even cracking a smirk. Love it!
Hey, that IS my smile :-)
@@DavesGarage haha.. nice
I like that
It sure is refreshing! As opposed to the faux over-excitement so frequent on youtube
You should hear what his wife says about when they bang.
Thank you for breaking it down like this. Almost nowhere on TH-cam people go into this kind of detail because the video maker will just assume their audience knows these things.
The TH-camr often doesn’t know so doesn’t explain. Had just learned of other TH-camrs and thus the knowledge is diluted every time
a couple of things to bear in mind:
-10gbe copper SFPs generate a LOT of heat and use a lot more power per port than DAC or fiber so avoid them where possible
-singlemode and multimode are about the same price now anyway, and singlemode is futureproof while multimode is not, so if you're going to put in fiber may as well go singlemode, future you will thank you
THIS-^ Ditching copper SPF+ virtually removed all my problems with cheaper gear like the Mikrotik..problems like it going from not working in a full 10gig environment to it never dropping a packet. well, almost never
Are you not worried about overmodulation by using singlemode on a short distance? Optics for singlemode on higher speeds are way more expensive too. You can get up to 40 Gbps over 150 meter of 2 strands of OM4 using 40GBase-SR-BD and up to 400 meters if you use 8 strands with an MPO/MTP connector.
The older adage was that you would use Multimode for very bendy applications (it works when twisted around a pencil) and singlemode for long, straight runs.
However, modern Singlemode fiber has a minimal bend radius of 1cm (slightly under half an inch), so that is also not a reason to choose multimode.
So this is basically to move files nothin special
As far as heat is concerned, does this apply to other SFP/SFP+ modules? Like 1Gbit for example?
I just put together my own personal Rack and homelab at home. I've been working and building them for work for years now but finally got around to building my own. I went with a mix of 2.5Gbps and 10Gbps. 2.5Gbps is for all the everyday PCs, access points, etc., 10Gbps is for my NAS and Workstation as well as my custom built Pfsense Firewall (my ISP Internet is 10Gbps Fiber). So far so good. Came across your video here as I was curious about 25G networking (already thinking about upgrading and I finished the setup yesterday).
Subscribed. Good info.
It's cheap enough to go 40Gbps/100Gbps by buying an out of warranty enterprise switch now. I run Single Mode fiber as transceivers now are better at not getting burnt out over short runs. Cabling is not the expensive part anymore....
When I was working with IT, 10 Mb/s was all the rage, after years of RG58 cabling the common common copper wire made all the difference. At the time, embedded controllers weren't a thing yet, so the ethernet hubs were repeating the same packets on all ports, and the machines had to run a collision detection on their own. Some gear used token ring on ethernet but we didn't.
Only when microcontrollers became powerful enough to check what was in some layers of the packets that we finally got the "switch" - where the packet was addresses to the port with the associated address. From that point on, on a network without collisions, the race for higher speeds made sense, and we had Cat5 (100 Mb/s) in the blink of an eye.
Thank you X the explainer Dave. It is happening all over again!
The first home networks I ran used 'thin ethernet' where if there was an issue _somewhere_ in the big daisy-chain, the whole network went down with no indication of where the fault lay. 10Mb RJ45 was _such_ an improvement. (A few friends and I installed it when, as students, we moved into a 6-person house. We had a spare 486, so that got put in the closet with a modem so that all of us could share a single 56K internet connection -- amazing for the time.)
I built a 10base2 network when I was a first year at university. The student accommodation didn't have networking so I built our own. Network switches were very expensive back then so we used the cable only 10base2 system. Didn't have any trouble with malfunctions, but I made all the cables myself very carefully.
Ah, the horrors of troubleshooting standing waves on 10base2 networks. Every time someone plugged or unplugged any PC, some PCs would end up in a dead zone and drop off the network, so I'd spend all morning crawling around under desks and work benches, adding a meter here, removing a meter there, trying to find a combination where every PC could se a signal.
The first network upgrade I ever did was to connect our 10base2 network to a cheap 8 port 10baseT hub, along with my PC and the company 'file server' under my desk. People weren't convinced that swapping one 10Mbit network for another would make any difference, but once I'd proven how much more reliable my connection was, I got the budget to buy 6x move 10baseT cards for everyone in the development office. Within a year, I'd convinced the director to fit structured cabling throughout the unit, and I was able to upgrade us to switched 100Mbit throughout, so the admin office could get the benefits too. I even got them to stump up the cash for, a *real* server (with a DAT drive for backups), a UPS, and replacement PCs for all of the seriously outdated 486 PC's that designers were still using. *8')
love your stuff! Nobody, I know, explains things better. I am taking my first networking class and this stuff is starting to make sense. Thanks.
Great to hear! I'm by no means an expert, just summarizing what I learned in doing the process for my own place! I'm sure true fiber experts found mistakes, but hopefully it's pretty accurate!
Changing the cables @ 7:51 earned a sub
Another amazing trip down memory lane. I worked in a major corporation who still had token ring in 1999. Thanks for a couple of triggers, they highlighted what it means to be on the spectrum. Appreciate all you do!
I worked for that that major corporation as well (probably). Joined as a tech illustrator (software manuals) using DrawMaster, and ended up building computers with Token Ring and OS/2. Had the first luggable 486 with the red screen.. It just fitted a short T/R card !
All this stuff seemed complex as hell and this one video made everything clear, I already spent a lot of time trying to find information about eg. DACs, fiber types, transceivers etc. Thank you a lot, this saves lot of browsing.
Very useful, educational, and lean video. I watched it twice and enjoyed viewing it each time and Dave makes excellent presentation with no fluff. ... Excellent
Fast home networks are fun 😁I made the 10G jump in 2016, and in 2017 I upgraded to a 32 port 40G Arista switch grabbed off ebay for very cheap. Since then I also scored a few smaller mellanox 12 port "56G" switches for dirt cheap. Naturally a large chunk of my home network is running @ 40Gbps - including the gaming/home theatre PC because why not, got the ports right :) Uh, you can say things have gotten a bit out of hand. Now .. its time to start a 100G upgrade... I might have a problem.
Unfortunately the crazy good deals are less common these days due to the still ongoing supply shortage on new enterprise grade networking equipment, making data centers and enterprises hold on to their network gear and more companies buying off the used market, driving prices up. Also many really good deals comes with some pretty severe caveats that takes some .. I mean a lot of work to overcome. Def not for everyone.
I had issues with fan noise acceptance in the house for anything enterprise. I went to 10GbE in 2020 when the early QNAP consumer level 10GbE switches came out... then got a second one. I was in a condo at the time, so I didn't have a good place to hide loud equipment. I'm in a house, now, so I can pretty easily dedicate a closet and isolate the noise pretty well, so I might do something faster... or just get a bigger, dedicated central 10GbE switch and keep the QNAP switches for edge switches. Though they don't do 2.5GbE at all, but small switches with a single or pair of 10GbE SFP+ ports and a few 2.5GbE UTP ports have gotten super cheap... Well, QNAP now has a 16 port 25GbE consumer level switch that's not particularly cheap (currently $1200), but supposedly quiet. So... I guess I'm running fiber everywhere.
(Yes, I realize I'm commenting on a 10 month old comment, but a lot is still about the same.)
THANK YOU! I really would have needed a video like this before I started with 10G... This is a great introduction and I'm sure will help many understand how it all works!
Glad it was helpful!
I just stumbled across your channel and I'm so happy I did. I'm in the process of upgrading my home network to 10 GB and your easy to digest explanation style is perfect. You definitely gained a new subscriber.
Dave, thank you! You saved our startup thousands of dollars with this video. We were under the impression that we would have to purchase a costly (Mellanox) network switch to connect our server to our SAN. On a side note, I remember being 11 years old setting up my first token ring coaxial network in my house. Those were the days!
I ran a Farallon Phone Net using AppleTalk down the hallway between my Mac SE and my sister's Mac (forget model). It worked, but I never had anything to send to my sister.
Much respect for a software engineer to give a master class lesson on network and hardware topics! My new favorite channel! 🙂
yes, most programmers I've worked with are dumb AF when it comes to hardware and some even didn't master basic OS functionality (but he was a great SAP-consultant). I always teach my younger peers that every IT'er is also a user. You can be adept in one field, but a total noob in another, and then there's Dave 😆
I remember the good old days using Coax in ring topology. One loose cable on a single computer that got bumped loose and the whole network on the ring goes down! I'd be crawling from PC to PC to figure out which one was loose. Changing to star topology was a pure dream.
Surely the point of a ring topology was to provide resilience?
@@deang5622 It was inverse resilience. One tiny connection failure brought the entire network down.
Thank you for the explanation of the DAC cable, this question was top of mind. Keep it up Dave!
According to Cisco. 100m cable is made of these 3 cables.
4 m from switch to patch panel. 90 m from patch panel to connecting point going inside a wall and shielded cable if preferred. And then 6 m from connecting point and to user.
Indoors, the recommended max is 100m (330 ft): 90m solid plus 10m patchcord allowance.
In the multi-mode vs single-mode discussion, I was under the impression that multi-mode's limitations aren't from signal loss per se, but the fact that the signal gets "smeared" across time from light taking more or less time to reach the other side depending on how much it bounces. If you try to hear someone speak in a room with lots of echoes, it's very difficult -- not because you can't hear the sound of their voice, but the fact you're hearing the end of one word at the same time as the beginning of the next.
Modal Dispersion becomes an issue with longer runs with multi mode. It also has to do with the type of light source is used, the grade of multimode cable. Single mode on the other hand doesn't experience modal dispersion. Multimode cable is more expensive, but the optics are cheaper. Singlemode cable is dirt cheap, but the optics are more money, and depending on the required bandwidth at the time of an upgrade, it may be necessary to replace the multimode with a higher grade (especially if one has 62.5um multimode installed and the upgrade path is 10G or higher. The 2 benefits with single mode is the fact that with single mode, the maximum bandwidth has not been reached yet, and in the case of service providers you can multplex many "colors" or wavelengths on a single strand, in the enterprise it is usually used in the form of bidi optics that use a single strand for both transmit and receive.
Yes, different wavelengths have slightly different path lengths. Over longer lengths you get a complete screwing up of the signal. Transceivers don't produce just the wavelength of light they're rated to, but bands at either side. So as the path differences build up, as a shorter wavelength bit is going low it'll be swamped by a longer wavelength from a different point in the signal being high. The receiver just sees this as a "high".
In multi-mode the more gradual change between core and jacket means this smudging is more pronounced.
With single-mode the path lengths are far more uniform (there's a much sharper transition from core to jacket, so all of the wavelengths reflect at once), so that smearing you pictured takes a lot longer to happen- tens of kilometers. Multiple close-but-distinct wavelengths can be crammed down the same fibre with no ill effect using WDM/CWDM/DWDM.
In 18 years of working with industrial fibre systems, multimode has been rare. The cost savings just aren't worth the lack of flexibility, range, and multiple-wavelength functionality. Multimode seems to be common in more pure-IT industries where things like FORJ passes and distance aren't a concern.
My house is also single-mode so it'll work fine on smaller scales too!
The discussion will not be important soon. The production volume of single mode is so big in China that it drives down the price of the whole thing to multi-mode level. Thus there is no incentive to use multi-mode. The effect will spread to the whole world several years later. It will be all single mode just because of price.
Multi-mode is easier to work with, but once cables are cheap enough that you just replace them if they fail, rather than trying to repair them, that doesn't matter so much.
@@markbooth3066preterminated cables already so cheap compared to the equipment to terminate them
Just for those who are new to this: SFP+ connectors are made for 100 to 500 mating cycles (the range of numbers you will find in different datasheets).
I learned more with this video then the many many others I've watched trying to find information. Thank you sir
Back in the late 90s i didnt own a switch or a hub. I had a cheap rj45 1 to 2 splitter. And we shared our dial up from the host computer. And played quake 2 and various other games with 3 computers this way, as long as host was the pc on the side of the adapter that didnt split.
And some years prior to that I vaguely remember playing a bit of doom over coax cables with bnc connectors with my dad.
Talking about networking brang back some memories. Thanks for the nostalgia trip Dave :D
Highly recommending going optical between buildings. Unless you want to swap 12 of 16 tranceiver cards after lightning strike.
If you're running a machine with PCI slots modern enough for those NICs, MDIX will probably already be in the stack. Regardless, I love the crossover callback. It's important for the younger generation to still understand these fundamentals I think. I love both the tech and videos about autism you put out, equally! Thanks!
1000MBASE-T and 10GBASE-T require MDIX, so crossovers haven’t been necessary since fast Ethernet (100Mbps)
MikroTik Routers have a fair price tag, but are not very easy to administer. Nevertheless a good solution for a homemade lab. Unfortunately, it's not very easy to get rid of all bottlenecks, e.g. I can't find a 10g card for my notebook...
@@claus7555 You might want to look into thunderbolt to 10g adapters, but they are very expensive, think $350 and up
@@VraccasVII I'm running a couple of cloud instances for free, so today, I removed my server rack completely from my home office to have a bit more space available. That's probably not forever...
Thanks for the kind words!
Thank you Dave, an excellent introduction. So many cable types !
Glad it was helpful!
Wow . Once years ago, had to setup trade fair, with fiber optic cable runs , it was fast for sure, but disassembling the runs without damaging the miles of cable well was tedious at best. Our in house ran over just two fiber feeds , speed was impressive. Well thanks again for teaching me something new. Take care.
Re: multimode vs singlemode - the cost difference has really come down in recent years, to the point where the cost difference for a home setup is probably only a few dollars. A lot of folks have started recommended that any greenfield deployments standardize on single mode for everything for simplicity. Since singlemode can do everything multimode can, and then some, you don't have to worry about where to use which type of cable.
I am autistic with ADHD I haven’t been diagnosed but both of my children are I appreciate your videos . I suffer from short term memory. I have difficulty reading and writing so it’s hard for me to express my feelings. I love your videos and I always look forward to your videos. Thank you. You mean the world to me. I always look forward to your videos.
What drivers are you using and what operating system do you use with your PCIE 10gtek card? Windows 11 doesn’t support those cards anymore as far as I can figure out. Only win 10?
I have a 10G portion of my home network with that MicroTik switch. Really good value, the 10G was faster than my SSD for the longest time lol.
I needed this instruction 7 years ago! 😆Had to do soo much reading, research, trial and error with Ebay parts to get my 10g VSAN lab running.
Wonderful video! I am a new subscriber here, but I work for the nations largest telecommunication provider. I have 5 Gbps symmetrical Fiber Internet at my house and I find it difficult to fully utilize that connection speed. For work I install & Repair FTTP or Fiber To The Premasis utilizing SFP+. We use standard, alternate, and XSGPON SFP modules. The difference between the three are optical frequency and XSGPON is the SFP+ Module that is capable of 5Gbps Single Mode Fiber. I also install & repair FTTN Single & Bonded pair DSL connections as well as IP-RT IP-CO internet connections. I truly love your videos! The way you explain everything is very unique!
I can't find the correct words to describe how amazing your videos are... thanks Dave!
I appreciate your hard work at 8:05
Still the best 'second' video for introducing people to home networking.
If they get interested in having a home network, better to get them introduced to everything and the future. More speed is better..
(Secret-note; first video is 'Warriors of the net'. If they survive that, they will have the right determination to actually use good network advice).
this video is an interesting approach it's like a news reporter reading out an instructional video
Very simple explanation of complicated networking. Thank you!
Good heavens, Dave, I have to watch this video about 20 times just to get an idea of what you are all talking about. I have a lot to learn! :-)
Thank you your explanation has made my life as an almost networking lay person who’s trying to upgrade slow cat 4/5 domestic network to fibre so I can improve the data speeds for all my devices TVs etc
So much content, Dave. You are one of the best true tech channels out there. Greetings from Colombia, South America.
Thanks for the kind words!
Dave this was great episode. I have been a networker since 1999 and really enjoyed your explanations. Super cool!
This is a great explanation for getting into fiber! Love it.
This was very insightful with perfect timing. I'm developing a 25G fiber multiplexer at work (three 25G fiber ports over 1 fiber), and was looking for software to use to test throughput. Iperf looks like a good option. Good work as always.
I breathed an audible sigh of relief when you fixed the routing of those cables.
Outstanding video. Very informative and done so in Lehman's terms. You the man! Thanks
Layman's terms, Lehman was a banker ;)
Loved this video, took me back to my days working in the 90s when we migrated my office building in London from cat3 to cat 5 with fibre between the floors. That project taught me a lot. It was running 16mb/s token ring with in a year I had moved the net builder 2 to the edge of the lans and replaced it with a pair of token ring switches with spanning tree. So each floor had a redundant link.
I was sure I had seen the first part of the video before. Just realised it was from one of your videos about 2 years ago.
Been working at a small ISP for some time now. We use SM BiDi (one fiber for send and receive) for Fiber to the Building and there are cases, where the signal blinds the receiver, if the distance is too short and/or SFPs get mismatched. Also, as the beam is much more focused and usually has a higher frequency, which translates to "way more 'impact' energy", the risk of permanent eye injury is much higher with SM when mishandled. SM can reach as far as 200km (120mi) with carrier grade SFPs and you might not even be able to look at it twice before running out of functional eyes.
Therefore, MM and TP are still valid options for short distances and/or hobbyists and unless you want to go faster than 10G, upgrading an existing inhouse copper network to fiber gains you nothing but trouble. (Exceptions apply.)
When I have to talk to a customer about that topic, it's mostly about the widespread misconception that fiber would be faster, better, stronger in any case. In reality each SM, MM and TP have their pros and cons. 10G are 10G no matter the physical medium, pings are the same, too - even with a media converter in between. Btw, broken fibers still DO happen, especially if you are a proverbial tinkerer.
At Consumer Grade, hardware with 10G support is rare and so are the use-cases for the average Joe. I see on a daily basis what people have connected to their home/soho routers and more than 80% - small businesses included - won't be able to make use of even those 10G. Consider yourself well equipped if you have devices that are capable of 2.5G without bottlenecks apart from your LAN.
Now, if you belong to the other 20% and you know what you are doing, your mileage my vary. In this case, I won't have to send you a technician (and then forward your ticket to Billing) and you can simply ignore me.
I have 2 GB fiber optic internet to the house, that's what I stream you on.
Just like you I have that weird fascination that "The future is actually here" since I'm using fiber optic for my internet connection. Directly to my house. Fiber optic internet. Blows my mind. I still remember hearing about Netflix, You know, the new DVD mail order place, talking about streaming video to everyone's houses and laughing because there's no way everybody is going to get on cable internet. And DSL can't handle it either.... And here I am with a freaking fiber optic connection directly to the backbone. It blows my mind.
One day I too will live the dream of fiber to the home!
Definitely bookmarking this video for future use when I upgrade my network.
Great video and channel. You are going a great job. Very knowledgeable with no BS.
Didn’t think I would learn anything from this video, but I did! Thanks for a great video.
Dave did I hear you say you had a screw loose? Lol somebody had to say it. As you the single mode, fiber optic, installing a connector on a fiber run , is a job for experts, the alignment critical, tooling used expensive. I was allowed to try it and got lucky ... had little interest in doing it again, the tech was impressed, I of course acted unsurprised at my success . Since such a rig is well beyond my resources , I shall go buy a lottery ticket. The two dollar dream ... will try to get a copy of your book, had a friend that suffered, and passed young. Before his last trip in, he stopped by and wanted to give me his 53 pink Lincoln, I was shocked it meant so much to him. I told him I'd store it till he came back. No he said I want you to keep it. . He didn't come back, I gave it to his mother. Why ...it hurt to drive it, my friend was gone . Natural causes ? Drug experiment ? I'll never know. The program was experimental . Have not thought about this for almost 50 years ....take care keep teaching
Man your videos are awesome. I have not done networking in years but ready to make my own network since I am into video editing and also clothing data files for my mother business. I guess time to need out. Baby steps but you are a wealth of knowledge thanks.
100G and 40G ports in switches can often be broken out into 4x25G or 4x10G links (twinax or fiber, not twisted pair). Computer NICs typically won’t break out that way (some Mellanox is the exception)
Afair Intel 710 can do bifurcation too. Do you know any interface, that would provide hardware LAG support? Mellanox can do hardware LAG offloading for VFs but I just need card that would be LAG aware. Simplifies network a little.
It’s typically the OS that does it rather than the card. You can create a team in Windows server, or a LAG in Linux. In VMWare you need a distributed switch to do an LACP LAG
@@ausnetting most of the time it is OS. However, some interfaces do have internal switch fabric. In Intel 710 this fabric is only there to allow connectivity between VFs (as access switch typically would not reply on same interface, as packet was originated from). Mellanox ASAP2 engine can do LAG for VFs without exposing that fact to guest or host machine. It surely works in Linux (aka Proxmox) but have not found (yet) info about Windows support for configuration interface
Dave, your videos always have excellent writing (I assume you're not speaking extemporaneously) . Clear and concise. I like 😀
"Two cards, one cable, and a lot less messy" at 4m12s is a slightly obscure and entirely hilarious reference!
Came here to say this. Lol.
I've been wanting to upgrade my 1GB ethernet for 3 years now. I've been told 2.5G is cheaper than 5GB & way cheaper than 10GB. I haven't look into it for at least a year and now after watching this video I've decided to take the plunge and finally upgrade. To what I have yet to choose. Every time I see and read instructional videos like this one I learn a bit more of what's needed. Brand names is what's keeping me from purchasing. I've read reviews of NIC cards that were name brand but had many complaints of burning up.
One thing you appear to have almost mentioned, is that fibre patch cords are always crossover. In fact even the in the wall structured cables are crossover too. When you connect them, either directly, with a single cable or between rooms, with the in the wall cable & 2 patch cords, you will always wind up with a crossover. You can see in this video, one fibre is yellow and when compared to the other end, it's the opposite side of the connector. Often, when you get a patch cable, the two connectors are loose, with a plastic clip to position them. With that, you have to make sure to cross them, if needed.
That 15 pin connector you said looks like a game port is for the old "ThickNet" 10base5 cable, which was used with a transceiver. That connector is for connecting the transceiver.
Even some 100M NICs didn't need a crossover.
The earliest LAN I worked on used triaxial cable. That's a cable with 2 separate shields. It was a proprietary Rockwell Collins network on the Air Canada reservation system in early 1978. It used Time Division Multiplexing (TDM), instead of packets or frames. It was also a ring and ran at 8 Mb.
BTW, some vendors will resell returned equipment and will shrink wrap it again before selling it.
Worth mentioning that there are also AOCs, which can be used similar to DACs but for longer distances. AOCs are still usually cheaper than buying two optics. The downsides here are that if the cable gets damaged, you have to replace the entire cable, where as if you used two optics and a cable, you only need to replace the cable.
Additionally, for 100gig+ over multimode, MPO cables have started becoming the norm, but these are just bundles of normal multimode fibre pairs.
This means they can also be used to run multiple fibers between locations, for example running fibres between floors in a building.
I've also used these with MPO cassettes as breakouts. You install a 100gig switch, with QSFP ports on one side, plug in a MPO cable, run the cable to a different room, connect to a cassette, which splits that MPO into normal multimode pairs, which can be used for up to 4x25G or 4x10G devices.
I feel like i’m walking away with a pocket full of specific, new knowledge. Excellent!
Thank you Dave! I really like how you format all of the information in your videos!
This guy speaks my language. & Thanks for the book suggestion.
I started a job in WAN last januari. We're using OS2 single mode for distances up to 80km, at 10, 40 or 100 Gb/s per fiber. Some SFP's will reach out to 120km, but we're not using those. Never work with live fibers! The lasers will cut and burn you, seriously! There's a lot of energy going down a fiber that needs to reach for kilometers. And you can't see them, they're outside of the visible spectrum. Get the distance wrong, and you'll burn out the SFP on the receiving end. Also, always have a ferrule cleaner handy. It's good practice to clean the ferrules every time you make a connection.
Fantastic translation with examples.Greetings trom Poland.
I would love to see a follow up video where you enable RDMA on you NICs. This should give you the same performance but without the CPU overhead of processing each packet when using SMB Direct (file copy). Also you should try out enabling SR-IOV on your VMs. This will give your VMs direct access to the NIC.
That is if the NIC support those features. I bought several 10/25 gig Mellanox Connect 4 dual NIC cards from E-bay on the cheap a couple of years ago. I am not sure if he bought the 4 or 5 series so it will vary on model.
RDMA is huge offload. However, you have to be cautious with SR-IOV. It gives some advantage in demanding applications on VMs and there is no denial for that. However, I have experienced issues with handover in HA Hyper-V cluster. Network connectivity was interrupted whenever handover happened and we were never exceeding used VF count on particular interface. Also, windows guests were unable to utilize failover networking. I have ended up with plain, old LBFO teams for VMs. I do prefer machines to be 1% slower in high-performance networking (not a big deal, when half of people are working remotely anyways) but without unnecesary downtimes.
Yeah rdma is great but the switch probably won't support PFC, which is kinda a requirement for RoCEv2
@@LampJustin If your Nic supports iWARP then you don't need PFC. Also Mellanox has some newer Technolgies (ZTR) that lets you get away without configuring PFC.
@@brylozketrzyn Agree about SR-IOV with HA clusters. If you only have a limited number of VMs that use SR-IOV and ensure that there are enough VFs available on any given node then you should be OK. I would also avoid LBFO as it is end of life. I don't think SR-IOV makes your network any faster, but it reduces CPU overhead somewhat.
Just wrestled with 96 25gb Twinax cables for 24 servers across 6 racks.
Of course, it was just plug and play as the cards and OS support 10gb and 25gb without drama :)
Just another datacenter day!
Just found this channel, and I'm very happy I did.
Thanks!
for future proofing, single mode is the same it’s been for 30+ years and still handles terrabits per second (with DWDM transceivers).
Go single mode if you can.
Thanks for the demo and info, have a great day
First time viewer, absolutely loves the content so far. I'm glad I got to see this. You've explained every little thing and definitely can tell you've put a ton of research beforehand. Absolutely worth subscribing. Thank you.
Nice to know the switch lets the different speeds to get along with each other and not slow the whole network down to the slowest connection.
This is not the era of hubs anymore. However some devices have "hidden" switch inside to increase number of ports, so you may have your bandwidth divided between downstream devices (yes, I am pointing at you, Mikrotik)
Hi Dave, love your video, plain and simple!
Glad you like them!
Excellent video! Well spoken to the point, extremely informative and enjoyable 🥂
Thank you, that was amazing in all relevant aspects.
Fellow old guy, that's wanting to start learning networking, here! Just stumbled across your wonderful jem! You got my sub and like! Looking forward to looking and learning more! Thank you for sharing your experience and knowledge! 😮
Another old guy here who was old when I went back to school a technical college at age 47. The networking aka Cisco class was my favourite difficult enough to push me but not so much that I didn't learn. I'd suggest in-person classes if you can if you're like me you need a push. At the very least Cisco Packet Tracer if you can get it to work without an account and the labs many of which are out in the wild to follow along on.
@@dg-hughes I've installed it and poked around on a couple of times. Certainly looks complex and stocked pretty good! Looking into those labs, pretty cool that we have so many cool "modeling" options that can get so close to real world practicality.
You should check out the 400G-ZR optics. They can do 400G, up to 120km. Combine this with the fact you can use different frequencies of light for DWDM, and an optical combiner/splitter, with up to 96 channels, you can get 38Tb/s down a single fibre. The QSFP optics do cost around £35K each though
There's some amazing fiber tech out there that's way above MY paygrade! I believe they can shoot multiple lasers in and extract them at the same bounce angle and this run multiple streams of light in a single tube, all kinds of crazy things! And it's not on/off, they can use multiple level of brightness and so on...
And now even 800G optics are being introduced around one of my work sites. That requires PAM4 modulation on 8 lanes of 106Gbps with QSFP-Double Density transceivers. From here on with 200G lanes it's going to have to be all optical, bypassing the electrical interface and piping fibre directly into the switch chip on the board using silicon photonics. If you have to ask - like me you probably can't afford them - well just yet anyway :).
I wish I were working with you,learning from you, Helping you in all those networking things,😢
I Like devices,computers and all those stuffs.
You’re amazing Sir.
Thank you very much much for this comprehensive video!
Back in the earlier days there was an electrical difference on the tx and rx sides of the transceiver, and the Ethernet isolation transformers had different winding ratios for each meaning that you had to use a crossover cable to line up tx-rx.
These days, electronics are better and more capable, and Ethernet isolation transformers use the same winding ratio for "tx" and "rx" meaning that the transceiver can use tricks to automagically rearrange them.
I love your video ho you explained all this so clearly!!! Wow thanks! I learnt a lot from this video - thanks!
Also beware of SFP+ FC2, these are 16/32GB Fiber Channel or iSCSI and look identical to a typical SFP+. You will get traffic over them, but they will usually default to max 1GB Ethernet, especially if they are HP or Dell branded (they don't make them just slap their label on them).
To make it even more confusing, some modern cards even support both SFP types, but that doesn't mean it works well, or at all. (FC and Ethernet are 2 completely different "languages").
I followed your previous video tutorial for my 10G setup.
Well explained, liked the demonstration technique and edit, thx!
I highly recommend you use single-mode fiber even in your home. You will never need to run a new fiber again vs multi-mode which may at some point need to be replaced to get higher speeds. Costs between multi-mode and single-mode are very similar now a days. I did a write up about my setup but I can't post the link here however my name and wiring a home for fiber should help you find it.
Absolutely fantastic video. Your videos always explain things so clearly. Thanks for making it.
Thank you, Dave for informative video.
Thanks for the demo my dude.
😍😍 those speeds are to die for
Thanks Dave, I wish I understood more
Thanks!
Finally this is video what i needed
Ooo I can hear the quality of connection on that copper cable. It's very niiice!😁
For SMB (the network file system protocol) the basic rule of thumb is approximately 1.1GB/s (plus or minus 5%) of real world throughput per 10Gbps of bandwidth. Assuming there are no resource constraints (storage, CPU, memory, PCIe bus, etc.). So a 1Gbps link can push an average of ~110 MB/s (+/-5%), and 25 Gbps can push ~2.75GB/s (+/-5%). Converting 2864.59 MB/s to GB/s (2864.59 / 1024) is 2.79 GB/s, which is within the expected real world range for 25Gbps.
These are not official numbers, just observed behavior after working with SMB for years.
University of Alaska Anchorage was using BNC connected coax into the 1990's; when working registration in 90-92, most of the registration devices were IBM 5150's, running TN3250 terminals over ethernet. They also (mostly) still had the matching monitors. That ended about '94... when all class enrollment went to phone or dial-up for continuing students. '95 saw 10-base over cat 5 TP into offices, replacing dialup via PBX lines not-connected to outside telephony.
Now a days, many colleges, including 2 year state colleges are running full 2G internet, at minimum, with internal switches capable of the same using RJ 45 connections with smart ports that shut off if you swap PC's that have a different IP address so the network guy(s) needed to be aware and reset the port. All phones are VOIP based phones so if you first note no dial tone, chances are your PC is not getting the internet either. Your port has shut down.
The 2 year state college I worked at was upgrading to 2G internet internally and externally and have wireless nodes for personal computers and/or phones. The internet and networking has come a long way since the 90's and the college also runs modern Dell computers, a combination of desktops and laptops.
My nerd credentials goes back to 10Base2 thinnet with terminators and BNC-T connectors, 10Base5 Thicknet with vampire taps, and token ring networks. There was actually no TCP/IP protocols on these networks originally. Used multiplexors to connect multiple sites across ISDN lines, etc. Then 10BaseT came along with IP addressing instead of the old IPX addressing in Netware networks. Life changed dramatically. I can hardly wait for terabit networking standards, because you know they will come.
You're an Awesome teacher Dave. Thanks.
My home network has gone from a mix of Thicknet and Thinnet, to Token-Ring, to normal twisted pair copper which was a godsend because as with USB devices, none of my ethernet connected devices ever became obsolete again. I absolutely love that I can mix my old 10Mb stuff with 10Gb on the same network. Same experience with USB-1.1 and USB-3 (4?) although it can really choke if you have a crappy chipset in your USB switch as was often the case.
Thank you Dave from THAILAND.
Thanks for demystifying the stuff! Working bottom up worked for an old vet like me :-D
Glad it helped!
Cat 5e is actually rated for 5Gb up to 40m, and 2.5Gb up to 100m.
Cat 6 is rated for 5Gb up to 100m and 10Gb up to 55m, and typically works up to 100m in most installations.
In short, you only need Cat6A if you need connections at 10Gb beyond 55m, and then fiber optic is probably a better option. Dave is right (as usual), 6A is considerably more troublesome to hand terminate. I’ve done quite a bit of it, and you want to avoid that if possible. 5e and 6 are notably easier to hand terminate.
Can't 5e do 10Gb on a short run? I think 40m.
@@TheBacktimer In practice, it usually works, but it’s not officially supported.
Technically if you crimp 2 phone lines together as a ghetto cat5e to could push 10gbps for a very short range. It’s mostly copper wires at the end of the day
@@suiton20 Nope, phone cable is almost always Cat 3 cable and it won’t even run 100Mb at 10ft.