I run my server, my computer, and my partner's computer over 10Gb currently. Mikrotik switch and sfp+ cards with fiber modules in each machine. It's so overkill, but it's great for my lack of patience in massive data transfers.
I have been doing 1gig but recently have been transitioning to move things over 10 gig. Installed 10gig adapter on the pfsense machine and the ISP router already. I have been looking for a switch that serves the purpose and this one looks promising. I want to do 10gig on the nas, proxmox server and the pfsense machine.
I run 10gig to my file server and router (in the same rack) and to my office, but I do want to replace the cat6 run to my office because those rj45 transceivers get hot.
Just to demystify this topic a little bit (because it's less complex than people think): (modern) Ethernet fundamentally just needs 8 wires to work, so any 8-lead cable can run any speed in theory. They are all wired the same electrically. What determines if you can practically run a high speed though is the signal quality - which is mostly a function of the length of the cable, but also the quality and Em noise separation. The higher speed, the cleaner the signal needs to be - so either better cabling, or shorter distance. In a typical home with relatively short wiring you can usually run much higher speeds than the cable is rated for, because the cables rating is usually designed around 100m or so. Thus, if you have a 10 or 20m cable you can run it far "above spec" without issue, because the short length more than compensates for the worse cable quality. If you run a higher speed than the signal quality can handle you will get packet loss. That "wall" hits pretty fast, so it tends to work either flawlessly or terribly (or not at all more likely). I would always recommend checking packetloss under stress when running a cable out of spec. It tends to be be fairly obvious if it's fine or not. TLDR: If you have old cables in your home, you should test them before upgrading for higher speed. Chances are you won't have to change them - or at worst, maybe only your longest cable run.
A lesson I recently learned is that SFP+ modules negotiate 2 speeds - between the switch and the module, and between the module and the PC in your case. The link speed shown in UniFi is only the former…it won’t show the link speed between the switch and the PC. You can only verify the links on either end of the module.
Then why did it auto-negotiate 1G on his machine that also reported 1G? I would think it should have shown auto-negotiated a 10G for the switch with his PC reported connected at 1G.
Don't forget, Windows file copy is only a single thread process so the numbers won't be that fast. Linus Tech Tips ran into the same problem when upgrading their network.
NiTeHaWKnz... Just remember that depends on the SMB version you are using... SMB 3.0+ can be multi-threaded and even multi-adapter using SMB-MultiChannel. You can check this looking at powershell with get-smbconnection and get-smbmultichannelconnection. In my case I have a dedicated 3.1.1 server utilizing 4 40Gb nics and the storage to back it and I am able to read at about 12 GB/s or ~48Gb/s.
The copy is working just fine and can hit 10Gbps as he saw in the first few seconds. The problem is the drive speed. It was able to go full speed because it was essentially filling up server RAM as a write cache. Once that cache is full, then you are only getting raw write performance.
This is no longer true as of SMBv3 and the CIFS version that supports the multithreaded CIFS implementation. This is recent (last 5 years or so) but has been present on Windows Server and Client OS since Windows 8.1 was released.
My house was built in 1997 before the cat 5e spec was even a thing, and was wired with decent enough quality cat 5 (no "e") that I actually can get 10G over the ~30ft from my living room to the garage.
I'm literally in the process of buying a condo and I discovered during inspection that all the phone jacks were wired with Cat5e, so I was already researching 10G over Cat5e when I saw this video. Excellent timing! I'm going to give it a shot after I move in.
I bought my condo 29 years ago and it was wired with CAT3 and daisy chained jack to jack, so not the greatest for Ethernet. However, when I got my cable modem, in the late 90s, my ISP actually ran cables in for me. I wanted the modem in my "office", which is at the opposite end of my unit from where the cable comes in. My ISP sent a 2 man crew who spent 3 hours fishing the cable up inside the wall between my living and bedrooms, along side an air duct, over my closet and bathroom ceilings and into my laundry room, where it crosses my ceiling, then down behind the water heater and through the wall into my office closet. While they did that, I had them pull in a couple of runs of CAT5 (5e wasn't available then). They even patched the drywall where they cut it. In addition, they put red "Data" tags on the coax.
I jumped from an HP ProCurve 24-port directly to the 48-PoE-Pro, and couldn't be happier. I also added a fan inside to keep things quite cool. It's a powerful device, and can be chained to multiple copies of itself.
Over 1G network sometime you have to "Hard code" the speed in both side. In a core network, we'll never use auto negotiate, we must hard code the speed at both side, it will more stable and will not suddent drop to other speed.
I have the Aruba S2500-48P. 48 ports of gigabit POE and 4 SFP+ ports. I used Finisar 10gb SFP+ multimode modules with 10M of OM3 LC to LC fiber cable. Both worked great connecting the Aruba switch to my Mikrotik 10gb SPF+ switch.
The main difference between Cat 5e and Cat 6 cabling is signal attenuation. To get a specific bandwidth we need a minimum amount of signal to noise ratio (SNR) at the receiver, and the sender only has X amount of power it can deliver down the line. So more attenuation means a shorter distance is needed before the signal strength is insufficient to achieve the needed SNR. However, other factors can also effect SNR, so in more electrically noisy environments Cat 5e can be insufficient at lower distances. In general for home and even office use, Cat 5e can carry 10 Gb/s Ethernet for 20-30 meters without issue in most situations. However, not always. Cat 6 has thicker wires that gives the signal a bit more surface area to work with, effectively lowering attenuation. Also makes DC power distribution via PoE easier. Cat 6 is often also having a spacer that spaces out the 4 pairs from each other, reducing crosstalk and improving SNR a bit more. Some cables have a shield as well to protect the cable from external electromagnetic interference. Then there is also Cat 7 cable on the market, but the networking industry don't really use it yet. But it tends to have individually shielded twisted pairs, as well as a shield around all four pairs. And on top of that the different pairs use different amounts of turns/twists per unit of length, this is to reduce crosstalk even further. But in short, the things of importance is the signal to noise ratio at the receiver. Even just hand wiring some loose copper strands with minimal twisting can run 10 Gb/s Ethernet as well, though not particularly far.
@@wayland7150 The Cat rating of the cable doesn't really affect latency. But yes, Direct Attached Copper is lower latency than a pair of SFP to Ethernet converters. Mainly since Direct Attached Copper is literally as the name implies, the Tx differential pair going to the SFT port just gets routed straight over to the Rx pair on the other SFP port. Ie, no logic, nor any protocol processing. DAC cables though have signal buffers to ensure that the signal makes it over, but this adds almost no appreciable latency. Regular optical SFP modules work in the same fashion, just instead the signal buffering drives the light source, or reads the optical sensor on the other side.
@@wayland7150 The average person wouldn't notice the difference in latency. You're talking hundreds of microseconds. and there is no reason to bother with Cat7 for 10 Gbps, it's not even recognized as proper cabling standard either. Beyond Cat 6 it goes up to Cat8. Again there is no purpose for it for 10 Gbps.
@@todayonthebench There is a huge different in latency between CAT6 and DAC. Forget The Average Person because they can't tell the difference between 100meg and 1gbit so have no use for 10gbit. The exceptional people who do care would be very keen on low latency because in some applications that's the crucial factor like gaming.
@@wayland7150 Any latency differences in cables on the local network gets dwarfed by the latency of the internet at large. So for Gaming the local network cable delay isn't really going to have an effect on latency. The latency of cables only starts to matter when one deals with local network storage for database applications, or for clustered computing workloads. Not gaming.
The X540 intel 10gbit adapters are sometimes finicky. I use them for all of our production equipment and they work fairly well. Check your switch's stats on that port for CRC errors. That would tell you if it's clean.
Tim, thanks so much for doing this test. I have a Ubiquiti US-16-XG UniFi 10G 16 port switch (12x SFP+ & 4x RJ45) on backorder and for RJ45, I have a bunch of CAT5e cables and nothing better, so I'm glad to see those will work for the time being. Because of cost, I was thinking of getting some used Mellanox CX311A 10Gig SFP+ cards + transceivers and fiber cables for my main PC, the Cisco server I have and two NAS's. Everything is in my office so all short runs.
That's awesome and gives me hope. I have a cat5e cable going up one side of my house, across the roof and down the other side. It is a 2 story house, so I think I may be coming close to that 55m, but at a guess, I think it's about 45m. This connects my 2 parts of my homelab, where I have rack servers sitting on a cupboard, and the other end to my office. I have wanted 10Gb, but always thought the cable wouldn't work so never tried, and don't really want to run another cable, though I have a whole spool of cat6 that I was gifted. Off to eBay to see if I can get some cheap 10Gb switches that support VLANS.
Thats great it worked for you. I tried connecting my XG-16 (in the office) to the Unifi Aggregation Switch in the garage about 9 months ago at 10gbe and it was a disaster. Admittedly the in-house wiring came "as is" when we rented but it is a high-spec new build house so I'd have guess cat-6 but it may be cat-5e and wiring distance is a bit hard to know but I'd guestimate at 100-150ft. Anyway, I was able to connect at 10gbe at the link level but the amount of packet loss I experienced was off the charts. Still you're success has me wondering again and it would certainly make my geek dreams come alive.
Also note, if you have the right SFP+ modules (aka, those which Unifi supports) I believe it should auto-negotiate to 10 gbe. With my UDM Pro to my Unifi Aggregation Switch it works fine.
Thanks! I will do some real world tests soon to measure packet loss! Also, I did buy official transceivers so not sure why my Windows PC chose 1Gbe but mu Linux server chose 10Gbe. Thank you for the tips!
Cool project. On your auto-negotiation issues, I have heard that SFP+ to RJ45 modules can have issues negotiating 10G on Ubiquiti and is just better to set the desired speed manually.
ive run cat6a non-shielded cable and keystones. i have no 10gb equipment (yet), however i do have some long POE+ runs. wanted a thicker gauge wire to reduce heat generation, and future 10gb expansion. pulling cable sucks sometimes.
@Tim, to be sure you have a stable 10Gig. Best to check your Switchports on the switch. Look if they do not flap and you do not see any errors on those ports.
I just upgraded from a 50mbt fixed wireless solution to a 2gbt fiber service (Thank god after waiting 20 years for REAL internet here!!) so I'm looking to upgrade my infrastructure here in the house. Running Cat5e as well so may just use what's already here. I run a lot of stuff over wireless but all of my core services are on hard lines.
The SFP+ to RG45 converters are notoriously quirky and its no surprise that auto-negotiation doesn’t work reliably. Just set things to 10Gbe (like you did) and all should be well. You might alway want to play around with setting jumbo frames (9k MTU). It can make a huge difference in transfer speeds at 10Gbe, especially to/from windows which does single-threaded IO for most file transfers.
I've watched all your videos, you're an awesome dude. When that 10Gb connection kicked in, I think that's the first off-script, out-of-character I've ever seen you. Most the time, it seems more like you're reading from a script; this was more akin to the first time you ever had an orgasm and didn't know what was going on. Absolutely loved it brother Timothy, but thanks for not showing us your pants afterward.
I have upgraded some Weeks Ago (USW-Pro-24), but just my Server (DAC, its in the Rack with the switch) and my Desktop (Upstairs, RJ45). I wired the house with CAT7 cable when we bought it 8 years ago. I ain't got working full 10 Gbit at first try at my Desktop. I thought I messed up some shields at the Patch Panel, but it was a faulty (bought) CAT6A Cable wich connected the Desktop with the Patchpanel. Greets from Germany
That is awesome. I love to bust myths... I'm currently running 1 gig through telephone wire which happens to have 8 wires... Everyone even said its not possible because its litterly not shielded at all.. Paket loss is a bit higher, I would say wlan standards. But speed and everything is awesome. Love to see you manged 10gig with cat5... Sometimes it makes me think cable industry is a bit... Sketchy and never really tests their new standards...
Cat 5e isn't shielded (although you can get it that way), it uses twisted pairs. If your 8 conductor phone cable used twisted pairs, then you'd probably be fine as long as you wired the pairs up correctly.
What sort of "telephone" cable? Phone cable is 3 pair CAT3, which is rated for 10 Mb Ethernet for 100M. However, CAT3 is getting scarce these days, so people often use CAT5. Shielding has little to do with it, unless you're in a noisy environment.
Nice that you got the cat5e cable to work. How long are your runs of cat5e in your tests? You said it was up in your attic but what are we talking about? 20ft? 50ft? More? Thanks.
For short runs (like in a typical home) and low EM noise environments, Cat 5E is often enough. But, remember its cable distance, not human walking distance. Also following paths for power lines (ie: HVAC, kitchen, bathroom) can cause issues. Living near a TV or radio transmitter can also cause problems.
Mr. Tim, you killed it again!! I have a piece of advice from a non-professional lol Try and put music behind the videos, music that fills the empty space, something calming and that corresponds with the emotion you show in the video to shift the feeling of the video to how you feel in the moment! Keep up the good content, Tim, you're an inspiration!
One thing to remember with music is that it oftentimes makes content less accessible. People with disabilities struggle with background music unless it's done really well. It's important to use audio 'ducking' properly when mixing music and voice.
Hi Tim. FYI. I had issues with my 10G connection to TrueNAS. I enabled Jumbo frames and Flow Control on my Unifi switch, 9000 MTU on TrueNAS and 9014 Windows and all is good now. I am getting 10G transfers
If your DHCP server permits, I recommend statically mapping most, if not all, IP addresses for your LAN. It's super easy and really helps create special firewall rules and what not.
Cool. I know you said the cable is twisted all over the place, from attic to basement, but do you have a rough idea how long the run is that worked at 10G?
I used a cat5e for 10g that was 120' give or take a few feet. I used it for years. I needed a second connection is the same location so I ran a cat6a and used that for the 10g. I didn't notice any performance or speed difference between the cat5e or the cat6a, if it were serval hundred feet maybe. At 10g speed the real bottleneck is the machines you're transferring files between. CPU is a factor but the storage drives that files are moved from/to. As Tim did I assume the source drive was an M.2 sending to HDD you get fast transfer till the drive cache fills then you get drive speed. It's not just limited to the destination drive. If you're copying from an SSD to another machine with an M.2 the limit will be the read speed of the SSD for the transfer speed
Not sure if mentioned before but the Copper SFP+ modules run hot and I would not recommend sitting them next to each other due to the heat they put out.
I have been messing with 10g. What I have experienced is that one or two SFP+ 10G ethernet modules isn't a big deal and works well when you need them, but too many in the same switch can overheat - leading to reliability issues. So, I'm using the 10g ethernet only in places that I am unable to run new cables and using DAC or fiber everywhere else. I'm also spacing out the ethernet modules so they don't generate too much heat next to each other.
Very exciting stuff. I have about 130ft run I need to do. All I have laying around is CAT6 and came across the same problem you did. Officially, MikroTik only supports SFP+ to RJ45 for 30 meter runs. The rest of my network is all Unifi, but unfortunately hard to get my hands on nearly ANYTHING 10 gig for a decent price. I even had to resort to ebay because Mikrotik devices are selling out fast. It's good to know that I'll be able to do something temporary until I can terminate my own fiber. By chance, do you know if LAG works with SFP+ 10 Gig? I can try to find a dual port 10 Gig PCIe card and just run 2 drops of cat 6 while I'm at it. That should get me up to 10Gig or higher speeds.
Your milage may vary. I have the same setup, over 5e, it would not connect at 10Gbe when forced, would negotiate at 5Gbe, but connection was unstable over high throughput, had to force it down to 2.5Gbe, which now works just fine. Either way, it's faster than 1Gbe, still a win!
Auto negotiation is a protocol that requires both sides to be using it to function properly. If you manually set 1 side to 10G you should match manual settings on the other side. Otherwise, YMMV and you may see inconsistent negotiation (half duplex etc.).
Beyond 100 Mbps auto-neg is a requirement. OS's sometimes provide a means of "manually" setting a speed but it doesn't do what people think. It's still using auto-neg.
Not sure about those RJ45 SFP but with DAC, you need a SFP+ DAC and SFP+ connector to get the full speed. Using SFP cable on a SFP+ connector wont give you 10GB/s only 1GB. Learned that the hard way..
But will it be _stable_ and _reliable_ . Iperf and file copy might show you good numbers, but will you experience unexplained file copy stalls, or weird disconnections mid transfer etc...those "wth just happened" issues....how has it been since this video was uploaded?
I have a 3ft cat5(not E) cable that will auto detect to 10gbps. I use a 4ft cat 7 cable that I feel works better for the 10gb run though. I have not done tests to see if there is a physical limitation of the cat5 cable compared to new cables.
If you get the correct SFP+ transceiver you would be able to have 802.3-bz with 2.5/5/10 Gbps speeds. On you Linux can you use `ethtool` to verify the speed.
I was pleasantly surprised when Cat5 (non-E) in my walls did 2.5gbe. Unfortunately my little itx gaming rig doesn't have a 10gbe NIC like my server does.
Awesome vid Tim, can you tell what kind of cable/connectors and length of the cat5e cable you are using ? Did you try to check how many packet lost/resend there was ?
@@TechnoTim Thanks for the replay, one question. the link states the cable is 6A, I really want to know what kind of 5e cable you tested as i am wiring my house and putting 5e next to 7a, as emergency backup cable in case something will happen to the more expensive on. would be great if that cable was 10Gb capable just in case :D
Worked in an old DC that had many generations of structured cabling through the place. Unfortunately the main patching bundles that linked the upper and lower floors was all Cat5 and Cat5e pulled around the turn of the century. Rather than pull all new cables we just ran any 10Gbe over the Cat5e and lived with the speeds we got. Most of the time it was still wayyyyy more bandwidth than the workloads running over it as all the serious data was running over fibre.
Would like to find a video (honestly havnt searched yet but will look) on a video for that iperf and windows 7 and unraid. I have 10g in both (and another in a win10pro machine) but not seeing decent speeds, could be the spinning rust not sure. But didn't notice a difference between the 10g and standard 1g....
I also bought a Cat 5e to use with my four QSFPTEK SFPs, it's cheaper than what you use in your video, as you don't have much budget, but my computer-to-computer transfer speed is about 800 Mbits per second
The concern with Cat5e cables at 10Gb speeds is usually to do with signal:noise ratios in the cable, 10GB specifies Cat6A mostly for the shielding and structural integrity of the cable to maintain those signal:noise ratios. Cat5e is rated to handle 100Mhz of signal with little to no noise, with Cat6A rated for up to 500Mhz of signal. In low noise environments and with carefully run cables it's theoretically possible to do a Cat5e run which will meet the requirements to get the signal across. I'd be interested to see what the packet loss is like if you hit the network from multiple hosts on different Cat5e runs at once over say 5-10 minutes of iperf testing, as iperf can actually log your packet loss during the process. Great video though! Was wanting to look at 10G and have a box of good Cat6 I wasn't sure could do the job, now I'm thinking it's just the right thing!
Excellent video Tim, BUT you failed to mention 1 CRUCIAL piece of information, how long is your CAT5e run? This is a question I have researched several times, and like you I saw varied answers. We desperately need to know HOW LONG THE CAT5e RUN IS?
A few things to note. SFP+ to RJ45 transceivers are known to have tighter limitations on 10gb distance, and if not for the fact this is a switch you’re running on, would have been better to seek this functionality natively as there’s a bit of a heat problem. Good to see this worked out for you though. Cat5e is not rated for 10G, so you’re playing the cable lottery. You should get cat6A when you have the opportunity. Most devices will autonegotiate at 1G when they detect the signal quality of Cat5e. Typically, if you truly won the lottery, it will autonegotiate at 10G thinking it’s cat6 or 6a.
my understanding renegotiation in switches, if you have 4 spf+ ports they are shared ports with the last 4 ports of the standard side of the switch, you use the either/or principle and not both...
9:47 Uhg. DHCP on UniFi ... I have a machine on two separate VLANS, one for storage access, and one for everything else on the network. Half the time name resolution gives me the IP for the storage network and the other half of the time, it gives me the IP I actually want back.
Was that a Saitek Eclipse keyboard? Still have one of those on my rack, those things were like bricks! Loved the Red backlights, since everyone did Blue and I hated blue LEDs lol
On short to medium runs usually cat5e isn't an issue with throughput as much as it is with it being damaged over the years or getting interference from other systems. Many people have Cat5e running in a industrial or somewhat hostile environment and they will not have a good time with 10GBE.
When I forget to some MAC address and a device drops off I dump my arp table to a text file. Maps ip to MAC and makes finding it easier. Has saved time at work and home.
Interesting. I use noctua fans in my home lab to keep it quiet so I can have the whole rack next to my monitors and desk. (No basement for me to stick it in anyway) for me a new run would be as easy as unplug and put in new cables because the only thing coming off the home lab is display cables and a kvm for my keyboard/mouse.. its kinda silly having a whole 48U rack next to you while gaming but you get used to it.
When doing an iperf3 test, always test with a Linux machine at both ends. Gives you near wire speed, whereas Windows tends to limit you to 7 and change gbps. Apparently, WSL gives you slightly more, but it is still limited.
Just a general note on auto negotiation of speed and duplex: you should really not mix auto negotiated and statically defined speed/duplex on a link. Most modern machines can handle this miss match and select the correct speed and duplex, as long as you are running 1gig or faster on the statically set machine. Ideally use auto negotiation everywhere, unless you have a specific reason for setting static speed and duplex. Side note, UniFi seems to have some issues with auto negotiation for anything above 1gig, so in your specific case you should consider setting the speed to 10gig and duplex to full on both your switch and your hosts, as it is best practise.
if you do switch to a fixed link speed, do it first on your edge switch and then afterward on the switch which is closer to your controller (in my case a UDM Pro) as in my experience as soon as the edge switch "fixes" to number while the other side is in "auto-negotiate" you'll break the link and no longer be able to reach the edge switch (and switch it too to whatever speed you're moving to).
My workshop to my house is 10GbE using exterior grade MM Fiber, but I need to extend it to other parts of the house. I will probably connect my wife's office with 2.5GbE because that's better than WiFi at least.
I don't know what you did with the cables to begin with. But if you put them in proper cable channels, then it would be easy to tape a new cable to the old one and pull it all the way back with the help of the old cable. No need to crawl in the attic... It could still work without a channel, but could snagg on something and if you stapled it, it obviously won't work at all.
my whole house is cat5e pure copper. I hit 10G no problems. I get about 777MB/Sec transferring to my Storage spaces 3 stripe 32KB, 64KB Allocation size in parity setup. Pure HDD Pool.
I currently have a Ubiquiti 16 port Poe switch and I just purchased a Ubiquiti 24 Port Pro Poe switch. Do I need to do anything prior to unplugging the 16 port and installing the 24 port? I assume the 24 port switch will just pop up to adopt and I just go through adopting and it will work?
At home my server is not that far away from my switch so I have the NIC's aggregated and the same with my workstation so cheap 2.5 gigabit and transfers are "fast enough" for now. When prices come down I'll upgrade.
👍👍👍👍👍 Thanks for sharing this! Currently running Cat6 at 1Gbps, WIFI6 & 1Gbps FTTH. Planning to start my Homelab too. First thing is to get a proper firewall like pfsense; eyeing the 6ports 2.5gb little monster. 10gbps devices looks a little pricy; might need to upgrade to 2.5/5 first. It's Not a question of if it could but: How far & reliably could we push at higher speed & the performance/quality of those devices carrying the traffic. The Category standard is used to determine the minimum speed of performance over a certain distance. Beyond that, your mileage differ. However it's great that you have gone before us and shared your knowledge & findings. Now I know my Cat6 cable should support 10Gb upto about 50m; plentyful enough for my small installation. Once again, thank you!
10Gbe can be finnicky or resiliant... it purely depends on the adapter and its tolerance... the mac mini m1 10gbe interface is not resiliant at all and will try to hold a 10gbe until it gets noise then it just drops down to 1gbe and never recovers... whereas the newer mac studio 10gbe interface will recover and yield an effective higher rate... over the cat 5e that i have thats about a 50 ft run between 2 walls, mac studio gets sustained 6.5g up and 9g down, but the mac mini will only get 6 up / down until it hits noise and it drops to 1gb for ever until you reset the adapter or pull the network cable and reconnect... so a lot of the cat 5e performance has to do with how resiliant the nic and software is.
Every time a new network device is inserted, you have to edit the file so that it assigns an IP by dhcp or enter a manual ip. Things that happen in Ubuntu Server XD. (but I don't remember if it happens in Debian)
Do you run 10 gig at home? 1 gig? Or WiFi only?
I run my server, my computer, and my partner's computer over 10Gb currently. Mikrotik switch and sfp+ cards with fiber modules in each machine. It's so overkill, but it's great for my lack of patience in massive data transfers.
My workstation is in the same room as my rack so, 10G between 3 servers and my workstation (CRS305-1G-4S+), and 1G everywhere else.
I have been doing 1gig but recently have been transitioning to move things over 10 gig. Installed 10gig adapter on the pfsense machine and the ISP router already.
I have been looking for a switch that serves the purpose and this one looks promising. I want to do 10gig on the nas, proxmox server and the pfsense machine.
The house is wired with cat7a running on 10gig, a bit overkill. cat 6 would be the best option as cat7 cables are freaking stiff
I run 10gig to my file server and router (in the same rack) and to my office, but I do want to replace the cat6 run to my office because those rj45 transceivers get hot.
Just to demystify this topic a little bit (because it's less complex than people think):
(modern) Ethernet fundamentally just needs 8 wires to work, so any 8-lead cable can run any speed in theory. They are all wired the same electrically.
What determines if you can practically run a high speed though is the signal quality - which is mostly a function of the length of the cable, but also the quality and Em noise separation. The higher speed, the cleaner the signal needs to be - so either better cabling, or shorter distance.
In a typical home with relatively short wiring you can usually run much higher speeds than the cable is rated for, because the cables rating is usually designed around 100m or so.
Thus, if you have a 10 or 20m cable you can run it far "above spec" without issue, because the short length more than compensates for the worse cable quality.
If you run a higher speed than the signal quality can handle you will get packet loss. That "wall" hits pretty fast, so it tends to work either flawlessly or terribly (or not at all more likely).
I would always recommend checking packetloss under stress when running a cable out of spec. It tends to be be fairly obvious if it's fine or not.
TLDR: If you have old cables in your home, you should test them before upgrading for higher speed. Chances are you won't have to change them - or at worst, maybe only your longest cable run.
Hey my new house was wired with Cat5e and 10G worked just fine (just over 8 like yours), surprisingly.
ROWL in da house!
Its actually kind of funny to see your 'live' reaction to this. This just shows how enthusiastic you are!
Yeah that reaction was completely relatable.
Cat 5e is a beast.... Thanks for doing the test and confirming what I already knew as a network engineer
A lesson I recently learned is that SFP+ modules negotiate 2 speeds - between the switch and the module, and between the module and the PC in your case. The link speed shown in UniFi is only the former…it won’t show the link speed between the switch and the PC. You can only verify the links on either end of the module.
Then why did it auto-negotiate 1G on his machine that also reported 1G? I would think it should have shown auto-negotiated a 10G for the switch with his PC reported connected at 1G.
@@jonmayer A fluke? Wasn’t fully seated in the SFP+ port? I’ve fought the seating issue before.
Don't forget, Windows file copy is only a single thread process so the numbers won't be that fast. Linus Tech Tips ran into the same problem when upgrading their network.
NiTeHaWKnz... Just remember that depends on the SMB version you are using... SMB 3.0+ can be multi-threaded and even multi-adapter using SMB-MultiChannel. You can check this looking at powershell with get-smbconnection and get-smbmultichannelconnection. In my case I have a dedicated 3.1.1 server utilizing 4 40Gb nics and the storage to back it and I am able to read at about 12 GB/s or ~48Gb/s.
Just use robocopy
The copy is working just fine and can hit 10Gbps as he saw in the first few seconds. The problem is the drive speed. It was able to go full speed because it was essentially filling up server RAM as a write cache. Once that cache is full, then you are only getting raw write performance.
This is no longer true as of SMBv3 and the CIFS version that supports the multithreaded CIFS implementation. This is recent (last 5 years or so) but has been present on Windows Server and Client OS since Windows 8.1 was released.
He literally compied files at 1 GB/s, which works out to just under 10Gbps. 1Gbps is equal to about 125MB/s.
My house was built in 1997 before the cat 5e spec was even a thing, and was wired with decent enough quality cat 5 (no "e") that I actually can get 10G over the ~30ft from my living room to the garage.
I've had the same doubts. And found basically the same answers kk I'm just so glad that you've made this video. Amazing! Congrats for your content.
I'm literally in the process of buying a condo and I discovered during inspection that all the phone jacks were wired with Cat5e, so I was already researching 10G over Cat5e when I saw this video. Excellent timing! I'm going to give it a shot after I move in.
I bought my condo 29 years ago and it was wired with CAT3 and daisy chained jack to jack, so not the greatest for Ethernet. However, when I got my cable modem, in the late 90s, my ISP actually ran cables in for me. I wanted the modem in my "office", which is at the opposite end of my unit from where the cable comes in. My ISP sent a 2 man crew who spent 3 hours fishing the cable up inside the wall between my living and bedrooms, along side an air duct, over my closet and bathroom ceilings and into my laundry room, where it crosses my ceiling, then down behind the water heater and through the wall into my office closet. While they did that, I had them pull in a couple of runs of CAT5 (5e wasn't available then). They even patched the drywall where they cut it. In addition, they put red "Data" tags on the coax.
@@James_Knott good guys
I jumped from an HP ProCurve 24-port directly to the 48-PoE-Pro, and couldn't be happier. I also added a fan inside to keep things quite cool. It's a powerful device, and can be chained to multiple copies of itself.
Over 1G network sometime you have to "Hard code" the speed in both side. In a core network, we'll never use auto negotiate, we must hard code the speed at both side, it will more stable and will not suddent drop to other speed.
What do you mean by hard code?
@@perfect.stealth To set speed manually in settings.
That's not a thing and doesn't do what you think it does. Gigabit uses auto negotiation no matter what.
Just moved into a brand new house. I made sure that they ran Cat6 to ensure a degree of expandability in the future. So glad I had that option.
I have the Aruba S2500-48P. 48 ports of gigabit POE and 4 SFP+ ports. I used Finisar 10gb SFP+ multimode modules with 10M of OM3 LC to LC fiber cable. Both worked great connecting the Aruba switch to my Mikrotik 10gb SPF+ switch.
Tim is the window into my future! I'm looking at running a little 10 gig to and from a couple of machines, but have some room to grow in the future.
What you definitely should try is if you got any retransmission. With crosstalk and all that you could have a lot of broken packets
this
where do i look for those? in the network interface stats?
@@AlexanderBukh ethtool will give you anything you need
@@LampJustin thanks!
The main difference between Cat 5e and Cat 6 cabling is signal attenuation.
To get a specific bandwidth we need a minimum amount of signal to noise ratio (SNR) at the receiver, and the sender only has X amount of power it can deliver down the line.
So more attenuation means a shorter distance is needed before the signal strength is insufficient to achieve the needed SNR. However, other factors can also effect SNR, so in more electrically noisy environments Cat 5e can be insufficient at lower distances.
In general for home and even office use, Cat 5e can carry 10 Gb/s Ethernet for 20-30 meters without issue in most situations. However, not always.
Cat 6 has thicker wires that gives the signal a bit more surface area to work with, effectively lowering attenuation. Also makes DC power distribution via PoE easier.
Cat 6 is often also having a spacer that spaces out the 4 pairs from each other, reducing crosstalk and improving SNR a bit more. Some cables have a shield as well to protect the cable from external electromagnetic interference.
Then there is also Cat 7 cable on the market, but the networking industry don't really use it yet. But it tends to have individually shielded twisted pairs, as well as a shield around all four pairs. And on top of that the different pairs use different amounts of turns/twists per unit of length, this is to reduce crosstalk even further.
But in short, the things of importance is the signal to noise ratio at the receiver. Even just hand wiring some loose copper strands with minimal twisting can run 10 Gb/s Ethernet as well, though not particularly far.
You can actually get some pretty long SFP+ cables. The advantage is very very low latency compared with CAT7.
@@wayland7150 The Cat rating of the cable doesn't really affect latency.
But yes, Direct Attached Copper is lower latency than a pair of SFP to Ethernet converters.
Mainly since Direct Attached Copper is literally as the name implies, the Tx differential pair going to the SFT port just gets routed straight over to the Rx pair on the other SFP port.
Ie, no logic, nor any protocol processing.
DAC cables though have signal buffers to ensure that the signal makes it over, but this adds almost no appreciable latency.
Regular optical SFP modules work in the same fashion, just instead the signal buffering drives the light source, or reads the optical sensor on the other side.
@@wayland7150 The average person wouldn't notice the difference in latency. You're talking hundreds of microseconds. and there is no reason to bother with Cat7 for 10 Gbps, it's not even recognized as proper cabling standard either. Beyond Cat 6 it goes up to Cat8. Again there is no purpose for it for 10 Gbps.
@@todayonthebench There is a huge different in latency between CAT6 and DAC. Forget The Average Person because they can't tell the difference between 100meg and 1gbit so have no use for 10gbit. The exceptional people who do care would be very keen on low latency because in some applications that's the crucial factor like gaming.
@@wayland7150 Any latency differences in cables on the local network gets dwarfed by the latency of the internet at large. So for Gaming the local network cable delay isn't really going to have an effect on latency.
The latency of cables only starts to matter when one deals with local network storage for database applications, or for clustered computing workloads. Not gaming.
The X540 intel 10gbit adapters are sometimes finicky. I use them for all of our production equipment and they work fairly well. Check your switch's stats on that port for CRC errors. That would tell you if it's clean.
Tim, thanks so much for doing this test. I have a Ubiquiti US-16-XG UniFi 10G 16 port switch (12x SFP+ & 4x RJ45) on backorder and for RJ45, I have a bunch of CAT5e cables and nothing better, so I'm glad to see those will work for the time being. Because of cost, I was thinking of getting some used Mellanox CX311A 10Gig SFP+ cards + transceivers and fiber cables for my main PC, the Cisco server I have and two NAS's. Everything is in my office so all short runs.
That's awesome and gives me hope. I have a cat5e cable going up one side of my house, across the roof and down the other side. It is a 2 story house, so I think I may be coming close to that 55m, but at a guess, I think it's about 45m. This connects my 2 parts of my homelab, where I have rack servers sitting on a cupboard, and the other end to my office. I have wanted 10Gb, but always thought the cable wouldn't work so never tried, and don't really want to run another cable, though I have a whole spool of cat6 that I was gifted.
Off to eBay to see if I can get some cheap 10Gb switches that support VLANS.
Oh dang that's awesome! Gives me hope for all of the cat 5e I've got running all over the house....now time to shell out some dough for 10G equipment!
Thats great it worked for you. I tried connecting my XG-16 (in the office) to the Unifi Aggregation Switch in the garage about 9 months ago at 10gbe and it was a disaster. Admittedly the in-house wiring came "as is" when we rented but it is a high-spec new build house so I'd have guess cat-6 but it may be cat-5e and wiring distance is a bit hard to know but I'd guestimate at 100-150ft. Anyway, I was able to connect at 10gbe at the link level but the amount of packet loss I experienced was off the charts. Still you're success has me wondering again and it would certainly make my geek dreams come alive.
Also note, if you have the right SFP+ modules (aka, those which Unifi supports) I believe it should auto-negotiate to 10 gbe. With my UDM Pro to my Unifi Aggregation Switch it works fine.
Thanks! I will do some real world tests soon to measure packet loss! Also, I did buy official transceivers so not sure why my Windows PC chose 1Gbe but mu Linux server chose 10Gbe. Thank you for the tips!
Cool project. On your auto-negotiation issues, I have heard that SFP+ to RJ45 modules can have issues negotiating 10G on Ubiquiti and is just better to set the desired speed manually.
ive run cat6a non-shielded cable and keystones. i have no 10gb equipment (yet), however i do have some long POE+ runs. wanted a thicker gauge wire to reduce heat generation, and future 10gb expansion. pulling cable sucks sometimes.
To have a continuous test of iperf run the following: iperf3 -c -P 8 -R -t 120
That TH-cam window zoom in and out was pretty slick
Thank you!
@Tim, to be sure you have a stable 10Gig. Best to check your Switchports on the switch. Look if they do not flap and you do not see any errors on those ports.
Where the files transfered successfully? Check the hashes :) TeraCopy does it automatically :)
I just upgraded from a 50mbt fixed wireless solution to a 2gbt fiber service (Thank god after waiting 20 years for REAL internet here!!) so I'm looking to upgrade my infrastructure here in the house. Running Cat5e as well so may just use what's already here. I run a lot of stuff over wireless but all of my core services are on hard lines.
The SFP+ to RG45 converters are notoriously quirky and its no surprise that auto-negotiation doesn’t work reliably. Just set things to 10Gbe (like you did) and all should be well.
You might alway want to play around with setting jumbo frames (9k MTU). It can make a huge difference in transfer speeds at 10Gbe, especially to/from windows which does single-threaded IO for most file transfers.
for 2.5gbe what mtu to set? o-O; is default good enuff?
I need to lay new wires and will probably shell out the cat7. It’s gonna be an expensive mess digging concrete
The amount of money I’ve spent at microcenter over the years is crazy
Didn’t know iperf existed, really useful. Thanks for that
I've watched all your videos, you're an awesome dude.
When that 10Gb connection kicked in, I think that's the first off-script, out-of-character I've ever seen you. Most the time, it seems more like you're reading from a script; this was more akin to the first time you ever had an orgasm and didn't know what was going on.
Absolutely loved it brother Timothy, but thanks for not showing us your pants afterward.
Yes you can, it really depends on the quality of the cable and distance along with how its run (EMF Exposure)
Love it Tim, congrats!!
I have upgraded some Weeks Ago (USW-Pro-24), but just my Server (DAC, its in the Rack with the switch) and my Desktop (Upstairs, RJ45). I wired the house with CAT7 cable when we bought it 8 years ago. I ain't got working full 10 Gbit at first try at my Desktop. I thought I messed up some shields at the Patch Panel, but it was a faulty (bought) CAT6A Cable wich connected the Desktop with the Patchpanel. Greets from Germany
That is awesome. I love to bust myths...
I'm currently running 1 gig through telephone wire which happens to have 8 wires... Everyone even said its not possible because its litterly not shielded at all..
Paket loss is a bit higher, I would say wlan standards. But speed and everything is awesome.
Love to see you manged 10gig with cat5... Sometimes it makes me think cable industry is a bit... Sketchy and never really tests their new standards...
Cat 5e isn't shielded (although you can get it that way), it uses twisted pairs. If your 8 conductor phone cable used twisted pairs, then you'd probably be fine as long as you wired the pairs up correctly.
What sort of "telephone" cable? Phone cable is 3 pair CAT3, which is rated for 10 Mb Ethernet for 100M. However, CAT3 is getting scarce these days, so people often use CAT5. Shielding has little to do with it, unless you're in a noisy environment.
Good video. I wired my whole home with Cat5e around 2000. When consumer grade 2.5 Gbe switches get a little cheaper I hope to pick one up.
Nice that you got the cat5e cable to work. How long are your runs of cat5e in your tests? You said it was up in your attic but what are we talking about? 20ft? 50ft? More? Thanks.
Thin cables aren't really compliant over distance so for best performance don't use them as patch cables for in-wall runs.
Most offices don't have that luxury as they are wired with cat5e
You might not have IPMI built in, but you should put a Pi-KVM or Tiny Pilot in front of your PC-Server conversion so you get those capabilities. :)
For short runs (like in a typical home) and low EM noise environments, Cat 5E is often enough. But, remember its cable distance, not human walking distance. Also following paths for power lines (ie: HVAC, kitchen, bathroom) can cause issues. Living near a TV or radio transmitter can also cause problems.
Indeed it's all about quality/ noise on the line. Hence cat6 is better isolated.
This is a great example of troubleshooting. 👏
Nice job! Thanks for the video.
Mr. Tim, you killed it again!!
I have a piece of advice from a non-professional lol
Try and put music behind the videos, music that fills the empty space, something calming and that corresponds with the emotion you show in the video to shift the feeling of the video to how you feel in the moment!
Keep up the good content, Tim, you're an inspiration!
Thank you!
One thing to remember with music is that it oftentimes makes content less accessible. People with disabilities struggle with background music unless it's done really well.
It's important to use audio 'ducking' properly when mixing music and voice.
Iperf was updated to the iperf3. Recommend you to try 👍. Great video, by the way!
Hi Tim. FYI. I had issues with my 10G connection to TrueNAS. I enabled Jumbo frames and Flow Control on my Unifi switch, 9000 MTU on TrueNAS and 9014 Windows and all is good now. I am getting 10G transfers
Thank you! I ill give this a try!
If your DHCP server permits, I recommend statically mapping most, if not all, IP addresses for your LAN. It's super easy and really helps create special firewall rules and what not.
Yea static ip for sure. Especially your servers!
Cool. I know you said the cable is twisted all over the place, from attic to basement, but do you have a rough idea how long the run is that worked at 10G?
I used a cat5e for 10g that was 120' give or take a few feet. I used it for years. I needed a second connection is the same location so I ran a cat6a and used that for the 10g. I didn't notice any performance or speed difference between the cat5e or the cat6a, if it were serval hundred feet maybe. At 10g speed the real bottleneck is the machines you're transferring files between. CPU is a factor but the storage drives that files are moved from/to. As Tim did I assume the source drive was an M.2 sending to HDD you get fast transfer till the drive cache fills then you get drive speed. It's not just limited to the destination drive. If you're copying from an SSD to another machine with an M.2 the limit will be the read speed of the SSD for the transfer speed
It's roughly 120 ft, but I think I can check cable length on my UDM!
Generally Cat5e while not specced for 10G will handle it up to about 40m to 50m. Shorter is better.
Not sure if mentioned before but the Copper SFP+ modules run hot and I would not recommend sitting them next to each other due to the heat they put out.
I have a 1 Gbps network, and mostly Cat 5E wiring, my transfer speed is around 800 MBits per second from computer to computer. I do Lan speed Test.
Either Auto Neg or hard set on both sides. If you have a mismatch of hard setting an interface then problems will occur.
I have been messing with 10g. What I have experienced is that one or two SFP+ 10G ethernet modules isn't a big deal and works well when you need them, but too many in the same switch can overheat - leading to reliability issues. So, I'm using the 10g ethernet only in places that I am unable to run new cables and using DAC or fiber everywhere else. I'm also spacing out the ethernet modules so they don't generate too much heat next to each other.
Very exciting stuff. I have about 130ft run I need to do. All I have laying around is CAT6 and came across the same problem you did. Officially, MikroTik only supports SFP+ to RJ45 for 30 meter runs.
The rest of my network is all Unifi, but unfortunately hard to get my hands on nearly ANYTHING 10 gig for a decent price. I even had to resort to ebay because Mikrotik devices are selling out fast.
It's good to know that I'll be able to do something temporary until I can terminate my own fiber. By chance, do you know if LAG works with SFP+ 10 Gig? I can try to find a dual port 10 Gig PCIe card and just run 2 drops of cat 6 while I'm at it. That should get me up to 10Gig or higher speeds.
Your milage may vary. I have the same setup, over 5e, it would not connect at 10Gbe when forced, would negotiate at 5Gbe, but connection was unstable over high throughput, had to force it down to 2.5Gbe, which now works just fine. Either way, it's faster than 1Gbe, still a win!
2.5 gbe is actually invented just for that. So 2.5gbe is compatible with cat5e. Otherwise the next step was directly 5gbe, which requires cat6(a)
Reqlly liked that video! Keep up the good work
Auto negotiation is a protocol that requires both sides to be using it to function properly. If you manually set 1 side to 10G you should match manual settings on the other side. Otherwise, YMMV and you may see inconsistent negotiation (half duplex etc.).
Beyond 100 Mbps auto-neg is a requirement. OS's sometimes provide a means of "manually" setting a speed but it doesn't do what people think. It's still using auto-neg.
Not sure about those RJ45 SFP but with DAC, you need a SFP+ DAC and SFP+ connector to get the full speed. Using SFP cable on a SFP+ connector wont give you 10GB/s only 1GB. Learned that the hard way..
But will it be _stable_ and _reliable_ . Iperf and file copy might show you good numbers, but will you experience unexplained file copy stalls, or weird disconnections mid transfer etc...those "wth just happened" issues....how has it been since this video was uploaded?
I have a 3ft cat5(not E) cable that will auto detect to 10gbps. I use a 4ft cat 7 cable that I feel works better for the 10gb run though. I have not done tests to see if there is a physical limitation of the cat5 cable compared to new cables.
If you get the correct SFP+ transceiver you would be able to have 802.3-bz with 2.5/5/10 Gbps speeds. On you Linux can you use `ethtool` to verify the speed.
I was pleasantly surprised when Cat5 (non-E) in my walls did 2.5gbe. Unfortunately my little itx gaming rig doesn't have a 10gbe NIC like my server does.
Officially cat5e (with e) is fully supported for 2.5gbe. That was the whole point of 2.5 gbe. Otherwise the next step was directly 5 gbe.
I didn't know you could run ethernet over spec. This is so cool!
Awesome vid Tim, can you tell what kind of cable/connectors and length of the cat5e cable you are using ? Did you try to check how many packet lost/resend there was ?
amzn.to/3yvdmdO o.5 ft
@@TechnoTim Thanks for the replay, one question. the link states the cable is 6A, I really want to know what kind of 5e cable you tested as i am wiring my house and putting 5e next to 7a, as emergency backup cable in case something will happen to the more expensive on. would be great if that cable was 10Gb capable just in case :D
Worked in an old DC that had many generations of structured cabling through the place. Unfortunately the main patching bundles that linked the upper and lower floors was all Cat5 and Cat5e pulled around the turn of the century. Rather than pull all new cables we just ran any 10Gbe over the Cat5e and lived with the speeds we got. Most of the time it was still wayyyyy more bandwidth than the workloads running over it as all the serious data was running over fibre.
Would like to find a video (honestly havnt searched yet but will look) on a video for that iperf and windows 7 and unraid. I have 10g in both (and another in a win10pro machine) but not seeing decent speeds, could be the spinning rust not sure. But didn't notice a difference between the 10g and standard 1g....
I also bought a Cat 5e to use with my four QSFPTEK SFPs, it's cheaper than what you use in your video, as you don't have much budget, but my computer-to-computer transfer speed is about 800 Mbits per second
The concern with Cat5e cables at 10Gb speeds is usually to do with signal:noise ratios in the cable, 10GB specifies Cat6A mostly for the shielding and structural integrity of the cable to maintain those signal:noise ratios. Cat5e is rated to handle 100Mhz of signal with little to no noise, with Cat6A rated for up to 500Mhz of signal. In low noise environments and with carefully run cables it's theoretically possible to do a Cat5e run which will meet the requirements to get the signal across.
I'd be interested to see what the packet loss is like if you hit the network from multiple hosts on different Cat5e runs at once over say 5-10 minutes of iperf testing, as iperf can actually log your packet loss during the process. Great video though! Was wanting to look at 10G and have a box of good Cat6 I wasn't sure could do the job, now I'm thinking it's just the right thing!
older tech books state up to 1 Gb
When’s the next video in the upgrade series coming out?
soon!
Excellent video Tim, BUT you failed to mention 1 CRUCIAL piece of information, how long is your CAT5e run? This is a question I have researched several times, and like you I saw varied answers. We desperately need to know HOW LONG THE CAT5e RUN IS?
I think it's about 120 ft, there's a command I can run on my UDM that tells me. Will report back!
A few things to note.
SFP+ to RJ45 transceivers are known to have tighter limitations on 10gb distance, and if not for the fact this is a switch you’re running on, would have been better to seek this functionality natively as there’s a bit of a heat problem. Good to see this worked out for you though.
Cat5e is not rated for 10G, so you’re playing the cable lottery. You should get cat6A when you have the opportunity. Most devices will autonegotiate at 1G when they detect the signal quality of Cat5e. Typically, if you truly won the lottery, it will autonegotiate at 10G thinking it’s cat6 or 6a.
my understanding renegotiation in switches, if you have 4 spf+ ports they are shared ports with the last 4 ports of the standard side of the switch, you use the either/or principle and not both...
Oh boy! CAT5e and 10Gbps..!!! You've got me thinking, for sure!
9:47 Uhg. DHCP on UniFi ... I have a machine on two separate VLANS, one for storage access, and one for everything else on the network. Half the time name resolution gives me the IP for the storage network and the other half of the time, it gives me the IP I actually want back.
Was that a Saitek Eclipse keyboard? Still have one of those on my rack, those things were like bricks! Loved the Red backlights, since everyone did Blue and I hated blue LEDs lol
It was! Good eye!
I guess this makes me feel better about running Cat6 though my house if I need 10 gig vs the jump to cat 6a. How long was that cat 5e run in feet?
Around 120 ft
Think its just a question of how long the 5e cable can be. For most homes port to port length is probably not an issue.
The amount of EM noise as well actually. Like close by power lines etc. Or microwaves.
On short to medium runs usually cat5e isn't an issue with throughput as much as it is with it being damaged over the years or getting interference from other systems. Many people have Cat5e running in a industrial or somewhat hostile environment and they will not have a good time with 10GBE.
When I forget to some MAC address and a device drops off I dump my arp table to a text file. Maps ip to MAC and makes finding it easier. Has saved time at work and home.
Why didn’t i think of that. Definitely using that one, thanks man!
4:24 The auto focus freaking out here made me laugh
🤣 That's one of those things you don't realize until after!
Hi Tim. Have you enabled Jumbo Frames ?
Hi! I have not yet. I am not sure what the benefits are.
Thinking of OCD, at 11.37, there are two spaces between dhcp4: and yes on the last line. :)
Haha! I noticed too when editing!
Interesting. I use noctua fans in my home lab to keep it quiet so I can have the whole rack next to my monitors and desk. (No basement for me to stick it in anyway) for me a new run would be as easy as unplug and put in new cables because the only thing coming off the home lab is display cables and a kvm for my keyboard/mouse.. its kinda silly having a whole 48U rack next to you while gaming but you get used to it.
When doing an iperf3 test, always test with a Linux machine at both ends. Gives you near wire speed, whereas Windows tends to limit you to 7 and change gbps. Apparently, WSL gives you slightly more, but it is still limited.
Good to know, really. Saving me few bucks. Thanks
Does it matter if you are using Cat5e jacks or do the jacks need to be Cat6a in the wall?
Why not use the cat-5 as a pull line for cat-9?
Just a general note on auto negotiation of speed and duplex: you should really not mix auto negotiated and statically defined speed/duplex on a link. Most modern machines can handle this miss match and select the correct speed and duplex, as long as you are running 1gig or faster on the statically set machine. Ideally use auto negotiation everywhere, unless you have a specific reason for setting static speed and duplex. Side note, UniFi seems to have some issues with auto negotiation for anything above 1gig, so in your specific case you should consider setting the speed to 10gig and duplex to full on both your switch and your hosts, as it is best practise.
if you do switch to a fixed link speed, do it first on your edge switch and then afterward on the switch which is closer to your controller (in my case a UDM Pro) as in my experience as soon as the edge switch "fixes" to number while the other side is in "auto-negotiate" you'll break the link and no longer be able to reach the edge switch (and switch it too to whatever speed you're moving to).
My workshop to my house is 10GbE using exterior grade MM Fiber, but I need to extend it to other parts of the house. I will probably connect my wife's office with 2.5GbE because that's better than WiFi at least.
I don't know what you did with the cables to begin with. But if you put them in proper cable channels, then it would be easy to tape a new cable to the old one and pull it all the way back with the help of the old cable. No need to crawl in the attic...
It could still work without a channel, but could snagg on something and if you stapled it, it obviously won't work at all.
Unless they were stapled in the rafters. :(
Great vid!! Someday I'll get >1 gig setup in this house!!😁👍
my whole house is cat5e pure copper. I hit 10G no problems. I get about 777MB/Sec transferring to my Storage spaces 3 stripe 32KB, 64KB Allocation size in parity setup. Pure HDD Pool.
Nice!
I currently have a Ubiquiti 16 port Poe switch and I just purchased a Ubiquiti 24 Port Pro Poe switch. Do I need to do anything prior to unplugging the 16 port and installing the 24 port? I assume the 24 port switch will just pop up to adopt and I just go through adopting and it will work?
At home my server is not that far away from my switch so I have the NIC's aggregated and the same with my workstation so cheap 2.5 gigabit and transfers are "fast enough" for now. When prices come down I'll upgrade.
Agg is definitely a great option!
@@TechnoTim - It's incredible how well it works.
How about error/crc counters during transmission?
That stereo audio at 3:22 scared the life out of me
👍👍👍👍👍 Thanks for sharing this! Currently running Cat6 at 1Gbps, WIFI6 & 1Gbps FTTH. Planning to start my Homelab too. First thing is to get a proper firewall like pfsense; eyeing the 6ports 2.5gb little monster. 10gbps devices looks a little pricy; might need to upgrade to 2.5/5 first.
It's Not a question of if it could but:
How far & reliably could we push at higher speed & the performance/quality of those devices carrying the traffic.
The Category standard is used to determine the minimum speed of performance over a certain distance. Beyond that, your mileage differ.
However it's great that you have gone before us and shared your knowledge & findings.
Now I know my Cat6 cable should support 10Gb upto about 50m; plentyful enough for my small installation.
Once again, thank you!
Im wondering what adapter are u using on PC side to get 10 gig ???
It's in the description!
10Gbe can be finnicky or resiliant... it purely depends on the adapter and its tolerance... the mac mini m1 10gbe interface is not resiliant at all and will try to hold a 10gbe until it gets noise then it just drops down to 1gbe and never recovers... whereas the newer mac studio 10gbe interface will recover and yield an effective higher rate... over the cat 5e that i have thats about a 50 ft run between 2 walls, mac studio gets sustained 6.5g up and 9g down, but the mac mini will only get 6 up / down until it hits noise and it drops to 1gb for ever until you reset the adapter or pull the network cable and reconnect... so a lot of the cat 5e performance has to do with how resiliant the nic and software is.
Every time a new network device is inserted, you have to edit the file so that it assigns an IP by dhcp or enter a manual ip. Things that happen in Ubuntu Server XD. (but I don't remember if it happens in Debian)