Why DAC Cables Kicks Fibers A$$, in DataCenter Rack - 1284
ฝัง
- เผยแพร่เมื่อ 6 ต.ค. 2024
- When working with high speed network server connections in the data center. There are more options. Fiber cables with SFP+, DAC or AOC or even RJ45 10Gbit. But this video is about picking DAC over FIBER,, for the short connections in the data center racks.
[Affiliate Links]
Link US - DAC Cables : amzn.to/3USrwAw
Link UK - DAC Cables : amzn.to/3H1dtTL
___________________________________________________________________________________________________
/ myplayhouse
For 3$ a month, you get an extra weekly "What's UP" update video. Just for my Patrons. The Support I resave on patreon is all used on stuff to make interesting videos on TH-cam.
My PlayHouse is a channel where i will show, what i am working on. I have this house, it is 168 Square Meters / 1808.3ft² and it is full, of half-finished projects.
I love working with heating, insulation, Servers, computers, Datacenter, green power, alternative energy, solar, wind and more. It all costs, but I'm trying to get the most out of my money, and my time.
Totally agree Morten. Fiber is good for distance, but not always practical or cost effective for short distances. Copper still work very well.
Hi Bert Blankenstein
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
DAC == Direct Attached Copper. DAC started out with all-copper but you can actually find fiber DAC cables. Which seems weird but it works. My assumption is it uses low-power lasers vs. MM/SM transceivers. Enjoyed your video! A lot of people use SFP/SFP+/QSFP+ at home for their labs. I couldn't get by without DACs because the cost is so small compared to fiber or RJ45 transceivers.
fiber DAC is called active DAC it very similar to fiber plus 2 optics just integrated and can't be separated and probably optic has better calibrated power for just enough power to transmit the distance as it could not be changed as with normal optic when connecting fiber of various lengths. DAC has least heat and uses least energy, then optics, and then there is SFP to ethernet adapters that burn power and get hot like there is no tomorrow.
Hi Doesnt Compute
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
@@MyPlayHouse I love ALL of your videos! :) You do a great job and love your DC setup!
@@vaidkun Yes, SFP and SFP+ RJ45 transceivers use a lot of watts and are short-range. But still nice to have! I use a mix of them here as well.
@@vaidkun direct fiber is called AOC = Active Optical Cable, perfect for distance > 7m. DAC cables are limited to 7m
Totally enjoyable video, funny and educational all in one, well done!!!
Hi @carlyleroberts3995
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
Very good video, even though I disagree!
In the data center, if you have 20 or 30 connections in a rack - even just to the top of the rack - the cables of DACs quickly become very difficult to manage. Fiber, on the other hand, takes up very little room, is more flexible, and is much much easier to pull to another rack (no need to push and pull an SFP through the tray). And, at least for us, dirty fiber/SFPs isn't an issue as the covers only come off the second before it's inserted, and fiber is never reused.
As single mode fiber and optics have gotten cheaper, we've moved to a "Single Mode Everywhere" design. For 10G we use LRM transceivers (as opposed to SR or LR) to limit the costs and the power use, and they have the added benefit of working with single mode or multimode in case we are forced to use MMF due to legacy equipment.
I'm not completely opposed to using DACs (and you make some good points), but I'd prefer to use fiber any day.
It is a lot of work checking cleaning and checking all those fiber cables with a microscope,, been doing that all week.
I worked in a few datacenters, no top of rack switches in the older ones, only the newest one had them. DAC was used to connect the NetApp Racks to each other and then fiber to the switches
Hi sitte
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
We've used AOCs instead of DACs in our datacenter, mostly to ensure the lowest latency possible while avoiding the issues around LC and other connectors.
Hi @walterstarr1588
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
Exactly
wait .. No DAC is actually lowest latency. Since you do not have to convert the signal to light and light back to digital.
@@MelroyvandenBerg Wasn't my experience in the real world.
There is also a reduction of latency in the DAC as there is no conversion with takes time. I believe FS has an article on their site with test results of fiber vs. copper.
Hi Geoff Seeley
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
In the rack, I prefer fiber interconnects between equipment I want electrically isolated - that way if one of my outdoor security cameras happens to pick up a surge from lightning, only the devices electrically wired to that particular switch would be potentially exposed. You'd be surprised at how easy it is for lighting to induce a charge on an ethernet cable.
If both switches are in the same room, and both switches are connected to the same search protector, would that really matter?
@@raiden72 So the lighting strike comes in to the (copper) cable on the camera to the switch doing POE. And blows it. If it's plugged into a REALLY GOOD and REALLY NEW UPS/surge protector, it will likely not feed-back into the house, office, DC circuits. Fiber cameras are rare, expensive, yada, yada..... Which is why well designed huge warehouse security systems will often run the copper cables (CAT5e/6a) to a small switch, one that handles say 8 cameras, and then it goes fiber to the core switches.
@@raiden72 It would if it travels over the network cable (ethernet or DAC). I'm not referring to surges coming over the power line, as those should be stopped by the UPS
@@Doesntcompute2k Precisely
Yes fiber would help on lightning,,, not that big of a problem here :-/
DAC Cables are very nice. We are using 3 of them in our two racks, for 2 Proxmox Hosts and one (now) in spare for a TrueNAS in the future. At home I use fiber, but I had to go through my whole appartment, so DAC is a little bit short and to thick...
Hi Chris93Nbg
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
For me, the two biggest deciding factors as to what type of cable I will use is length and price.
DACs are shorter and cheaper, whereas fibre are longer and more expensive.
I use, I think it's 15 cm SFP+ 10 Gbps DACs to go between my NAS systems (which have been decommissioned).
And then I have four 1 m QSFP28 100 Gbps DACs for my compute nodes to go to my Mellanox 100 Gbps switch.
And then I think I have to 100 m long AOC cables, one of which goes to my system which runs the LTO-8 tape drive, and the other used to go to my micro HPC cluster headnode, but now goes to the second port on my consolidated server.
(Yes, I run 100 Gbps Infiniband in my basement.)
I have a litle 10Gbit in my datacenter but only 1Gbit to my PC.
I like DACs for short distances in the same rack. I use Digitus and they work in all servers and switches i have. As GBICs I use TP-Link. 10gbit cost about 30€ and work perfectly in DL380.
All fiber-cables and gbics are way cheaper than years ago. But switches with more than 4 sfp+ ports are still expensive. I have one 16 port sfp+ and hope it will never get damaged because it was really expensive (~3000€ at the time of purchase). And now switches with that many ports are hard to find and often not available.
@@deepspacecow2644 I completely agree! And Arista (used) as well. Ebay can be your friend on this. Both for SFP-, SFP+-, QSFP-based and copper (RJ45) switches. I have a 48p RJ45 10Gbps Arista I got for $640USD w/4 QSFP+ and a 96p RJ45 10Gbps Cisco Nexus for around $800USD w/6 QSFP+. I've used DACs because of cost, ease-of-use, etc. They just work. "Cisco DACs" tend to be compatible with most NICs and switches. If this is for home lab or a small office, it's the way to go.
@@Doesntcompute2k I bought an edgecore as4610-54p. Nice switch, 48 poe ports, 4 sfp+, 2 qsfp+, all open source for $80. I want to use fiber because ebay optics are dirt cheap, I want experience with fiber cables, and keystone lc jacks in a patch panel will look nicer than a dac hanging around.
Hi Matthias Lange
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
100% agree fiber for long runs greater than 200M - thanks for another great video!
I think you should switch to fiber before 200M :-)
@@MyPlayHouse game changer cable gives you 1g at 200M works 👍 - I'd go to it before fiber!
Funny, you just answered my question in my head of last days. I was checking 10Gbit options for my Rack with NAS and Hyper-V HPE server stuff and was thinking about Mikrotik 10Gb switch and if i can connect via DAC cable as i found Mikrotik DAC so cheaper in comparison with SFP+ modules + cable itself, Fibre or RJ45. And i just did not understand, why the hell is that so cheaper, i was not sure, if it would even work, or if DAC is just some connection only between switches itself.
I have a video comming next week with "all" the connection options. :-)
I work in a datacenter as a security guard and I keep on encountering discarded LC cables, UTP cables and DAC cables(!) in the e-waste container. So far I am still unsuccessful in explaining why companies would throw away their valuable DAC cables!? Sure, they might not match the length requirements of new builds; but DAC cables do not break as easily as LC cables. In addition to this; DAC cables are rare and more expensive than ANY type of UTP cable. Do you know any scenario in which DAC cables which seem undamaged are better thrown away as e-waste?!
I agree with you,, that is a lot of money to throw out..
Another advantage of DAC cables is low power consumption (arround 0.3 Watts at each end), however length of DAC is limited to 7m. That's the reason, why I am using an AOC cable (power consumption arround 1 Watt at each end) to connect two switches at different rooms at my home, all other connections are either connected using DAC or 2.5 Gb SFP+ RJ45 transceivers or 8x2.5Gb + 10Gb SFP+ switches. Meanwhile these kind of switches became cheaper, If you buy them at Aliexpress. Regarding DAC and AOC resellers, I totally agree FS provides good quality, no issues.
:-) I have a 8 port 2,5Gb with 10Gb SFP+ in order from Aliexpress :-)
@@MyPlayHouse I guess, similar to my switch, which runs now for two month, without any issues, once I've understood the UI. Price 77€ is fine, however two complains, switch does not Support SNMP and Management IP via VLAN, however, good product. Best regards from Bavaria to Denmark (or Portugal)
Since upgrading to 10gbe both home and shop, we went all dacs. so easy and simple.
Good choice!!
I do agree to your point of maintainability, but you also get more flexibility with fiber cables. In my case, the switch is 24x25Gbps+2x100Gbps ports, the fiber cables allow up to 100Gbps, the SFPs I got were 25Gbps SFP28 from FS, and the NICs are all 10Gbps (SFP+), the 25Gbps NICs are still around 250€ each and for 4 servers is quite an investment, so the plan is to keep as much as I can 25Gbps capable and replace the NICs to 25Gbps when prices lower a bit more. I do understand your point about the DAC cables, but you lose a bit of flexibility it really depends you your specific case and how often you're going to change things.
We also use 25/10Gbit SW with 2 or 4 100Gbit uplink.. But as we use a Fiber x felt (switchboard) we can´t use DAC at all :-(
We do use Direct Attached Copper to interconnect switches with LAG.
Hi aRndBelgianGuy
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
Cable managing more than a dozen DACs in the same rack quickly becomes a problem since the cables are thicker and you cannot adjust the length of each cable by redoing the plugs like RJ45/BNC/others. Fiber optics, although has to be deal with care they are much thinner and easier to cable manage when the number of cables becomes a problem.
That is somewhat truth,, it is not possible to make them longer or shorter.. but they are really not that much thicker than what we use in RJ45 connections,, there are thinner versions out there :-)
Power usage has become a hot topic these days. Can you do a power draw comparison between fiber (sfp modules) and dac? This would highlight how much power you can save with dac cables instead of using fiber
Would be interesting,, but I would kind of have to measure the power usages of a PCI card,, and that would be complicated.. humm
@@MyPlayHouse Maybe it would be easier to test with a switch? And if the results are not distinguishable with 1 SFP/DAC cable you can just add mor SFP modules until you see a difference in power consumption.
For my DC... We have a bin of a 10G DAC cables... But a lot of our servers go 25G now.... My company stock AOC (Active Optical Cable), it's a cable (fibre + 2 transceivers hard wired), and it's nice to manage yes, but one faulty wire... Left a bad experience for me, much rather regular DAC copper cables 😂
Does DAC do 25Gbit,, now? I thought it is mostly 10Gbit
@@MyPlayHouse there are 25G DACs, even 100G QSFP DACs haha...
I use dual port 10 gigabit SFP+ NICs with DAC in my home network to connect my workstation to my server. Works great for iSCSI/NFS/CIFS and internet connectivity. It was actually quite a bit cheaper than buying 10 gigabit RJ45 ethernet NICs. I had to force the driver to install on Windows 10 on my workstation. The installer refused to want to install on anything other than Windows Server.
:-/ My tiny Lenovo,, does not do 10/25Gbit
I don't know about you Morten, but i've already switched to 100Gb ethernet network and fiber is the way to go :) DACs -> thick, limited to 1-3m, expensive (!)
Working with 100G i can say leave Multimode and go Singlemode 2km easier management and cheap enough. However i do like DACs when possible because they are cheap. 100G-QSFP28 3m like 40€ you cant get there with optics. for 25G we'd be at around 25€ for 3m all new. 100G i a bit thicker than the normal SFP ish cables but manageble
@@niklasp.5847 i got QSFP28 CWDM4 modules 25euro each + 5 euro for 5m OS2 cable :)
BTW DACs can go to 25m. Just depends on the vendor and cabling. And they work to that distance BUT yeah, for 100G and 400G we use fiber. And I do agree with some other comments: SM vs. MM is the choice for those runs. Even if you don't want to run it "1.5Km," SM just gets it done without any issues. Short runs? MM all the way.
We do use 100Gbit between switches,, but from switch to Server it is 25Gbit and 10Gbit on older stuff.
Done a bit of service work, and agree that DAC is really handy for top-of-rack connections *unless* your server needs to be extended regularly with the DACs remaining connected via articulated cable management arms - such as top-loading dense storage servers like the Gen8/Gen9 HPE 45xx, designed to stay online during disk swaps.
In such cases, you can get bend radius right with fibre and not have to worry much after that, but I've seen what looked like copper DAC cables reaching fatigue/work hardening failure from regular disk changes on top load servers.
(aside: Pretty sure I was the first, and also possibly the last guy who ever had to install 2*IB + 3*Cat5 + 16*12G SAS cables through the HPE Gen9 4520 cable management arm, to allow online disk changes in the 4520 chassis while the 2 server blades stayed connected to 2 multipathed/multihomed JBOD chains; HPE Lustre Scalable Storage was *only barely* that scalable. My pictures of what I made work went in the installation guide, and were... daunting.)
Anyway - I've seen instances where arm extend/retract cycles and challenge of keeping strain off the DAC SFP connectors when such servers were fully extended led to intermittent network failures - making the disk replacement hazard window more hazardous, and leading to "complicated" service cases that made life great for the field engineers. :^\
Hi Jason Ozolins
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
Definitely love the convenience of DAC cables.
Great,, on more on the way.
The 10Gbe transceivers get dangerously hot. Mine are currently at 85 degrees celsius. They will burn you.
Yes,, that is really weird.
DAC all the way ideal solution if you stay within the same cabinet, Or to cross connect two node / server HCI solution without a very expensive switch. And if there is something wrong it's easy to diagnose by replacing the cable,
Hi andre van Gijsel
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
This is likely a non-issue for most people, but no one else mentioned it. The DAC cables have 3X the latency of the fiber connections. We're talking about 0.3NS VS 0.1NS, but if your system uses lots of small packets, this would be relevant.
These units are incorrect. Speed of light propagation time over 3 metres is 10 nanoseconds.
I think Cain-san said about 10GBase-T copper, 10GBase-T has relatively large delay, about 3ms, because of forward error correction which is pretty complex multi stage algorithm. Imagine 1.5MB of data is stored on the FIFO of FEC! While Passive DAC that Morten-san shows is much faster than 10GBASE-T, moreover passive DAC is slightly faster than fibre optic.
Actually the opposite is correct, DAC has lower latency because at such short distances the double conversion between electric and optic signals with fiber takes too much time
Here they say 300ns vs. 100ns
community.fs.com/blog/10gbase-t-vs-sfp-which-one-is-the-best-option-for-10gbe-data-center-cabling.html
@@MyPlayHouse www.arista.com/assets/data/pdf/Copper-Faster-Than-Fiber-Brief.pdf
Morten sir, the SFP's and fiber "they use power" they use less power than old rj45
I would not be so sure.
@@MyPlayHouse I would be. You say "oh but those get hot" rj45 get WAY hotter it's just that their processing in rj45 based switches is offloaded to the switch. If you've ever taken switch apart you would see the heatsinks. fiber just does light, it doesn't need to do electric signals so it's way less power especially in low distance
I must say, I do like me some Infiniband 40Gb copper 😎 QSFP+
I like me some 25Gbit :-)
DACs are fantastic but as you said a lot bigger than fiber... can get bulky pretty easily if you have lots of connections to make 😅
And FS dacs are still on the thin side, Juniper or Arista 10-25G we use at work are a huge PITA, thick, hard to bend and if you ever need to remove one from a bundle in a cable organizer good luck 😂they stick together like glue!
Hi Exploited89
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
We use OSI DAC cables at work and they work great! thin and easy to manage.
I have a question, really appreciate if you could answer it, isn't fiber the only option for connections over 10GbE? Now that the 25 or even 100 GbE switch / NIC's are semi-affordable, what would you recommend using with these switches (DAC or AOC)?
Well 25Gbit DAC is avalable in SFP28 and 100Gbit in QSFP : www.fs.com/de-en/products/50483.html?attribute=10068&id=3763082
And.. it's faster.. Yes DAC has lower latency then fiber. Because in fiber needs to be converted to light. and the other end needs to be converted back, which is causing more latency.
Thank you for watching 🤗
Thank you!
I have an unusual case, I want to make a small board that connects 2 DAC passive cables at 100GB speed.
But I can't get the host system to work through my board and 2 DAC passive cables. The DAC cable is 50cm long. The lines on the board are 15 cm long; there is an overlap of RX TX signals on the board.
Tell me where to look? Who did something like this?
I have not yet seen 100Gbit SFP,, that is usually QSFP
What will happen if you sew into the eeprom QSFP28 DAC cable the information that it is 2 times longer (the cable is 1m - we sew up that it is 2m).
Will it work? Will there be communication between NIC ports?@@MyPlayHouse
Say I have a home with 5 PCs and 9 POE cameras, and a few other ethernet devices for smart home, like a Hue hub, SmartThings hub, etc. The cameras are on a different VLAN, and the PCs contain some VMs that are on their own VLAN, running little web servers. I am interested in the PCs having 10GB connections to a switch, but don't need more than 100mbps for the cameras. Should I use multiple switches for this? I don't know how I could get a managed switch that has PoE ports for the cameras but faster ports for the computers. The maximum distance between computer and switch is about 15 meters and I can easily run a new type of cable, be it DAC, Cat6, etc. Do you have any suggestions?
I think it will be cheaper for you to have a faster 10Gbit SW with just a few ports,, and another SW with 12-ish POE ports.
A 16-ish port 10Gbit SW becomes,, expensive.
Same split - longer runs, fiber (noting the correct quality is needed as using 1G fiber to run 10G links is temperamental) and for the short ones, DAC. I have some very short 10G DAC runs that I believe are 30cm or 25cm long, through to about 2.5m. The SPF(+) compatibility can be a royal pain in the @ss
I just buy fiber MM SR cable at OM3/4 which handles out to 100G and handles everything up to that, so no need for OM1/2.
Hi Wiebo Westerhof
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
Cool haircut!
Thanks! I think
I wonder if there's any difference in latency between DAC and Fiber because of the additional electric->optical->electric conversion. Anyone having any real info on that?
There is a little; For very-short-range (i.e.,
It must take a little time with the double conversion.
Theres a paper from arista, easily findable via Google
👍wow great I can't afford fiber yet
You will get there,, and DAC is cheaper :-)
👍thank you very much as soon as I can buy this copy of the fiber optic sizzles@@MyPlayHouse
Why not AOC?
Prices,, and fragile,, would be my down side to AOC,, But DAC cables stopes at about 7meter,, and from there AOC is the better choice.
Look at this guy... Imagine not having spare satellites to jump connections around. Couldn't be me :p On a more serious note... Yes please don't use fiber on the actual rack...please use DACs. Make life for everyone easier please. So you can get the QSFP in fiber. It is called MTP (multi termination push-on) fiber. Which uses MTO (multi termination optic). So one side is a MTP and 4 LC
Hi Waterflame
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
Is this preferred over 10GBASE-T? (when using 10 Gbps, obviously)
Yes,, this is higher quality.. somehow.
have any luck rebuilding raid 50 arrays. ? I have a equallogic ps4100 that needs love, can you help?
I have no experiences with that :-/ sorry.
i use only dac cable
Good choice!!
Why not just use cat6 for 10gbit?
You can,, but SFP,, just feels so much better.
I agree. Feeling good does matter
I agree that feeling good does indeed matter! 😁
Agreed
Hi merlingt1
Thank You very much! glad you liked the video :-)
Thank you for watching! :-)
I borrowed a new house from my grandparents, the story repeats
Nice to borrow a house :-)