I got a "cheap" QNap 8 port GbE 3 port sfp+ switch a few years ago for ~$200 and having it for 10gig between my workstation, Truenas server and proxmox server has been worth the price of switch, SFP+ cards and DAC's. Am now looking at upgrading to a switch with more ports (probably the TP-Link TL-SX3008F), there are quite a few more options out there now that are affordable that aren't power hungry and possibly loud rackmount switches than a few years ago and the Qnap (QSW-308S) is still available for ~$150 if you only need a few 10gig ports. That said direct connection works just fine. Nice bunch of upgrades, glad you are happy with it.
You are not the first to post how you've setup your system. I find it interesting how many ways we can move away from 1Gigabit networking! In the future, I may find a cheap 10GbE switch. Thank you for watching and commenting! :)
I actually installed TrueNAS Scale, which has a Virtualisation suite of apps. No money needed to change hands. Being able to address the hardware with pass-through was a breeze. IPMI, Graphics Card, 10GBe, all part of the "free" package. I get 7 Giga Bits per second (875 Mega Bytes per second) on my rig. Editing in Blackmagic Design Davinci Resolve is smooth as silk, with no low res proxies needed. Huge BM 4K-12 bit from BM camera.
i get 8.8Gbps with intel 10G cards going from an ubuntu machine with an i7-7700 to a pfsense router with an i5-7500 to a proxmox server with ex-2690v4's, all using X520's
That big case at the beginning is some old Thermaltake case I had laying around. To be honest, I don't actually remember it's model number but I think it's probably a Thermaltake Armor Revo.
Isn’t there SMB limitations to IO and speed? Would agree if mapped as Iscsi volume would be ideal for video editing workflow. Are you using disk pass thru?
This is actually a bare metal install of TrueNAS. Off camera, I tried an iSCSI connection, but I didn't really see much of performance increase. I mean... yes, it was a bit better, but not enough to use it exclusively. I think for a test and exclusively for transferring files to the NAS, I might configure FTP on it and see how that goes. Thank you for watching and commenting!
Yeah, might something worth checking out.... I am already testing various methods to transfer files etc. If I find something that make sense for my needs and actually has better and consistent performance, I'll post an update. Thank you for watching and commenting! :)
@@be-kind00 So....CIFS/SMB is pretty much universally supported now (even under Linux/macOS), so that actually isn't much of an issue nor an advantage anymore. If you're not used to dealing with (or mapping) users from Windows to users on FreeBSD/Linux, then SMB is actually easier to set up because SMB has its own user permissions management which is separate from the host OS. For NFS to really work well, it is best if your Windows users is the same as your host OS's users, which, usually, unless you have like Active Directory or LDAP running, is usually and generally NOT the case (because Windows, right?). As far as performance is concerned -- its a mixed bag. Phoronix has done testing with it and there isn't a clear winner.
Recently, I've been having a lot of issues with my TrueNAS server. I have a studio where we work on video editing, and currently our server is stuck at 100Mbps. I don't know what to do since we don't have an IT professional for this area. I wanted to build a server from scratch. What components do you recommend?
Check your cabling! Especially any patch cable(s) your recently added. Always use known good cables and I would stick to at least cat6 cables. For many reasons, I would be very hesitant to offer any hardware recommendations to be honest... However, here is a TruesNAS hardware guild: www.truenas.com/docs/core/gettingstarted/corehardwareguide/ Your environment and use case will dictate what hardware you need.
I wish I had a tower chassis to run TrueNAS on instead of a 1U server. Between Kevin and I, we have two 1U servers. I picked the quietest of the two. In our channels newest video, the performance of the 2.5GbE Interface I installed on my wife's iMac is actually quite impressive. With USB adapters, an inexpensive 2.5GbE switch and suitable spec network cables, anyone can have a decent (and relatively cheap) LAN. Thank you for watching and commenting!
You can find some "cheap" switches like Brocade ICX6610 with good amount of SFP+ ports for 250$ 300$ in US (which is quite cheap for that kind of switch), you also can find some datacenter switches like Arista ones also. With a switch you could do some LACP and have 20G with the 2 ports.
Yeah, I eventually buy a switch. I'll probably also eventually upgrade key machines on my LAN to 10GbE to warrant spending money on a switch. Hehe I did mess around with multichannel SMB a bit which utilizes both ports. It did increase file transfer performance... but not enough to stick with it. Thank you for watching and commenting!
It is nice! I was messing around transferring various file types and size amounts, I too hit 1.02GB/s bursts. It's just blazing speeds. Thank you for watching and commenting!
I had heard that usb/thunderbolt dongels were not reliable for networking on Unix derivatives for networking. Are you just using it for file transfers or in place of a NIC for all network activities?
Maybe I'm a bit late but just want to add some friendly comment: The cache size limited to half is no more, last version of TrueNAS corect that. For the sync as you said not very good, even with backup you have to resilver and it can be long... A better cheap solution is to add a slog drive to your pool (a NVME Intel optane SSD of a least 16GB (yes only 16GB...) on a PCIe card is working very well for me) and enable sync. It's a bit slower (maybe 10%) but much safer. If you can, add a second mirrored SSD for the slog.
No way! You are not late at all! I ended up moving on to Unraid and I bought a license for it, but I am now regretting it. The performance of it after the data moves from the ssd cache to the spinners makes it impossible for me to edit directly from the server. I am currently saving up for some new parts and HDDs to make another (and permanent) storage server. I think I'll move back to TN. Thank you for sharing your knowledge. This is exactly what I hoped would happen with this channel!
@@KMHardware The next version of TrueNAS (end of the year) should eliminate one of the main drawback of TrueNAS: you will be able to add a drive to a vdev to expand it's capacity. There's also a big change for the apps: use of Docker instead of Kubernetes, a good thing for more basic usage as Kubernetes is harder do manage. And for the speed with mechanical drives in TN you have to understand how ZFS really works because it's not very intuitive and there's a lot of bad informations about it. The YT video "Explaining ZFS LOG and L2ARC Cache: Do You Need One and How Do They Work?" from @LAWRENCESYSTEMS chanel is very informative (I watched it multiple time...)
Hey, thanks for the heads-up on TrueNAS updates! Adding drives to vdevs and Docker apps sound like they'll make life easier. And hey, big shoutout for the Lawrence Systems video recommendation!
i just switched to 10G with cheap SFP+ cards between my media server and VM server and its so quick now that with Jellyfin on the VM server, I can't tell - I mean, you wouldn't know the software accessing the drives is on a different machine since the bottleneck has changed...its absolutely fantastic, our world should be on 10G by now, or at LEAST fully 2.5G ffs
10 GbE is awesome! I only have a connection between my main computer and server... but I'll be completing other connections soon. Thank you for watching and commenting!
Iv just gone 10gbe between my main pc and my dell r720! I did go with a switch and its allot cheaper if you go with fiber with appropriate transceivers. Not only that but using fibre results im much lower power consumption on the switch compared to 10gb over cat 7 copper
I would rather have fiber too!! When I was looking around to purchase equipment, I found that the transceivers were cheap! It was the SFP NIC and SFP capable switch that was expensive. Also, for the iMac, USB to fiber converters are hundreds of dollars. So, I went 10GbE over copper.
@@KMHardware For the iMac, depending on how far of a run you would need -- you COULD always run Cat7 to the iMac via a USC 10 Gbps NIC, but then to the SFP+ switch, use a SFP+ to RJ45 transceiver, and that way, the rest of your systems that are physically closer and capable of using SFP+ cables - you can do that, and the others, will use Cat7 RJ45. (Also depends on whether you would be able to and/or are willing to spend more as Qnap makes an 12-port 10GbE switch, 8 of which are combo RJ45/SFP+ ports which means you can plug in whatever transmission medium you want (SFP+ or RJ45) on those 8 combo ports. It's more expensive compared to the cheaper 10 GbE switch options, but you CAN do that.)
I just realized that my other 2U server (the one that is stupid LOUD) actually has SFP+ ports! I could pick up two 10GbE transceivers and install an SFP+ add-in card for my editing rig, (as well, an optic patch cable of course) and I could be running 10GbE over fiber quite cheap! I can still run 2.5GbE to my wife's iMac over cat 7. My only concern right now is how to quite down the fans on my other 2U server. I understand it can be done... (have to use some SNMP tool to edit some particular data/file on the server) but I have never attempted to do it.
If you're already doing a 10 get you may as well do dual Port 10gb nics or more and Link aggregate them that way you can have a 20 gig or 40 gig connection. You'll probably cover this Also if you're not super worried about power loss or Colonel panics and you want to risk losing the last 5 Seconds of Rights I would definitely run it in asynchronous mode and that way the second it gets to Ram the server will tell the computer that it is written to disk. Edit: you did cover it but I'm going to keep it here just as a written version of basically what you said for other people. I'm all over the place today Also if you're planning on pushing 50 ft of Ethernet or anything close to that nic to nic is not going to work most likely just simply doesn't have the voltage
Thank you very much for the comments! To your final point. Very correct! I'm only running 25 feet of cat7 cable. Nic to nic is OK. I mean, a switch would be nice... but I just don't want to spend money I don't have on one. Lol
I got a "cheap" QNap 8 port GbE 3 port sfp+ switch a few years ago for ~$200 and having it for 10gig between my workstation, Truenas server and proxmox server has been worth the price of switch, SFP+ cards and DAC's. Am now looking at upgrading to a switch with more ports (probably the TP-Link TL-SX3008F), there are quite a few more options out there now that are affordable that aren't power hungry and possibly loud rackmount switches than a few years ago and the Qnap (QSW-308S) is still available for ~$150 if you only need a few 10gig ports. That said direct connection works just fine. Nice bunch of upgrades, glad you are happy with it.
You are not the first to post how you've setup your system. I find it interesting how many ways we can move away from 1Gigabit networking! In the future, I may find a cheap 10GbE switch. Thank you for watching and commenting! :)
Don't see Huawei servers very often. Love to see other home lab guys out there
Heck Yes! I think they are pretty good servers to be honest. Thanks for watching and commenting! More videos coming soon.
Cool. You talked about switches being expensive, the Serve The Home channel made a video about 10Gbps cheap switches, this may help you
Good find! I will be watching that video for sure. Thank you for watching and commenting!
True but sth also made a video recently about a "cheap" 100G switch and that was $2.000 but saying that i have bought my 10g sfp+ switch for ~250€
I actually installed TrueNAS Scale, which has a Virtualisation suite of apps. No money needed to change hands. Being able to address the hardware with pass-through was a breeze. IPMI, Graphics Card, 10GBe, all part of the "free" package. I get 7 Giga Bits per second (875 Mega Bytes per second) on my rig. Editing in Blackmagic Design Davinci Resolve is smooth as silk, with no low res proxies needed. Huge BM 4K-12 bit from BM camera.
That is awesome! What's your physical hardware? Just curious what other ppl are running on. Thank you for watching and commenting!
i get 8.8Gbps with intel 10G cards going from an ubuntu machine with an i7-7700 to a pfsense router with an i5-7500 to a proxmox server with ex-2690v4's, all using X520's
Which is that rack case/computer? Looks like some Inspur machine...
That big case at the beginning is some old Thermaltake case I had laying around. To be honest, I don't actually remember it's model number but I think it's probably a Thermaltake Armor Revo.
@@KMHardware I meant servers' rack case... Or what server is that
Oh! It's a Huawei 2288 v2
Another option besides sync off is throughput. It is a bit safer, and useful.
I am not familiar with this setting. Would you care to elaborate how its achieved? I would certainly like to test this. Thank you for the comment!
Isn’t there SMB limitations to IO and speed? Would agree if mapped as Iscsi volume would be ideal for video editing workflow. Are you using disk pass thru?
This is actually a bare metal install of TrueNAS. Off camera, I tried an iSCSI connection, but I didn't really see much of performance increase. I mean... yes, it was a bit better, but not enough to use it exclusively. I think for a test and exclusively for transferring files to the NAS, I might configure FTP on it and see how that goes.
Thank you for watching and commenting!
What about using NFS instead of SMB? I saw this done in another channel. @kmprojects2023
Yeah, might something worth checking out.... I am already testing various methods to transfer files etc. If I find something that make sense for my needs and actually has better and consistent performance, I'll post an update. Thank you for watching and commenting! :)
@@KMHardware in terms of NFS I am not just referring to file transfers but also normal file server operations for all client types.
@@be-kind00
So....CIFS/SMB is pretty much universally supported now (even under Linux/macOS), so that actually isn't much of an issue nor an advantage anymore.
If you're not used to dealing with (or mapping) users from Windows to users on FreeBSD/Linux, then SMB is actually easier to set up because SMB has its own user permissions management which is separate from the host OS.
For NFS to really work well, it is best if your Windows users is the same as your host OS's users, which, usually, unless you have like Active Directory or LDAP running, is usually and generally NOT the case (because Windows, right?).
As far as performance is concerned -- its a mixed bag.
Phoronix has done testing with it and there isn't a clear winner.
Recently, I've been having a lot of issues with my TrueNAS server. I have a studio where we work on video editing, and currently our server is stuck at 100Mbps. I don't know what to do since we don't have an IT professional for this area. I wanted to build a server from scratch. What components do you recommend?
Check your cabling! Especially any patch cable(s) your recently added.
Always use known good cables and I would stick to at least cat6 cables.
For many reasons, I would be very hesitant to offer any hardware recommendations to be honest...
However, here is a TruesNAS hardware guild: www.truenas.com/docs/core/gettingstarted/corehardwareguide/
Your environment and use case will dictate what hardware you need.
Cool project, I am currently working on a new old stock Xeon tower for smaller scale TrueNAS to use with FCP editing over 2.5G network adapters.
I wish I had a tower chassis to run TrueNAS on instead of a 1U server. Between Kevin and I, we have two 1U servers. I picked the quietest of the two. In our channels newest video, the performance of the 2.5GbE Interface I installed on my wife's iMac is actually quite impressive. With USB adapters, an inexpensive 2.5GbE switch and suitable spec network cables, anyone can have a decent (and relatively cheap) LAN.
Thank you for watching and commenting!
@@KMHardware yea i agree. 2.5 is a great alternative and much cheaper and less power hungry.
You can find some "cheap" switches like Brocade ICX6610 with good amount of SFP+ ports for 250$ 300$ in US (which is quite cheap for that kind of switch), you also can find some datacenter switches like Arista ones also. With a switch you could do some LACP and have 20G with the 2 ports.
Yeah, I eventually buy a switch. I'll probably also eventually upgrade key machines on my LAN to 10GbE to warrant spending money on a switch. Hehe
I did mess around with multichannel SMB a bit which utilizes both ports. It did increase file transfer performance... but not enough to stick with it.
Thank you for watching and commenting!
Yea it’s pretty nice. I can get bursts of 1GB/s over thunderbolt to sfp+ adapter to my TrueNAS NVMe machine.
It is nice! I was messing around transferring various file types and size amounts, I too hit 1.02GB/s bursts. It's just blazing speeds. Thank you for watching and commenting!
@@KMHardware lol i think i spend a half hour just transferring files back and forth just to watch the speed lol
Haha! Yup, I'm guilty of that too.
I had heard that usb/thunderbolt dongels were not reliable for networking on Unix derivatives for networking. Are you just using it for file transfers or in place of a NIC for all network activities?
@@be-kind00 my only computer is a MacBook Pro. It’s connected via the qnap thunderbolt to SFP+ when I’m at my desk for everything.
Maybe I'm a bit late but just want to add some friendly comment: The cache size limited to half is no more, last version of TrueNAS corect that. For the sync as you said not very good, even with backup you have to resilver and it can be long... A better cheap solution is to add a slog drive to your pool (a NVME Intel optane SSD of a least 16GB (yes only 16GB...) on a PCIe card is working very well for me) and enable sync. It's a bit slower (maybe 10%) but much safer. If you can, add a second mirrored SSD for the slog.
No way! You are not late at all! I ended up moving on to Unraid and I bought a license for it, but I am now regretting it. The performance of it after the data moves from the ssd cache to the spinners makes it impossible for me to edit directly from the server. I am currently saving up for some new parts and HDDs to make another (and permanent) storage server. I think I'll move back to TN. Thank you for sharing your knowledge. This is exactly what I hoped would happen with this channel!
@@KMHardware The next version of TrueNAS (end of the year) should eliminate one of the main drawback of TrueNAS: you will be able to add a drive to a vdev to expand it's capacity. There's also a big change for the apps: use of Docker instead of Kubernetes, a good thing for more basic usage as Kubernetes is harder do manage.
And for the speed with mechanical drives in TN you have to understand how ZFS really works because it's not very intuitive and there's a lot of bad informations about it. The YT video "Explaining ZFS LOG and L2ARC Cache: Do You Need One and How Do They Work?" from @LAWRENCESYSTEMS chanel is very informative (I watched it multiple time...)
Hey, thanks for the heads-up on TrueNAS updates! Adding drives to vdevs and Docker apps sound like they'll make life easier. And hey, big shoutout for the Lawrence Systems video recommendation!
i just switched to 10G with cheap SFP+ cards between my media server and VM server and its so quick now that with Jellyfin on the VM server, I can't tell - I mean, you wouldn't know the software accessing the drives is on a different machine since the bottleneck has changed...its absolutely fantastic, our world should be on 10G by now, or at LEAST fully 2.5G ffs
10 GbE is awesome! I only have a connection between my main computer and server... but I'll be completing other connections soon. Thank you for watching and commenting!
Iv just gone 10gbe between my main pc and my dell r720! I did go with a switch and its allot cheaper if you go with fiber with appropriate transceivers. Not only that but using fibre results im much lower power consumption on the switch compared to 10gb over cat 7 copper
I would rather have fiber too!! When I was looking around to purchase equipment, I found that the transceivers were cheap! It was the SFP NIC and SFP capable switch that was expensive. Also, for the iMac, USB to fiber converters are hundreds of dollars. So, I went 10GbE over copper.
@@KMHardware
For the iMac, depending on how far of a run you would need -- you COULD always run Cat7 to the iMac via a USC 10 Gbps NIC, but then to the SFP+ switch, use a SFP+ to RJ45 transceiver, and that way, the rest of your systems that are physically closer and capable of using SFP+ cables - you can do that, and the others, will use Cat7 RJ45.
(Also depends on whether you would be able to and/or are willing to spend more as Qnap makes an 12-port 10GbE switch, 8 of which are combo RJ45/SFP+ ports which means you can plug in whatever transmission medium you want (SFP+ or RJ45) on those 8 combo ports. It's more expensive compared to the cheaper 10 GbE switch options, but you CAN do that.)
I just realized that my other 2U server (the one that is stupid LOUD) actually has SFP+ ports! I could pick up two 10GbE transceivers and install an SFP+ add-in card for my editing rig, (as well, an optic patch cable of course) and I could be running 10GbE over fiber quite cheap! I can still run 2.5GbE to my wife's iMac over cat 7.
My only concern right now is how to quite down the fans on my other 2U server. I understand it can be done... (have to use some SNMP tool to edit some particular data/file on the server) but I have never attempted to do it.
@@KMHardware I haven't gotten round to quieting down my fans ether. It's been on my to do list for ages lol
I need to research this. Probably a very niche topic.. but maybe video worthy.
If you're already doing a 10 get you may as well do dual Port 10gb nics or more and Link aggregate them that way you can have a 20 gig or 40 gig connection. You'll probably cover this
Also if you're not super worried about power loss or Colonel panics and you want to risk losing the last 5 Seconds of Rights I would definitely run it in asynchronous mode and that way the second it gets to Ram the server will tell the computer that it is written to disk.
Edit: you did cover it but I'm going to keep it here just as a written version of basically what you said for other people. I'm all over the place today
Also if you're planning on pushing 50 ft of Ethernet or anything close to that nic to nic is not going to work most likely just simply doesn't have the voltage
Thank you very much for the comments! To your final point. Very correct! I'm only running 25 feet of cat7 cable. Nic to nic is OK. I mean, a switch would be nice... but I just don't want to spend money I don't have on one. Lol
amateur
Good luck with your channel! Thank you for watching. You now have 3 subs! (I subscribed from my personal account)