@Benjamin O'Neal I'd recommend an Optane drive, or enterprise flash of some kind. But yes, you'll need a quick, write-endurant SSD for DeDup. Also, I do have a PO! Craft Computing 1567 Edgewater St. NW, #51 Salem, OR 97304
My server sits in the media unit, it hosts pihole, file server, plex and gives output to the TV via a VM. My wife just thinks it's another pc ! Total wife approval factor.
Do you own a lot of home movies that you would like to save? Maybe some videos of your grandparents gone? A server may be an excellent use case for that. (One of the many I ended up investing into a used enterprise server) I'm also a DJ in an underground scene where songs are called plates, which, to make an explanation simple, they are unreleased tunes that are played with a limited time span, and sometimes get brought back for a throwback tune in a mix. It is VITAL to me to have a way to keep those songs saved and backed up so i have my favorite, (and also very unique and exclusive) setlist. Sorry for the really long rant, but those are my use cases for having a server at home, just wanted to give you some extra insight!
I have 4 ESXI virtual servers, and 1 server operating as the NAS, with 24 60GB 15k drives, and a external drive chassis with another 25 600 GB drives. The NAS server provides storage to the ESXI servers (they all boot of iscsi luns, so they have no physical drives in them), networking is provided by 2 Brocade/Rukus 24 port POE 1GB, 10 10Gb and 4 40Gb ports. All of my services, firewall, active directory, media server, virtual desktops, etc run as virtual machines. Using vlans, and VMwares features, I can move my firewall, AD, backup server, etc between the virtual machines without ever having any downtime. For me, as an IT security professional, it makes sense, and is fun to play with, and can setup environments where malicious traffic cannot escape.. Downside.. is around a $300 electricity bill every month..
Years ago people at work bought small HP home servers, as they were cheaper than buying a NAS enclosure. I never bothered, as I have all my rust onboard, I was planning to upgrade, but Covid pit paid to that. Ah well.
For people with multiple PC's that don't like the limitations, you can install a lancache server which proxies the steam connection. When you download a game it caches it on the server. Sure, your desktop PC won't save any space, but uninstalling and re-installing games will happen in a flash since all of the files needed for it are already on your own lan.
"In a flash", for example if you play a 120GB game, its size is equal to 960Gb, a common LAN speed is 1Gbps, so it takes you 960s =16min to download game, and more to install it. That's super annoying having to wait almost 30 min to play your favorite game
@@bacphan7582 120GB games can be counted on one hand, and all of them are competitive so they are already latency focused, i.e. completely out of scope of this option
@@bacphan7582 If its your favourite game though.. You wouldn't be constantly deleting and reinstalling it. You'd keep it installed. :P I use lancache.. Yeah its not perfect but if you have multiple gaming pc's in your house then its pretty useful, especially for win updates too. For example: If all of us buy a new game and need to download it. Technically only one of us will have to download it which takes roughly twice as long or more than the other systems downloading it straight from lancache. (Also good for people who are limited on bandwidth but not really relevant to this) So it still cuts down download time and makes a big difference. I'd prefer this rather than having to download a 100GB game on 5+ computers. Yeah... This video would probably be more useful in some situations but due to also having a few gaming laptops in the house. I'd rather not expose my server publicly just because my games are hosted on my server. Secondly, if I go somewhere where the connection is not great, trying to play it from your server on a slow connection wouldn't be the best idea but if you had them locally installed, then you might just about get away with it. Both solutions have their own benefits. I'd definitely recommend this video if you don't game anywhere other than your home but if you do, I'd probably recommend lancache. I'd still recommend lancache for other benefits even if you don't use it for caching your game downloads.
From experience I can also mention the following: - BattleEye games such as Warzone will work from the iSCSI drive but won't work from network drives, huge bonus to iSCSI - The SMB share can sometimes be faster because the protocol is "simpler" (compared to full-blown NTFS) - For sharing the same drive across multiple computers, the solution goes like: Create a BTRFS/ZFS Volume snapshot (with CoW) based off the actual steam library. Have each client use a snapshot. This has some drawbacks indeed but with a list of pre-defined games it works wonders. - Another alternative to iSCSI is using Dokany. Run the mirror sample to the UNC path and you'll have an almost-right alternative to iSCSI. Some Anti-Cheat solutions will still complain about this, though
i played destiny 2 over smb 2-3 years ago, so maybe not amm battleeye games. or did you meant activision's RICOCHET, bc cod uses their own, and not battleeye
When using the same drive across multiple computers with iSCSI with replication/snapshots, would that drive be read only or it would be two way sync? Trying to find a solution for video editing on 2 computers directly from NAS with iSCSI and dedup
@@RodrigoeBeta I've found some other solutions all these years. iSCSI generally works with a whole "device", you usually get corruption if you attempt to mount a device in multiple locations (assuming the server even lets you do this). The solution I've found is a ZFS pool (normal pool, not virtual block drives) and NFS 4.2 with "nconnect=16,async" and 4K block size. In my scenario, I give clients an overlayfs mounted path, such that their modifications are personal, while still sharing a common base. Dedup in this case works on a file level. In my case, the clients are all Linux, but I assume this can work correctly with SMB as well. It depends on whether you want your clients to see each other's modifications, but regardless, I've moved away from iSCSI as it's use cases are usually quite specific and not suitable for scenarios like shared access to data
we need a few sequels to this for sure, focussing on at least two different scenarios. 1. One steam account that accesses the network attached steam library from multiple pc's 2. Several steam accounts accessing a single game from different pc's simultaneously (this will certainly need ssd's as storage medium server-side) Great video though, keep it up!
*Connect to the iSCSI on each other computer and you should just be able to search for the game files in steam, verify, and play them. As f* or muti accounts it (in therory) would work the same process jsut with a different account. I can get a buddy of mine to test his account from my seocnd comptuer next time he is at my house. I will say it still could fail since scuzzywasn't really made for 2 connectsions. But since they are really only reading it may work fine.
Okay I ran a test using famly sharing with an old steam accout we call that PC2. PC1 is my main rig with my main steam profile that I installed all the games with and orignally moved the to the scuzzy with. I was able to start up a game by borrowing using famly share on PC2 connected to the same scuzzy. I was able to play it for a few min on its own then started the same game on PC1 from the the scuzzy and it ran fine on both for the 5 min until steam kicked me out because once the owner starts the game while family sharing they kick you after that time.
@@teetertech FOr anyone who decides to try this, be aware that the drive is showing up as local, so things like "last access" times and permissions (user/owner) information are going to be edited by the system even if you are 'only reading'. In a lot of cases that won't cause issues, but if you have a game with a large data file being read from multiple machines you *will* eventually run into file corruption from conflicting file updates. Again, it's just a steam library, so it might not cause you a ton of headaches to have to reinstall things or verify files on a regular basis, but it's still a problem to be aware of.
@@PoignantPirate yeah I had a game update and it disappeared on the other. So it definitly is a big no no but it was cool to see it work but long term it will just ruin things.
@@PoignantPirate What if you have two different pc's, one playing a game at a time? (ex. my SO works days, I work nights, we play the same games between shifts). As long as one of us powers down our pc before the other turns theirs on would that prevent the file corruption? Also, so the way to have multple, (2 in my case) pc's connect to the same Steam Library would be to install Steam on the server, connect both pc's by the same iSCSI LUN, just one would be kicked off while another uses it?
This is really cool. I do remember putting a Steam game on my NAS to see if it would work and not being too surprised when it didn't. Had no idea it was as simple as how Windows reads network versus local disks. Thanks for making this super simple. I might have to try this purely for fun, or perhaps as a different storage config in the future.
Most of my games work fine but there are certainly exceptions; a lot of MSFT games for example seem to take offense, Halo and Forza in particular seem to not much like being on the network. Sometimes it works, sometimes it just doesn’t at all.
@@1armbiker Ahh. ok. thanks for the warning. I've been using Steam and Epic Games stores. Anything M$FT I'll keep local (or buy through the other stores).
This by far even today is the best video for getting this setup smoothly. I am using Cobia and even with the few changes I followed this guide today and worked like a charm. Thank you!
A deduplication video would be awesome. Just showing the space savings by having 2 similar iSCSI steam libraries on the same ZFS pool sounds like a great demonstration
Yeah true, but ZFS deduplication cannot be recommended!!! It only sort of works if you a don't reboot and can store the deduptable in RAM or b you have a really fast persistent L2Arc like an Optane drive. Tbh it's not worth the drawbacks right now. The only dedup that works alright without those drawbacks is vdo. Also deduplication isnt really required for the space saving in this instance as you can just install all the games and then just snapshot that drive and make another share of off that
Modzilla how would you go about that? Install games, create snapshot of Steam dataset for each user/computer, have each user’s iSCSI point to their respective snapshot? What happens for game updates?
@@ReturnJJ yeah that's where it falls apart a little. You'd need to a another clone/snapshot. But I think that would be a fair trade since all the save states are stored somewhere else and this way you don't need to download updates more than once.
@@LampJustin I just had an idea, what if the Steam Library has its games installed and updated using a VM but there is no actual gaming down on there, and then have snapshots for each user and a snapshot for the Steam Library VM so that the VM can commit game updates and installs? I don’t know why I randomly thought of that while I was bored, but there’s that.
This video damn straight may have solved my game storage dilemma! Been using my Razer Blade for gaming since the main pc gpu died and the RB only had the 500gb nvme and 1TB HD. I would use an external ssd to transfer games. I’m getting ready to switch my home network to 10gbe at the new house! I can get ssd loading speed on the laptop through a TB3 dock and access my whole library this way now!
I... need to think of what I need to say here. You are doing everything that my mind has been dreaming of for the last decade. Im 23 and still fairly young and inexperienced in the job market so money isnt easy to come by as far as jobs go. Now that i finally have the gaming PC I want and a server, with your channel, I will have the godliest tech setup before I even own a home XD But seriously. Every video of yours I watch has answered some sort of question, or put to the test some sort of theory ive wanted to try, but up until now, never had the computing horsepower or experience to try. Your videos are AWESOME and very helpful, plus its nice to enjoy a beer with someone over the internet! So Cheers to the future bud! Glad to be a subscriber, and I cant wait for more gaming, and server content.
I have been running like this for several years now and it has worked flawlessly. One added benefit is that, even if you run on spinning rust, TrueNAS will cache the data in memory (or L2arc). So after a couple of days you will probably not even hit the disks. Windows does the same off course, but presumably you run some other thing on your Windows host (or reboot/turn off) and then the cache is lost.
Having been a nerd all my life and worked in IT. and telecoms I genuinely learnt something tonight when watching your video! I now have iscsi working on my truenas core server to my pc. been using freenas/truenas for about 9 years and never knew what it was! Thanks :) subscribed.
I'm finally at the point with my Steam library (where I am having multiple copies of the same games installed on multiple systems) where this video actually came in REALLY handy. Thank you!!!
It's cool to finally see someone else do this. I ran my game library off my NAS for about a year and half. Eventually when NVMe got cheaper and I needed my NAS storage fore more projects I moved my games back to local drives. Here are some things of note I ran into: -Windows ISCI can be temperamental. Had to set the service to wait before making a connection, since my network drivers weren't loaded fast enough on boot up. -Some newer game with anti-cheat detect something isn't quite normal. Either the game won't launch or runs awful, since the anti-cheat does intrusive scans in the background. -Unlike SMB, ISCI doesn't do any integrity checks when transferring. File corruption is possible but fortunately steam has its verification tool to fix this. -DON'T USE EPICS LAUNCHER! The final nail for my ISCI partition was when I installed UE4 to do some development. It filled the drive with cache files and failed to clean up after itself.
Hi Jeff!!! Love your content. I've used many of your videos for my homelab, especially with the chenbro 1u servers. I built a truenas server with the chenbro 12 bay server and was able to achieve read writes in the same ballpark that you achieved in your video!!! Thank you so much for your time and your expertise. can't wait to see what you come up with next!
I love you. I discovered your channel about 2 weeks ago because of looking for a decent video how to run a 1-GPU-2-Gamers-System. You're definitely my guy. This content is gold 4 me.
I've been subscribed for months and just built my first home NAS out of an old tower I had lying around. Then I find out you already made the perfect tutorial for EXACTLY what I wanted to do! I never uninstall games for fear of losing something, but I can only fit 2 hdds in my itx gaming computer. Now I've got more than enough space. Would be interested in seeing if an existing SCSI pool can be expanded with extra drives in future.
For the iSCSI share, you can use RDMA enabled hardware (10gigabit sfp+ Infiniband mellanox connectx2 in IT mode) with a modified hosts file that points directly to the iSCSI share (or truenas server) so that both devices can communicate without any overhead. NOTE: both cards must be in IT mode if they are the same and also it's a good idea to add the IP address of the client that's requesting the connection of the share.
For years iSCSI has lurked provocatively in the background as "what's next" yet the first time I see implementation details, Jeff is installing his Steam Library to it Fantastic!
thanks for the great video, at about @13:30 you discuss the Z drive and the E drive, however you only show the Z drive read/write speeds, would be interested to compare that to the E drive you set up
5:37 -- iSCSI downside. Great video 👍 I slammed into this "2 initiators, 1 target" conundrum this week. Came here to get another viewpoint confirming this challenge. I am not going to do filelocking or a cluster-aware set up.
Just found this video. Overall, this is a nice video and is useful. Couple of things: iSCSI LUNs can be accessed from multiple servers simultaneously, and if you move an iSCSI LUN from one server to another that can mount the FS then the permissions will be maintained; the SIDs may not be resolved by the new server but it will, in fact, see the SIDs in the ACLs. Also, you created an E: drive but then benchmarked a Z: drive. And SMB is not commonly referred to as SaMBa; SaMBa is a free software implementation of the SMB protocol. Thanks.
Wow! You're right! I'm still new to iSCSI, so I tried connecting a Windows VM to the same iSCSI LUN volume I made for my laptop...and it worked right away! Awesome! For those that have an Asustor NAS, this works! It seems we can't share Btrfs snapshots, but this LUN alternative seems way better anyway.
Hey Craft, great video! I have noticed the UPS in your rack is showing 94% continuous load which is bit high. I'd plan on splitting it up with another UPS to give your servers longer run time and less chance of UPS failure. It's best to keep it under 75% load.
It's working ! I was using SMB but after watching this video I configured iSCSI Service in my TrueNAS server and in the iSCSI Initiator app which are connected directly through PC to PC hence only me has access to it.
Great stuff! Immediately went and did the same using my Synology NAS spare capacity -- surprisingly crystaldiskmark was able to write/read at 118MB/s over a simple gigabit network! I will personally only use this new drive for very large install, for games that require one very large loading at the start (e.g. flight sim), vs. those who have frequent smaller loading (e.g. MMos)
Im so glad im not the only person doing this for their home network. I havent really run into many issues doing this via networked attached storage pointing to an unraid share, as i need multiple clients to access the same Library, but excited to see your video on de duplication though as I hadn't considered it as an option.
So the benchmarking at 12:48 appears to be of the network drive mapping (Z:) rather than the ISCSI drive (E:). Can we get the benchmark of the ISCSI drive too?
And aside from my other comment.. You can take it one step further, and take advantage of all the features of ZFS and Truenas. Boot from an iSCSI lun, you gain all the advantages of ZFS and Truenas as you noted, but that extends to the boot drive as well, and you can use replication and cloud sync to backup your entire environment, not just the Steam library, and you gain significant performance bumps. Add to that the use of cheap Infiniband cards (can be had for less than $50) and Infiniband switches (found a 36 port one for $100), and you have a 40Gb or 56Gb infrastructure depending on which version you get).
Excellent and informative video. I used the steps outlined in this video to accomplish the same thing using a Synology NAS. Interestingly, I've found I'm getting much better and consistent read/writes to the iSCSI disk versus a mapped drive via SMB. I was always getting wonky performance via SMB with file transfers, only sometimes maxing out my 1Gbps network connection and more often transfers were in the 150-250Mbps range. Tweaking the NIC parameters, drivers, etc...nothing helped so I chalked it up to SMB. Making the same transfers via an iSCSI mounted disk on the same NAS is almost always maxing out my network connection. I'm a happy camper with this solution.
Yes, the difference with smaller files is immense. Even a 10gbit connection can feel totally wonky over SMB when you transfer f.e. a games folder. with iscsi it just chucks along at whatever the NIC can transfer.
If someone would be worried about the speed over the network connection, once you've setup more then one folder for your steam library, you can always move the game around between these. You can do that by doing a right click on the (installed!) game in your library -> properties -> local files -> move install folder. Steam will show you a list of all available folders and you can easily move the game over from a network drive, to your local drive. This way, you can store all your games externally and move them just on your PC for playing. Of course it takes some time to move the files over the network but its probably still faster than installing the game over the internet and may or may not result in better performance for the game (or at least loading times).
@@nevoyu It probably really is but I haven't tried it on my own yet to say much about it. Just wanted to point out a similar but different solution since not everyone has the luxury of running a wired local connection, an actual NAS or even broadband internet connection. For quite some time, I also just used a raspi 3 with an external USB drive for a NAS and while this is fine for some tinkering and basic setups, you'll run into some bottlenecks.
@Tim v H Generally a NAS is used for multiple user or device access. They generally also have more redundant storage of the data for Better long term data integrity. And less risk of loss. Depending on the Network speeds and drives used a NAS can be as fast or faster than local storage. If you are a one machine gamer, then yeah you probably don't have a use for a NAS. But many people have multiple machines multiple users that all need to store data they don't want to lose,, so a NAS makes sense
You're videos are so good I almost had this memorized after one watch. To bad I'm dumb and forgot it needs to be a zvol not a dataset with quotas (face palm) Great job man!
At 8:31 the option for Sharing platform from what characteristics it is depend on? The physical characteristics of the disks true nas is based of (like 512e or 4kn) ? The disk's characteristics the other platform has? (assuming it is a proxmox based on 512 logical / physical drives). The OS the platform relies on? (I mean if it is capable of using 4k as the minimum amount of blocks? - which platform wouldn t) Thank you
That 4TB/5TB ~$60-70 argument used to be something I considered until power+packaging became my main concern over a non-hotswap rack that goes down every year for cleaning, every 2nd year for upgrade and every 5th year for fan/PSU replacement. Last week I picked up a 16TB Toshiba MG08 for $320. That's roughly ~$20/TB but the 9 platter packaging is so dense that I might pick up a few to make my first RAID volume and use my 60GB refurb SSD for failover journaling. I'm still getting everything settled to make this new disk my main video storage replacement and migrate my other disks to move deploy volumes for apps, games, backups and non-Win platforms. So considering power and disk health, the only thing I really worry about is the moment my disks reach 100K hours. What is your main health concern with your equipment and how do you retire your disks after they reach a certain age?
Thank you for mentioning the facts about ISCSI, I've seen so many misleading videos about this solution. I'm trying to build a network setup with some SSDs in raid 0 connected to PCs all over the house equipped with 5 to 10 gigabit network cards/adapters. When I get a game working right, the latency is not much worse than a local SSD. The problem is getting third party launchers working with it. Battle Net (blizzard games) will not work anymore, doesn't support NAS setups, GOG can be solved with some writing in command prompt and a registry edit, Origin can sorta be solved but only if you're playing games right from Origin. Some anti-cheat games, specifically ones with battle eye (rainbow six siege) and Origin games bought and launched through Steam but still using the Origin DRM WILL NOT WORK. I think it's a matter of inputing some kind of permission in the command prompt and having it start with extra permissions but I for the life of me cannot get the syntax right. I'm using SMB right now. Anyone have any ideas?
I mount a vhd on startup that is stored to my nas. Blizzard works just fine this way and while I haven't tried Origin I have used Ea Desktop and that works fine in my limited experience. I oy play 2 games through EA
You can also do this with portable applications where you can set a NAS as your pool resource and any computer on the network (with access) can open the executables. I do this with Office, some of adobe CC, and even roms for my Rpi's. But if your bandwidth is good enough you can even switch it to a SAN or iScsi
I did roughly the same thing however I used a virtual hard drive and then mounted it locally from the network. It seemed to work just fine but every once in awhile it would hiccup and I had to tell steam where the game library was. I will be doing this
100% answered the question I had. I mounted my iscsi target to VM thinking I could share the lun between my main computer and the VM. this explains why that failed.
Good point about spinning disk performance compared to SSDs. It can take some config tweaking, but old hardware can still be remarkably capable. I've been able to push arrays of ~20 2.5" 15k SAS drives to about 500 MB/s write. For the speed test, the iSCSI-backed drive has a filesystem mounted on E:, but the volume being tested was the SMB network share on Z:? It's also nice to see that the iSCSI sharing settings of TruNAS are more beginner-friendly now! From my days as an ESXi admin, I remember iSCSI being very fussy about latency and heavy packet loss or interruption. Have you noticed any issues like that, or have you put measures in place (e.g. QoS/CoS) to mitigate them? Man do I wish I had Gen9 HPE kit in my lab 😅 EDIT: reordered content.
How and where do you do the tweaking for your network speeds? That's my next task for my TrueNAS lol. I'd like to be able to utilize the speeds I'm paying for.
The tests at 12:45 are done to your Z: drive, which you said was a Network (and I'm assuming SMB) share. What kind of performance do you get in that same test for the E: drive, which is the iSCSI drive? Without the comparison between SMB speeds and iSCSI speeds for random access reads, I don't really understand why you would care to use one or the other.
Going to have to try this out at some point :) I do have a steam library set up on a standard smb share on my NAS at the moment, but i use it to archive games i'm not playing recently instead of actually running games from it. (use steam's move folder feature to move games from one steam library to another, so i just move a game from the NAS library to one on my local SSDs when i want to play)
For anyone it may help, if you use capital letters for the target name, the wizard won't complete. This is in TrueNAS-13.0-U3.1. Great tutorial and videos.
Thanks for this video!! Also wanted to say I love your content but somehow missed this video until now.. setup is almost identical in TrueNAS Scale for those wondering..
5:40 so can't have two users to play the same game on the same share storage device? (still learning) I was thinking just using a NAS as a storage device, so instead downloading the game again, I can just simply keep it stored in the NAS so either me or the other user (my dad) to play the same game
If you want to build a true nas server and the drives you have already contain data you can't loose, how do you solve this issue. My issue is my emby server ( and I use the term server loosely) is simply a spare windows computer with large drives full of media. If I pull out those drives, wipe the boot drive and build a truenas rig, how do you a build a storage pool without loosing the media I have?
A more simple approach is to create a veracrypt disk on your server. When you mount one on your computer it works like a local disk. I just checked my crystal disk mark speeds and they're similar to these results so the performance is comparable.
In a follow-up video could you cover iscsi snapshots? Specifically, if there is some way to have scheduled snapshots from a golden volume. The idea is to have one machine that keeps a golden volume up-to-date and once a day the client volumes get replaced with another snapshot of the reference. This might not be perfect though since I think the steam library folder may also house some saves, and those would be overwritten. I'm just not sure. I think maybe mapped samba for most games is best, then iscsi for the games that complain, then local install + samba link saves for the games that complain and keep the saves in the steam library.
@ 1st sponsored segment: Most phones that still include chargers include ones that don't take hours to charge your phone. It tends to take an hour, max.
14:00 so if you download GTA there you have what two options, play it through NAS and wait for ages in loading screens, or transfer the files over the NAS HDDs to your local SSD? Actually a lot of people with slower internet speed might benefit from that anyway, as transfer through one drive might be faster than their internet speed. Cos like, it's fine to have a storage of games saved, it was even more so the factor in the past with offline games. But with high enough internet speed it doesnt really matter anymore, in cities you can easily get 1-2Gbps. But yeah for games that dont actually require high read speeds, being just on your HDD might save you time and storage. I wonder tho if you make raid 0 or raid 5 if the speeds increase and by how much. I'm also interested if you can put SATA SSD into the array instead of HDD, i heard you can use it as a cache, but i dont know how that really translates into the whole play, if anyone would explain
I am running this Setup for some years now. Truenas has a Lot of Features that can Help Speed Up your storage while giving you more space to expand. I got some icydock hotswapable 2.5" trays which Serve as my Main storage. Only Problem will be Direct storage which only works with nvme. I Hope truenas Soon has nvme over Fiber.
Hi! I have a desktop back at home, but that's 6000 miles away. Do you think that I'll be able to use my 8tb HDD on my desktop as a library NAS and remote it all the way here from my laptop? Tysm.
@@theangelofspace155 It would not be worth it. It would be DESPERATELY SLOW and almost unusable. So many problems with that (plus latency issues) -- not worth it.
I have my steam library on a share on my server running unraid, on an unassigned 2TB SSD, been doing this for about 3 years works perfectly I can play games from whichever gaming machine in my house, cool video though!
I run my Plex Server on a separate system. It connects via SMB and the other backup over NFS. I just edited a nice fstab file and point plex at the host directory. iSCSI doesn't allow sharing. This allows my main machine to transcode those nice BluRay and UHD disks and copy them to the plex server remotely.
It's difficult when everybody is switching to 10 gig. I switched and started getting about 75% more throughput on a plain old HDD. Now I get about 380MB transfers on all HDD's, more drives will just add. Ore capacity and more speed.
I have a very basic pool on truenas, 12tb (4x4tb) ZFS and it works great as a network drive holding my steam librairy. I opted out of iSCSI for the very specific dealbreaker you mentioned as I would have a hard time accessing my saves from different PCs (I game on 2 different rigs depending on time of the day as I work from home too!)
If you use Steam, you should be getting the appropriate save files through the steam cloud. With the de-duplication, you can just set up two iSCSI drives and fill them with the same games, and the NAS will take care of de-wasting the space.
This is especially a good idea for Mini ATX/ITX builds with the newer 2.5Gb ethernet standard; you can easily run your OS from an M.2 disk and not have to worry about mass storage :)
Multi-mount volumes are possible using Windows Server, if you want to have each PC running server with Cluster Shared Volumes. These show up as local mounts, but oh my CSVs are a real PITA. SAN's like Nimble support "multi-initiator access", which is a pre-req for using the SAN with CSVs. Such volumes can be mounted by multiple hosts, but again, we're talking servers, not desktops. The same setting is also required for MPIO, and works normally in VMware with multi-host shared datastores. Anyway, I use Steam Library sharing using mapped drives. No UNC paths, only drive letters. It works A-okay. Now the case for iSCSI does matter for companies like Blizzard who detect that libraries are running on shares, even if they are mapped drives. They disallow this because Blizzard has fallen to the dark side of late and is strongly anti-privacy and anti-choice.
Some iSCSI best practices: * You should use a secondary NIC dedicated for storage traffic if you can. 10Gb is definitely beneficial, even if you have a direct connection to TrueNAS * 1Gb connections really benefit from Jumbo Frames (9000+ MTU vs 1500). If you have pretty much any managed switch it will do this , but everything in the chain (Nic on windows, switch ports, TrueNAS) has to have it set. 10Gb there’s not much of a difference. * MPIO is supported on most Linux and Windows Server OS’s. Windows Desktop does not unfortunately support multi path, but you can setup multi path in TrueNAS to effectively double bandwidth if you have two NICs and the OS supports it. You see this a lot with Hypervisors. * Do not use Deduplication if you care about performance and have low Ram. It’s a bad time and honestly not worth it unless the workload dictates it.
I never had much issues running my steam library or other games on my SMB share mapped as a local drive. Can I expect significant speed increases compared to a normal SMB share even on just a gigabit network card?
At 12:08 not sure if intentional but you have a glitch and then your audio is out of sync. It's a nice touch if you planned on it to show potential issues with the iSCSI drive over the network however.
This is really cool. I was suffering running launchbox from a network drive. It would freak out every time. I ended up buying a ssd to add the retro collection to it (I have 2 firecudas nvme for main system in raid and now 2 cheap wd ssd in raid on my pc) I think I will move the ssd to truenas and do this. I have a 10gbe link but no drives fast enough to test it hahahaha.
OK i can apparently share iSCSI even with a pocket router i got 10 years ago for $20 with a USB drive hanging off it and that has been running OpenWRT since about a week after i got it. That's nice. Not sure i'm going to do this, because performance will be naturally limited and single client limit, but i appreciate knowing that i can.
I haven't tested this myself but if you run virtual machines and connect the iSCSI target to the hypervisor you might be able to use the disk on multiple VMs in read-only mode. This will show you a warning though as you are sharing resources between VMs which are not meant to be shared. However, I would view it like mounting an ISO file of which I used some to install multiple systems simultaneously, before.
this is awesome....this method could mean my dropbox sync could be allocated to this drive and then backed up as required and mean it is on my server and not my desktop....how good is that!....could come in handy for all kinds of things to be honest.....thanks for this jeff 👍👍
I had started out using a mounted virtual disc that was stored on my Unraid NAS, and while it did work fine and all applications and games just saw it as another disc drive, there was one glaring problem I couldn't get over. No matter what I did, no matter how many guides I followed, etc, I could NOT get the mounted virtual drive to disconnect cleanly on windows shutdown and sleep. Because it wouldn't cleanly disconnect, every time I restarted my system the drive would fail to reconnect because my Unraid NAS still showed it as connected even though it wasn't. So I gave up and moved everything to a mapped network drive. Some applications (like Origin games launcher) didn't want to play nice at first, but there seems to be easy workarounds for every troublesome application I've come across so far. So for me, mapped network shares are turning out to be the better option.
That's not an iSCSI limitation, but a Windows file system limitation. I've absolutely used clustered filesystems with multiple clients per target in the enterprise server space (Oracle FS, etc).
A quick question, would this affect the loading time of the games I installed? I assume it will load up slower due to it being a network drive running on hard disk format.
Overall, a good video. I would like to see a better solution than using deduplication for having multiple copies of the Steam Library though because whilst your server has 64 GB of RAM, so ZFS dedup is not a problem for you - if you have a server without that much RAM, ZFS dedup, as a function of your vdev space, can become a problem for said ZFS dedup to work properly. It would be nice if there is a solution that combines the pros of both without the RAM requirement that ZFS dedup has. Thanks.
My Steam Library stays on SMB share and it seems to be happy. It is true that if I try to install applications on SMB share, some programs prone to be angry, but Steam Library has been working fine for me so far.
It recycling places can shift off discs I just bought 20tb for around the same as a new 2tb SATA.. Depending on your controller/set up you read the smart data, and don't buy too old one or too many cycles. Hours are fine and most are sold at 20k mark or after 2-3 years. Any drive new or used can have faults or fail, so old or new doesn't really make much difference. But...... You have to factor in buying a sas card (HBA) and cables or backplane to make it work.
The important thing with used drives is not to trust them. You should always back up your data, but especially so with less trustworthy drives. Most people using used enterprise drives use them in a RAID, so that when they fail, they can keep operating until a replacement is found
Pay no attention to demon Jeff trying to escape at 12:10. Just go buy a pint glass from craftcomputing.store to keep him at bay.
I was starting to wonder :D
It wasn't an impression of Max Headroom? :)
@Benjamin O'Neal I'd recommend an Optane drive, or enterprise flash of some kind. But yes, you'll need a quick, write-endurant SSD for DeDup.
Also, I do have a PO!
Craft Computing
1567 Edgewater St. NW, #51
Salem, OR 97304
steam deck is coming so show this in #ARCH with #KDE
@jeffgeerling has RED SHIRT JEFF, you have the demon Jeff hahahaha
Would LOVE to see a full-on series on practical home uses for server hardware :)
My server sits in the media unit, it hosts pihole, file server, plex and gives output to the TV via a VM. My wife just thinks it's another pc ! Total wife approval factor.
Do you own a lot of home movies that you would like to save? Maybe some videos of your grandparents gone? A server may be an excellent use case for that. (One of the many I ended up investing into a used enterprise server) I'm also a DJ in an underground scene where songs are called plates, which, to make an explanation simple, they are unreleased tunes that are played with a limited time span, and sometimes get brought back for a throwback tune in a mix. It is VITAL to me to have a way to keep those songs saved and backed up so i have my favorite, (and also very unique and exclusive) setlist.
Sorry for the really long rant, but those are my use cases for having a server at home, just wanted to give you some extra insight!
I have 4 ESXI virtual servers, and 1 server operating as the NAS, with 24 60GB 15k drives, and a external drive chassis with another 25 600 GB drives. The NAS server provides storage to the ESXI servers (they all boot of iscsi luns, so they have no physical drives in them), networking is provided by 2 Brocade/Rukus 24 port POE 1GB, 10 10Gb and 4 40Gb ports.
All of my services, firewall, active directory, media server, virtual desktops, etc run as virtual machines. Using vlans, and VMwares features, I can move my firewall, AD, backup server, etc between the virtual machines without ever having any downtime.
For me, as an IT security professional, it makes sense, and is fun to play with, and can setup environments where malicious traffic cannot escape.. Downside.. is around a $300 electricity bill every month..
Years ago people at work bought small HP home servers, as they were cheaper than buying a NAS enclosure. I never bothered, as I have all my rust onboard, I was planning to upgrade, but Covid pit paid to that. Ah well.
@@squeekymouse89 my wife learned the hard way not to unplug anything, as she unplugged the Pi-hole and "WiFi stopped working" :)
For people with multiple PC's that don't like the limitations, you can install a lancache server which proxies the steam connection. When you download a game it caches it on the server. Sure, your desktop PC won't save any space, but uninstalling and re-installing games will happen in a flash since all of the files needed for it are already on your own lan.
I'm disappointed that they still haven't adopted the native Steam cache domain name that only need a dns server to fetch the IP for it
"In a flash", for example if you play a 120GB game, its size is equal to 960Gb, a common LAN speed is 1Gbps, so it takes you 960s =16min to download game, and more to install it.
That's super annoying having to wait almost 30 min to play your favorite game
@@bacphan7582 120GB games can be counted on one hand, and all of them are competitive so they are already latency focused, i.e. completely out of scope of this option
Yes SIR! This comment needs more likes!
@@bacphan7582 If its your favourite game though.. You wouldn't be constantly deleting and reinstalling it. You'd keep it installed. :P
I use lancache.. Yeah its not perfect but if you have multiple gaming pc's in your house then its pretty useful, especially for win updates too.
For example:
If all of us buy a new game and need to download it. Technically only one of us will have to download it which takes roughly twice as long or more than the other systems downloading it straight from lancache. (Also good for people who are limited on bandwidth but not really relevant to this)
So it still cuts down download time and makes a big difference. I'd prefer this rather than having to download a 100GB game on 5+ computers.
Yeah... This video would probably be more useful in some situations but due to also having a few gaming laptops in the house. I'd rather not expose my server publicly just because my games are hosted on my server. Secondly, if I go somewhere where the connection is not great, trying to play it from your server on a slow connection wouldn't be the best idea but if you had them locally installed, then you might just about get away with it.
Both solutions have their own benefits. I'd definitely recommend this video if you don't game anywhere other than your home but if you do, I'd probably recommend lancache.
I'd still recommend lancache for other benefits even if you don't use it for caching your game downloads.
From experience I can also mention the following:
- BattleEye games such as Warzone will work from the iSCSI drive but won't work from network drives, huge bonus to iSCSI
- The SMB share can sometimes be faster because the protocol is "simpler" (compared to full-blown NTFS)
- For sharing the same drive across multiple computers, the solution goes like: Create a BTRFS/ZFS Volume snapshot (with CoW) based off the actual steam library. Have each client use a snapshot. This has some drawbacks indeed but with a list of pre-defined games it works wonders.
- Another alternative to iSCSI is using Dokany. Run the mirror sample to the UNC path and you'll have an almost-right alternative to iSCSI. Some Anti-Cheat solutions will still complain about this, though
i played destiny 2 over smb 2-3 years ago, so maybe not amm battleeye games.
or did you meant activision's RICOCHET, bc cod uses their own, and not battleeye
When using the same drive across multiple computers with iSCSI with replication/snapshots, would that drive be read only or it would be two way sync? Trying to find a solution for video editing on 2 computers directly from NAS with iSCSI and dedup
@@RodrigoeBeta I've found some other solutions all these years. iSCSI generally works with a whole "device", you usually get corruption if you attempt to mount a device in multiple locations (assuming the server even lets you do this). The solution I've found is a ZFS pool (normal pool, not virtual block drives) and NFS 4.2 with "nconnect=16,async" and 4K block size. In my scenario, I give clients an overlayfs mounted path, such that their modifications are personal, while still sharing a common base. Dedup in this case works on a file level. In my case, the clients are all Linux, but I assume this can work correctly with SMB as well. It depends on whether you want your clients to see each other's modifications, but regardless, I've moved away from iSCSI as it's use cases are usually quite specific and not suitable for scenarios like shared access to data
we need a few sequels to this for sure, focussing on at least two different scenarios.
1. One steam account that accesses the network attached steam library from multiple pc's
2. Several steam accounts accessing a single game from different pc's simultaneously (this will certainly need ssd's as storage medium server-side)
Great video though, keep it up!
*Connect to the iSCSI on each other computer and you should just be able to search for the game files in steam, verify, and play them. As f* or muti accounts it (in therory) would work the same process jsut with a different account. I can get a buddy of mine to test his account from my seocnd comptuer next time he is at my house. I will say it still could fail since scuzzywasn't really made for 2 connectsions. But since they are really only reading it may work fine.
Okay I ran a test using famly sharing with an old steam accout we call that PC2. PC1 is my main rig with my main steam profile that I installed all the games with and orignally moved the to the scuzzy with. I was able to start up a game by borrowing using famly share on PC2 connected to the same scuzzy. I was able to play it for a few min on its own then started the same game on PC1 from the the scuzzy and it ran fine on both for the 5 min until steam kicked me out because once the owner starts the game while family sharing they kick you after that time.
@@teetertech FOr anyone who decides to try this, be aware that the drive is showing up as local, so things like "last access" times and permissions (user/owner) information are going to be edited by the system even if you are 'only reading'.
In a lot of cases that won't cause issues, but if you have a game with a large data file being read from multiple machines you *will* eventually run into file corruption from conflicting file updates.
Again, it's just a steam library, so it might not cause you a ton of headaches to have to reinstall things or verify files on a regular basis, but it's still a problem to be aware of.
@@PoignantPirate yeah I had a game update and it disappeared on the other. So it definitly is a big no no but it was cool to see it work but long term it will just ruin things.
@@PoignantPirate What if you have two different pc's, one playing a game at a time? (ex. my SO works days, I work nights, we play the same games between shifts). As long as one of us powers down our pc before the other turns theirs on would that prevent the file corruption? Also, so the way to have multple, (2 in my case) pc's connect to the same Steam Library would be to install Steam on the server, connect both pc's by the same iSCSI LUN, just one would be kicked off while another uses it?
This is really cool. I do remember putting a Steam game on my NAS to see if it would work and not being too surprised when it didn't. Had no idea it was as simple as how Windows reads network versus local disks. Thanks for making this super simple. I might have to try this purely for fun, or perhaps as a different storage config in the future.
My entire Steam library has worked perfectly for over two years on my NAS as a normal network share.
@@jsncrso Yep. just create shared folder on NAS, map shared folder in Windows, install and done.
Most of my games work fine but there are certainly exceptions; a lot of MSFT games for example seem to take offense, Halo and Forza in particular seem to not much like being on the network. Sometimes it works, sometimes it just doesn’t at all.
@@1armbiker Ahh. ok. thanks for the warning. I've been using Steam and Epic Games stores. Anything M$FT I'll keep local (or buy through the other stores).
This by far even today is the best video for getting this setup smoothly. I am using Cobia and even with the few changes I followed this guide today and worked like a charm.
Thank you!
A deduplication video would be awesome.
Just showing the space savings by having 2 similar iSCSI steam libraries on the same ZFS pool sounds like a great demonstration
This!
Yeah true, but ZFS deduplication cannot be recommended!!! It only sort of works if you a don't reboot and can store the deduptable in RAM or b you have a really fast persistent L2Arc like an Optane drive.
Tbh it's not worth the drawbacks right now. The only dedup that works alright without those drawbacks is vdo. Also deduplication isnt really required for the space saving in this instance as you can just install all the games and then just snapshot that drive and make another share of off that
Modzilla how would you go about that? Install games, create snapshot of Steam dataset for each user/computer, have each user’s iSCSI point to their respective snapshot? What happens for game updates?
@@ReturnJJ yeah that's where it falls apart a little. You'd need to a another clone/snapshot. But I think that would be a fair trade since all the save states are stored somewhere else and this way you don't need to download updates more than once.
@@LampJustin I just had an idea, what if the Steam Library has its games installed and updated using a VM but there is no actual gaming down on there, and then have snapshots for each user and a snapshot for the Steam Library VM so that the VM can commit game updates and installs?
I don’t know why I randomly thought of that while I was bored, but there’s that.
That was the last straw for decision to build 10gbit network - thank you.
Some used Intel NIC and a Microtik 5pt 10Gb switch and DAC cables is a cheap easy way to get 10GB
Haven’t noticed any lag from my server! Saved me so much space, thank you.
This video damn straight may have solved my game storage dilemma! Been using my Razer Blade for gaming since the main pc gpu died and the RB only had the 500gb nvme and 1TB HD. I would use an external ssd to transfer games. I’m getting ready to switch my home network to 10gbe at the new house! I can get ssd loading speed on the laptop through a TB3 dock and access my whole library this way now!
I... need to think of what I need to say here.
You are doing everything that my mind has been dreaming of for the last decade. Im 23 and still fairly young and inexperienced in the job market so money isnt easy to come by as far as jobs go. Now that i finally have the gaming PC I want and a server, with your channel, I will have the godliest tech setup before I even own a home XD
But seriously. Every video of yours I watch has answered some sort of question, or put to the test some sort of theory ive wanted to try, but up until now, never had the computing horsepower or experience to try. Your videos are AWESOME and very helpful, plus its nice to enjoy a beer with someone over the internet! So Cheers to the future bud! Glad to be a subscriber, and I cant wait for more gaming, and server content.
I have been running like this for several years now and it has worked flawlessly. One added benefit is that, even if you run on spinning rust, TrueNAS will cache the data in memory (or L2arc). So after a couple of days you will probably not even hit the disks. Windows does the same off course, but presumably you run some other thing on your Windows host (or reboot/turn off) and then the cache is lost.
Having been a nerd all my life and worked in IT. and telecoms I genuinely learnt something tonight when watching your video! I now have iscsi working on my truenas core server to my pc. been using freenas/truenas for about 9 years and never knew what it was! Thanks :) subscribed.
I'm finally at the point with my Steam library (where I am having multiple copies of the same games installed on multiple systems) where this video actually came in REALLY handy.
Thank you!!!
It's cool to finally see someone else do this. I ran my game library off my NAS for about a year and half. Eventually when NVMe got cheaper and I needed my NAS storage fore more projects I moved my games back to local drives.
Here are some things of note I ran into:
-Windows ISCI can be temperamental. Had to set the service to wait before making a connection, since my network drivers weren't loaded fast enough on boot up.
-Some newer game with anti-cheat detect something isn't quite normal. Either the game won't launch or runs awful, since the anti-cheat does intrusive scans in the background.
-Unlike SMB, ISCI doesn't do any integrity checks when transferring. File corruption is possible but fortunately steam has its verification tool to fix this.
-DON'T USE EPICS LAUNCHER! The final nail for my ISCI partition was when I installed UE4 to do some development. It filled the drive with cache files and failed to clean up after itself.
Hi Jeff!!!
Love your content. I've used many of your videos for my homelab, especially with the chenbro 1u servers. I built a truenas server with the chenbro 12 bay server and was able to achieve read writes in the same ballpark that you achieved in your video!!!
Thank you so much for your time and your expertise. can't wait to see what you come up with next!
I love you. I discovered your channel about 2 weeks ago because of looking for a decent video how to run a 1-GPU-2-Gamers-System. You're definitely my guy. This content is gold 4 me.
I've been subscribed for months and just built my first home NAS out of an old tower I had lying around. Then I find out you already made the perfect tutorial for EXACTLY what I wanted to do! I never uninstall games for fear of losing something, but I can only fit 2 hdds in my itx gaming computer. Now I've got more than enough space. Would be interested in seeing if an existing SCSI pool can be expanded with extra drives in future.
For the iSCSI share, you can use RDMA enabled hardware (10gigabit sfp+ Infiniband mellanox connectx2 in IT mode) with a modified hosts file that points directly to the iSCSI share (or truenas server) so that both devices can communicate without any overhead. NOTE: both cards must be in IT mode if they are the same and also it's a good idea to add the IP address of the client that's requesting the connection of the share.
Amazing, you're always making my dream projects. I get satisfaction by proxy.
For years iSCSI has lurked provocatively in the background as "what's next" yet the first time I see implementation details, Jeff is installing his Steam Library to it Fantastic!
thanks for the great video, at about @13:30 you discuss the Z drive and the E drive, however you only show the Z drive read/write speeds, would be interested to compare that to the E drive you set up
5:37 -- iSCSI downside.
Great video 👍
I slammed into this "2 initiators, 1 target" conundrum this week. Came here to get another viewpoint confirming this challenge. I am not going to do filelocking or a cluster-aware set up.
Just found this video. Overall, this is a nice video and is useful. Couple of things: iSCSI LUNs can be accessed from multiple servers simultaneously, and if you move an iSCSI LUN from one server to another that can mount the FS then the permissions will be maintained; the SIDs may not be resolved by the new server but it will, in fact, see the SIDs in the ACLs. Also, you created an E: drive but then benchmarked a Z: drive. And SMB is not commonly referred to as SaMBa; SaMBa is a free software implementation of the SMB protocol. Thanks.
Wow! You're right! I'm still new to iSCSI, so I tried connecting a Windows VM to the same iSCSI LUN volume I made for my laptop...and it worked right away! Awesome!
For those that have an Asustor NAS, this works! It seems we can't share Btrfs snapshots, but this LUN alternative seems way better anyway.
This is a place I need to remember 6:40
Hey Craft, great video! I have noticed the UPS in your rack is showing 94% continuous load which is bit high. I'd plan on splitting it up with another UPS to give your servers longer run time and less chance of UPS failure. It's best to keep it under 75% load.
Well aware....
It's working ! I was using SMB but after watching this video I configured iSCSI Service in my TrueNAS server and in the iSCSI Initiator app which are connected directly through PC to PC hence only me has access to it.
Great stuff! Immediately went and did the same using my Synology NAS spare capacity -- surprisingly crystaldiskmark was able to write/read at 118MB/s over a simple gigabit network!
I will personally only use this new drive for very large install, for games that require one very large loading at the start (e.g. flight sim), vs. those who have frequent smaller loading (e.g. MMos)
Which diskstation do you have?
@@MrMackster01 an old 916+ with 2x 4TB WD RED -- nothing fancy. Just tried running Flight Sim 2020 from it and loading time were just fine
Im so glad im not the only person doing this for their home network. I havent really run into many issues doing this via networked attached storage pointing to an unraid share, as i need multiple clients to access the same Library, but excited to see your video on de duplication though as I hadn't considered it as an option.
So the benchmarking at 12:48 appears to be of the network drive mapping (Z:) rather than the ISCSI drive (E:). Can we get the benchmark of the ISCSI drive too?
E:\ Volume was crated "inside" the Z:\ Volume, so basically it is the same test E or Z as it's going to use the same path to the HDDs
And aside from my other comment.. You can take it one step further, and take advantage of all the features of ZFS and Truenas. Boot from an iSCSI lun, you gain all the advantages of ZFS and Truenas as you noted, but that extends to the boot drive as well, and you can use replication and cloud sync to backup your entire environment, not just the Steam library, and you gain significant performance bumps.
Add to that the use of cheap Infiniband cards (can be had for less than $50) and Infiniband switches (found a 36 port one for $100), and you have a 40Gb or 56Gb infrastructure depending on which version you get).
But those speeds are irrelevant if your storage can't transfer that fast
@@mrmotofy I have 2 25 arrays, most of the disks are mirrored stripes.. the storage can easily keep up.
Excellent and informative video. I used the steps outlined in this video to accomplish the same thing using a Synology NAS. Interestingly, I've found I'm getting much better and consistent read/writes to the iSCSI disk versus a mapped drive via SMB. I was always getting wonky performance via SMB with file transfers, only sometimes maxing out my 1Gbps network connection and more often transfers were in the 150-250Mbps range. Tweaking the NIC parameters, drivers, etc...nothing helped so I chalked it up to SMB. Making the same transfers via an iSCSI mounted disk on the same NAS is almost always maxing out my network connection. I'm a happy camper with this solution.
Yes, the difference with smaller files is immense. Even a 10gbit connection can feel totally wonky over SMB when you transfer f.e. a games folder. with iscsi it just chucks along at whatever the NIC can transfer.
Your timing is great. I figured I would just use a mapped drive on my fresh NAS for steam games. Thanks for saving me the headaches!
Gotta get me one of those oyster IPA. Got a guy that always drops by and scams my beer. It’s Revenge time. Thanks Jeff!
If someone would be worried about the speed over the network connection, once you've setup more then one folder for your steam library, you can always move the game around between these. You can do that by doing a right click on the (installed!) game in your library -> properties -> local files -> move install folder. Steam will show you a list of all available folders and you can easily move the game over from a network drive, to your local drive. This way, you can store all your games externally and move them just on your PC for playing. Of course it takes some time to move the files over the network but its probably still faster than installing the game over the internet and may or may not result in better performance for the game (or at least loading times).
Or you use a local wired connection that doesn't suck. 1g is more than enough for games.
@@nevoyu It probably really is but I haven't tried it on my own yet to say much about it. Just wanted to point out a similar but different solution since not everyone has the luxury of running a wired local connection, an actual NAS or even broadband internet connection. For quite some time, I also just used a raspi 3 with an external USB drive for a NAS and while this is fine for some tinkering and basic setups, you'll run into some bottlenecks.
@@4x4cheesecake_19 Yep the Pi 3 is the bottleneck a 4 is so much better in most ways
@Tim v H Generally a NAS is used for multiple user or device access. They generally also have more redundant storage of the data for Better long term data integrity. And less risk of loss. Depending on the Network speeds and drives used a NAS can be as fast or faster than local storage. If you are a one machine gamer, then yeah you probably don't have a use for a NAS. But many people have multiple machines multiple users that all need to store data they don't want to lose,, so a NAS makes sense
You're videos are so good I almost had this memorized after one watch. To bad I'm dumb and forgot it needs to be a zvol not a dataset with quotas (face palm) Great job man!
Dude, you rock man. Very straightforward instructions.
What a great tutorial. I really am thinking about switching over to TrueNAS. Great job!
At 8:31 the option for Sharing platform from what characteristics it is depend on? The physical characteristics of the disks true nas is based of (like 512e or 4kn) ? The disk's characteristics the other platform has? (assuming it is a proxmox based on 512 logical / physical drives). The OS the platform relies on? (I mean if it is capable of using 4k as the minimum amount of blocks? - which platform wouldn t)
Thank you
That 4TB/5TB ~$60-70 argument used to be something I considered until power+packaging became my main concern over a non-hotswap rack that goes down every year for cleaning, every 2nd year for upgrade and every 5th year for fan/PSU replacement. Last week I picked up a 16TB Toshiba MG08 for $320. That's roughly ~$20/TB but the 9 platter packaging is so dense that I might pick up a few to make my first RAID volume and use my 60GB refurb SSD for failover journaling. I'm still getting everything settled to make this new disk my main video storage replacement and migrate my other disks to move deploy volumes for apps, games, backups and non-Win platforms. So considering power and disk health, the only thing I really worry about is the moment my disks reach 100K hours. What is your main health concern with your equipment and how do you retire your disks after they reach a certain age?
Thank you for mentioning the facts about ISCSI, I've seen so many misleading videos about this solution. I'm trying to build a network setup with some SSDs in raid 0 connected to PCs all over the house equipped with 5 to 10 gigabit network cards/adapters. When I get a game working right, the latency is not much worse than a local SSD. The problem is getting third party launchers working with it. Battle Net (blizzard games) will not work anymore, doesn't support NAS setups, GOG can be solved with some writing in command prompt and a registry edit, Origin can sorta be solved but only if you're playing games right from Origin. Some anti-cheat games, specifically ones with battle eye (rainbow six siege) and Origin games bought and launched through Steam but still using the Origin DRM WILL NOT WORK. I think it's a matter of inputing some kind of permission in the command prompt and having it start with extra permissions but I for the life of me cannot get the syntax right. I'm using SMB right now.
Anyone have any ideas?
I mount a vhd on startup that is stored to my nas. Blizzard works just fine this way and while I haven't tried Origin I have used Ea Desktop and that works fine in my limited experience. I oy play 2 games through EA
Finally, I remember I personally requested for this. Thank you sir.
You can also do this with portable applications where you can set a NAS as your pool resource and any computer on the network (with access) can open the executables. I do this with Office, some of adobe CC, and even roms for my Rpi's.
But if your bandwidth is good enough you can even switch it to a SAN or iScsi
I like the approach ... instead of stressing out, simply enjoy your pc tech hobby with a good beer
cheers mate
I did roughly the same thing however I used a virtual hard drive and then mounted it locally from the network. It seemed to work just fine but every once in awhile it would hiccup and I had to tell steam where the game library was. I will be doing this
This was my method till the vhd crashed.. 8tb of my games lost. Inc 3tb of mods and configs.. it HURT!
Def go the iscsi method
100% answered the question I had. I mounted my iscsi target to VM thinking I could share the lun between my main computer and the VM. this explains why that failed.
Thanks for this!!!! This was exactly what I was looking for (not for Steam but other use cases)
Good point about spinning disk performance compared to SSDs. It can take some config tweaking, but old hardware can still be remarkably capable. I've been able to push arrays of ~20 2.5" 15k SAS drives to about 500 MB/s write.
For the speed test, the iSCSI-backed drive has a filesystem mounted on E:, but the volume being tested was the SMB network share on Z:?
It's also nice to see that the iSCSI sharing settings of TruNAS are more beginner-friendly now! From my days as an ESXi admin, I remember iSCSI being very fussy about latency and heavy packet loss or interruption. Have you noticed any issues like that, or have you put measures in place (e.g. QoS/CoS) to mitigate them?
Man do I wish I had Gen9 HPE kit in my lab 😅
EDIT: reordered content.
How and where do you do the tweaking for your network speeds? That's my next task for my TrueNAS lol. I'd like to be able to utilize the speeds I'm paying for.
The tests at 12:45 are done to your Z: drive, which you said was a Network (and I'm assuming SMB) share.
What kind of performance do you get in that same test for the E: drive, which is the iSCSI drive?
Without the comparison between SMB speeds and iSCSI speeds for random access reads, I don't really understand why you would care to use one or the other.
Super informative and to the point! - Keep up the great vidoes mate!
Great video, Jeff! And great timing I,ve been wanting to build a steam game sever 😎
Going to have to try this out at some point :)
I do have a steam library set up on a standard smb share on my NAS at the moment, but i use it to archive games i'm not playing recently instead of actually running games from it.
(use steam's move folder feature to move games from one steam library to another, so i just move a game from the NAS library to one on my local SSDs when i want to play)
For anyone it may help, if you use capital letters for the target name, the wizard won't complete. This is in TrueNAS-13.0-U3.1. Great tutorial and videos.
Thanks for this video!! Also wanted to say I love your content but somehow missed this video until now.. setup is almost identical in TrueNAS Scale for those wondering..
5:40 so can't have two users to play the same game on the same share storage device? (still learning)
I was thinking just using a NAS as a storage device, so instead downloading the game again, I can just simply keep it stored in the NAS so either me or the other user (my dad) to play the same game
please do a video on de-duplication this info was very useful and i love your tutorials
If you want to build a true nas server and the drives you have already contain data you can't loose, how do you solve this issue. My issue is my emby server ( and I use the term server loosely) is simply a spare windows computer with large drives full of media. If I pull out those drives, wipe the boot drive and build a truenas rig, how do you a build a storage pool without loosing the media I have?
Guess you'll be off the Alaskan brewery Christmas card list with that review. Brave of you to try and keep on drinking it.
The first couple 49th State I had were wonderful. And there's always ANYTHING from Anchorage....
@@CraftComputing I'm not a stout drinker so I'll take your word on them :)
Just used this guide to setup a steam extension for my PC! Ordered some 10G stuff to see how the performance compares to my current 1G network.
What a timely video I'll be setting up one of these very soon
Something you may also want to check out is the diskless config that use by gaming cafes in Asia.
A more simple approach is to create a veracrypt disk on your server. When you mount one on your computer it works like a local disk. I just checked my crystal disk mark speeds and they're similar to these results so the performance is comparable.
Jeff thanks so much for the awesome video!! This helps me to learn very quickly!!!!
In a follow-up video could you cover iscsi snapshots? Specifically, if there is some way to have scheduled snapshots from a golden volume. The idea is to have one machine that keeps a golden volume up-to-date and once a day the client volumes get replaced with another snapshot of the reference.
This might not be perfect though since I think the steam library folder may also house some saves, and those would be overwritten. I'm just not sure. I think maybe mapped samba for most games is best, then iscsi for the games that complain, then local install + samba link saves for the games that complain and keep the saves in the steam library.
@ 1st sponsored segment: Most phones that still include chargers include ones that don't take hours to charge your phone. It tends to take an hour, max.
Hmm, this wasn't what I was looking for, but definitely gonna get back to it. iSCSI definitely sounds relevant.
14:00 so if you download GTA there you have what two options, play it through NAS and wait for ages in loading screens, or transfer the files over the NAS HDDs to your local SSD?
Actually a lot of people with slower internet speed might benefit from that anyway, as transfer through one drive might be faster than their internet speed.
Cos like, it's fine to have a storage of games saved, it was even more so the factor in the past with offline games. But with high enough internet speed it doesnt really matter anymore, in cities you can easily get 1-2Gbps. But yeah for games that dont actually require high read speeds, being just on your HDD might save you time and storage.
I wonder tho if you make raid 0 or raid 5 if the speeds increase and by how much.
I'm also interested if you can put SATA SSD into the array instead of HDD, i heard you can use it as a cache, but i dont know how that really translates into the whole play, if anyone would explain
I am running this Setup for some years now. Truenas has a Lot of Features that can Help Speed Up your storage while giving you more space to expand. I got some icydock hotswapable 2.5" trays which Serve as my Main storage. Only Problem will be Direct storage which only works with nvme. I Hope truenas Soon has nvme over Fiber.
Hi! I have a desktop back at home, but that's 6000 miles away. Do you think that I'll be able to use my 8tb HDD on my desktop as a library NAS and remote it all the way here from my laptop? Tysm.
Nope, this is for local network storage only.
@@CraftComputing Okay, thank you so much. Excellent video by the way!
@@CraftComputing so if he VPN to his home he cant use this?
@@theangelofspace155 he sure *could* but it would be incredibly slow due to the high latency and generally low upstream speeds of residential internet
@@theangelofspace155 It would not be worth it. It would be DESPERATELY SLOW and almost unusable. So many problems with that (plus latency issues) -- not worth it.
I have my steam library on a share on my server running unraid, on an unassigned 2TB SSD, been doing this for about 3 years works perfectly I can play games from whichever gaming machine in my house, cool video though!
Bet you cant install blizzard games....
I appreciate you putting this video together. I'm going to attempt to create an iSCSI host and initiator with Linux servers and PCs.
Thank you so much for this video i have been trying to figure this out for a long time
I run my Plex Server on a separate system. It connects via SMB and the other backup over NFS. I just edited a nice fstab file and point plex at the host directory. iSCSI doesn't allow sharing.
This allows my main machine to transcode those nice BluRay and UHD disks and copy them to the plex server remotely.
And here I am, trying to be happy with my 10TB NAS being accessed through a 1Gbps interface.
It's difficult when everybody is switching to 10 gig. I switched and started getting about 75% more throughput on a plain old HDD. Now I get about 380MB transfers on all HDD's, more drives will just add. Ore capacity and more speed.
I have a very basic pool on truenas, 12tb (4x4tb) ZFS and it works great as a network drive holding my steam librairy. I opted out of iSCSI for the very specific dealbreaker you mentioned as I would have a hard time accessing my saves from different PCs (I game on 2 different rigs depending on time of the day as I work from home too!)
If you use Steam, you should be getting the appropriate save files through the steam cloud. With the de-duplication, you can just set up two iSCSI drives and fill them with the same games, and the NAS will take care of de-wasting the space.
“Servers in the garage”. Hmm, I’m in Texas so that would be thermal throttling hell 11 months of the year!
This is especially a good idea for Mini ATX/ITX builds with the newer 2.5Gb ethernet standard; you can easily run your OS from an M.2 disk and not have to worry about mass storage :)
Multi-mount volumes are possible using Windows Server, if you want to have each PC running server with Cluster Shared Volumes. These show up as local mounts, but oh my CSVs are a real PITA. SAN's like Nimble support "multi-initiator access", which is a pre-req for using the SAN with CSVs. Such volumes can be mounted by multiple hosts, but again, we're talking servers, not desktops. The same setting is also required for MPIO, and works normally in VMware with multi-host shared datastores.
Anyway, I use Steam Library sharing using mapped drives. No UNC paths, only drive letters. It works A-okay. Now the case for iSCSI does matter for companies like Blizzard who detect that libraries are running on shares, even if they are mapped drives. They disallow this because Blizzard has fallen to the dark side of late and is strongly anti-privacy and anti-choice.
Some iSCSI best practices:
* You should use a secondary NIC dedicated for storage traffic if you can. 10Gb is definitely beneficial, even if you have a direct connection to TrueNAS
* 1Gb connections really benefit from Jumbo Frames (9000+ MTU vs 1500). If you have pretty much any managed switch it will do this , but everything in the chain (Nic on windows, switch ports, TrueNAS) has to have it set. 10Gb there’s not much of a difference.
* MPIO is supported on most Linux and Windows Server OS’s. Windows Desktop does not unfortunately support multi path, but you can setup multi path in TrueNAS to effectively double bandwidth if you have two NICs and the OS supports it. You see this a lot with Hypervisors.
* Do not use Deduplication if you care about performance and have low Ram. It’s a bad time and honestly not worth it unless the workload dictates it.
I never had much issues running my steam library or other games on my SMB share mapped as a local drive. Can I expect significant speed increases compared to a normal SMB share even on just a gigabit network card?
I've tried both over 1G and 10G, and frankly for games and with a moderately sized HDD raid, the speed difference wasn't really noticeable.
@craftcomputing, fyi, just checked like 4 days ago, but SAS3 8TB SSDs (7.68TB) drives can be had for
Greatly informative, I like how you break it down.
🤔 I think I watch your other channel, Redneck computer geek?
The only downside about this channel is the free advertisement for drinking alcohol
At 12:08 not sure if intentional but you have a glitch and then your audio is out of sync. It's a nice touch if you planned on it to show potential issues with the iSCSI drive over the network however.
This is really cool. I was suffering running launchbox from a network drive. It would freak out every time. I ended up buying a ssd to add the retro collection to it (I have 2 firecudas nvme for main system in raid and now 2 cheap wd ssd in raid on my pc) I think I will move the ssd to truenas and do this. I have a 10gbe link but no drives fast enough to test it hahahaha.
playnite is way better
The suggestion to get some beer was a good one. I did just that and it is satisfying.
OK i can apparently share iSCSI even with a pocket router i got 10 years ago for $20 with a USB drive hanging off it and that has been running OpenWRT since about a week after i got it. That's nice. Not sure i'm going to do this, because performance will be naturally limited and single client limit, but i appreciate knowing that i can.
I haven't tested this myself but if you run virtual machines and connect the iSCSI target to the hypervisor you might be able to use the disk on multiple VMs in read-only mode. This will show you a warning though as you are sharing resources between VMs which are not meant to be shared. However, I would view it like mounting an ISO file of which I used some to install multiple systems simultaneously, before.
this is awesome....this method could mean my dropbox sync could be allocated to this drive and then backed up as required and mean it is on my server and not my desktop....how good is that!....could come in handy for all kinds of things to be honest.....thanks for this jeff 👍👍
I had started out using a mounted virtual disc that was stored on my Unraid NAS, and while it did work fine and all applications and games just saw it as another disc drive, there was one glaring problem I couldn't get over. No matter what I did, no matter how many guides I followed, etc, I could NOT get the mounted virtual drive to disconnect cleanly on windows shutdown and sleep. Because it wouldn't cleanly disconnect, every time I restarted my system the drive would fail to reconnect because my Unraid NAS still showed it as connected even though it wasn't.
So I gave up and moved everything to a mapped network drive. Some applications (like Origin games launcher) didn't want to play nice at first, but there seems to be easy workarounds for every troublesome application I've come across so far. So for me, mapped network shares are turning out to be the better option.
Awesome guide i used this and added the drive to my steam deck and the read write speeds rival the SD card speeds.
That's not an iSCSI limitation, but a Windows file system limitation. I've absolutely used clustered filesystems with multiple clients per target in the enterprise server space (Oracle FS, etc).
A quick question, would this affect the loading time of the games I installed? I assume it will load up slower due to it being a network drive running on hard disk format.
Overall, a good video.
I would like to see a better solution than using deduplication for having multiple copies of the Steam Library though because whilst your server has 64 GB of RAM, so ZFS dedup is not a problem for you - if you have a server without that much RAM, ZFS dedup, as a function of your vdev space, can become a problem for said ZFS dedup to work properly.
It would be nice if there is a solution that combines the pros of both without the RAM requirement that ZFS dedup has.
Thanks.
What about and iscsi LUN? I have 4 esxi hosts talking to one NAS with a central file store on it. They all see the same data ?
I did this with my game libary a few months back good how-to man!
My Steam Library stays on SMB share and it seems to be happy. It is true that if I try to install applications on SMB share, some programs prone to be angry, but Steam Library has been working fine for me so far.
So when you say "Used Enterprise Drives", how do you find them and how do you verify if it worth it?
He has eBay and Amazon affiliate links in the description. Generally should be fine for home use. I'd say it's worth it for the money IMO.
@@computersales Yeah, I'm pretty happy with the 8 1Tb SAS2 drives I got for $18 each. Had to send one back but no big deal.
It recycling places can shift off discs I just bought 20tb for around the same as a new 2tb SATA..
Depending on your controller/set up you read the smart data, and don't buy too old one or too many cycles. Hours are fine and most are sold at 20k mark or after 2-3 years.
Any drive new or used can have faults or fail, so old or new doesn't really make much difference.
But......
You have to factor in buying a sas card (HBA) and cables or backplane to make it work.
@@guywhoknows yeah the two SAS2 breakout cables were almost as much as the controller and you can't really cheap out on it
The important thing with used drives is not to trust them. You should always back up your data, but especially so with less trustworthy drives.
Most people using used enterprise drives use them in a RAID, so that when they fail, they can keep operating until a replacement is found
This is cool Jeff! Thanks for sharing!