This is kinda blowing my mind. I havent built a pc since 2011 and havent really paid close attention to the advancements. Things have changed soooooo much!
Am always proud of my fellow swedes. Fractal Design Node 304 (10 year old design) with 3 mirrored pairs is "my precious" storing all my data. Let's look if Terra holds at least half of this time :)
I took most of your parts but decided to build it with a jonsbo n3. I dont regret, its a really neat NAS with the option for 8 3.5 inch disks + 2 2.5 inch SSDs and the option for one additional nvme. Also added my sfp+ card of course for some speed :D
I love when some of my favorite creators decide to make content that happens to be relevant to me, all at the same time! I'm currently planning a DIY NAS inside a Jonsbo N3 case, which I've already ordered. For the platform, I'm look at using an ASRock IMB-X1231 motherboard paired with an Intel Core i5-14500 (probably). My main reason for this combo is ECC support (I know, it's a polarizing topic lol). I'm still on the fence about installing TrueNAS bare-metal or as a VM inside Proxmox. I want to give Frigate a try using a Coral USB Edge TPU and I'm worried the extra layer is going to make things complicated. Great content as always, Tim! 😁
Thank you! Yes, you can pass anything through to a VM. I run ECC in my servers but when it comes to consumer stuff, I don't bother even if it's providing a server role 😅
I love this application. I am looking to build something similar, although hampered by my low level of software skills. I am hoping to re-use my large Lian-Li tower with fanless PSU and CPU for the perfect homelab - and more space for drives! I am more confident having seen this video. Thanks for the effort :D
I was renovating a house but because of different reasons I was forced to temporary move into an appartment where I cant have my rack (well, I can't hide it enywhere, wife is not keen on having it in the living room). So this is just perfect timing for a build like this. I will be repurposing Define R5 case from my gaming pc for this build, it gives me a lot of space and is also very quiet. But I'm thinking of using Terra for the new gaming PC.
I've read on the forums, that you should pass through the whole HBA card/sata controller and not the disk by itself. Apparently you lose some SMART functions that are required for ZFS to detect errors and do its self-healing. I've never actually tested this, so just speculating here.
Thanks! Yes, always HBA for sure if you can. I had to do individual disks though because I was using the same controller. 2 for TrueNAS and 2 for Proxmox VMs.
@@TechnoTimTinkers Tim do you have any concern about losing data because you passed through only the drives. If you read anything on the TrueNAS forums they insist this is a formula for disaster. Yet, I have done this exact same thing on my testing machine for a while now and never had a problem. Interested in hearing your point of view on this. Thanks.
Nice build. Been debating building something akin. But would love to include space for 6 + 3.5s maybe 2 2.5 or NVME and room for a 3090. Do all my NAS, AI, plex, etc. on one box and even maybe run a gaming VM on it in down time. i9-9900k still so with that would need room for AIO. Less I upgrade the CPU and explore more cooling options. But for your use case this is not bad at all.
Thanks! I totally know what you mean! Once you start thinking of all of the use cases you start adding more and more. I had to think about this build as if I were 99.9% of the home users who just wanted something good looking, quiet, low power, and would never fill almost 30TB or storage 😂
I had the same wishes to build in the Terra for my NAS. I love the aesthetic of the Terra, but I will need expandability. My main server has been in a Corsair ATX tower running Proxmox and TrueNAS Scale virtualized. I wanted to separate everything. I went with the Jonsbo N3 and TrueNAS Scale Bare Metal. Jonsbo N3 was the same price as the Terra with expandability for the future. I really didn't want to drop 1k on a 8 bay Synology even though it would have been much less work. The Jonsbo N3 will be solely a NAS and now I'm continuing a separate build for my main server. I went with a Jonsbo C6 for the main server. I enjoy the idea of having everything separate.
Does your Mini ITX motherboard support PCIe Bifurcation? If so, you could make a 'Ultra Fast' storage option using the PCIe 4x4x4x4 Option with (4) NVMe M.2 SSDs.
I really hope Fractal ends up making a Terra+ that can fit at least four 3.5" HDDs. It's such a wonderful design and it'd be perfect for a smaller NAS if it just a had a bit more room.
@5:10 had same idea for another build but these "simple" peices were being sold on Amazon Canada for $94 for 6 three that idea out the window until i got a buddy with a 3D printer to make a dozen for $10.
you don't need to build out an 800 dollar machine to do this though add some ram to a 9th gen coffee lake box for half the price and have room to expand and add more drives while using more power for a year or two until you build something better - how most people should and will do it - why be extravagant in this economy
Great video, I like the build. Do you have any concern to a virtualizing TrueNas? I have a standalone TrueNas server and then I have another server that I use for virtualization, I went with XCP-NG as my hypervisor. I said it works perfectly fine, but I wonder if I could get a little bit more bang for my buck by consolidating down to one server?
Sadly they have effectively discontinued TrueNas Core, it's still going to get security updates for sometime, but that's it, The Register did a good article on this back in march.
Hi there. First of all congratulations for your video. I'm a newbie on this stuff, especially on linux environments, and after tried a few months a working units with Truenas Scale, it seemed too complex to get it working also as a media center using Emby, and also a few other things like qbitorrent, etc. due to the annoying permissions sets... So you triggered me a desire to have a try on this combo Proxmox + Truenas Scale, but i miss some skills on that part of getting the whole hhd drives pass-through available to Truenas. I'm aware that this wasn't the purpose of your video, but can you please help me out? Or maybe have a dedicated video for that? 🙏 Thanks in advance
I promise I updated the firmware and microcode 3 times 😅
27 วันที่ผ่านมา
How are the Exos disks doing soundwise please? I am building a NAS which will be right next to me and was looking at these disks, but I am afraid they will be quite loud (especially when not idle).
One issue is that your TrueNAS VM can't really utilize Proxmox's extra/unused RAM for the ARC cache. You have to choose a static allocation ratio. Why not just host the ZFS volume(s) from Proxmox directly? For the TrueNAS GUI?
And I've just built a nas with a ryzen 670e motherboard with a 9700x , a really bad idea after undervolting it by 0.1 volts and a maximum 300mv extra undervolting on PBO I can get it down to 50 Watts idle when set to 45 Watts tdp , making my 6000 ram run at full speed using expo uses extra about 10 Watts . that is with the igpu only and one ssd.
How did you manage to get the drives passed directly to truenas. I was under the impression that proxmox always had a layer that hides the bare metal drive and that the only way to directly pass it is with a sad controller. This has been a big pain for me personally given my hardware setup. Would love to know how you do it
If you don’t want to get a separate HBA card, as long as there is a SATA controller that has its own IOMMU group, you can just pass that through to the TrueNAS VM and it should allow TrueNAS to get direct control of the hard drives just like bare metal installations. I have been able to do that on a X99 machine, which has two SATA controllers and one of which has its own IOMMU group. Everything works perfectly in TrueNAS core, including all the disk info, SMART info etc. the only issue I found was the VM will lose control of the SATA controller after resuming from sleep and so far I haven’t find a solution to it. I eventually went back to importing the ZFS array to Proxmox itself and it works perfectly with sleep which allows me to use this machine as a on-demand NAS storage and avoiding the high idle power consumption of X99.
I have also tried what’s shown in this video to pass through the disks to the VM and it seemed to work. Everything appears normal except for one thing that is I cannot get the correct disk info, such as serial number, temperature and SMART info, which makes me feel shaky about it and went the other way around.
@@cyberbass4 o wow nice. Thanks for that. Might give it a try. Unfortunately my case can’t use an hba card because of the custom backplane of the dell workstation
@@Ezza232499 Yeah. Hope you are lucky enough to have such a SATA controller on your motherboard which can save you a lot of trouble. It is actually easier than passing through individual disks and all ZFS related stuff works perfectly, except for the sleep part.
If your mounting a network share from a TrueNAS VM to the host proxmox. Be aware that you will probably have to manually start and LXC or VM that uses it. As even if you start the TrueNAS VM first. The share won't be ready in time, and the container/VM won't start. How I get around this is running the containers from a VM using autofs or creating the volumes from NFS shares, and forcing (in my case docker) to wait for the mount before starting.
Good question, not sure. It's a great looking case too! I would just look at the specs and see how much wdith you have next to the PCIe card slot and then see if there is enough depth and width. It's tough to know for sure which is why I went out on a limb when I built this.
@@TechnoTimTinkers I appreciate it. New to servers and home labs. Trying to figure out storage for my photos as a photographer and dad. As well as a media collector
I also have coupled Proxmox and TrueNAS together and just love the benefits (console, VM snapshots, backups). Especially migrating went flawlessly. However, to have disk spindown and SMART I bought a PCIe SATA controller and passed that one through. Only issue I had was that I had to tell Proxmox to NOT try to import the TrueNAS rpool on bootup.
7:00 🤣
lol, if you need to expand add another one and have the storage as a Ceph clusters :)
This is kinda blowing my mind. I havent built a pc since 2011 and havent really paid close attention to the advancements. Things have changed soooooo much!
Am always proud of my fellow swedes. Fractal Design Node 304 (10 year old design) with 3 mirrored pairs is "my precious" storing all my data. Let's look if Terra holds at least half of this time :)
I do wish they would make a NAS in this same style! I'm here for it!
I'm about to build my first NAS, using that same case. Parts coming in by the day, can't wait!
the 304 is so nice. 6 HDDs is a sweet spot for me
@@therick0996 can take up to 8hdds, with some 3d printing
The older define revisions and node 804 are fantastic too.
I took most of your parts but decided to build it with a jonsbo n3. I dont regret, its a really neat NAS with the option for 8 3.5 inch disks + 2 2.5 inch SSDs and the option for one additional nvme. Also added my sfp+ card of course for some speed :D
I love when some of my favorite creators decide to make content that happens to be relevant to me, all at the same time! I'm currently planning a DIY NAS inside a Jonsbo N3 case, which I've already ordered. For the platform, I'm look at using an ASRock IMB-X1231 motherboard paired with an Intel Core i5-14500 (probably). My main reason for this combo is ECC support (I know, it's a polarizing topic lol). I'm still on the fence about installing TrueNAS bare-metal or as a VM inside Proxmox. I want to give Frigate a try using a Coral USB Edge TPU and I'm worried the extra layer is going to make things complicated.
Great content as always, Tim! 😁
You can pas though that coral to a vm, so it wont be an issue
Thank you! Yes, you can pass anything through to a VM. I run ECC in my servers but when it comes to consumer stuff, I don't bother even if it's providing a server role 😅
Where can you find the ASRock IMB-X1231?
I love this application. I am looking to build something similar, although hampered by my low level of software skills. I am hoping to re-use my large Lian-Li tower with fanless PSU and CPU for the perfect homelab - and more space for drives!
I am more confident having seen this video. Thanks for the effort :D
I was renovating a house but because of different reasons I was forced to temporary move into an appartment where I cant have my rack (well, I can't hide it enywhere, wife is not keen on having it in the living room). So this is just perfect timing for a build like this. I will be repurposing Define R5 case from my gaming pc for this build, it gives me a lot of space and is also very quiet. But I'm thinking of using Terra for the new gaming PC.
This is exactly the build I want to do! I can see space for a couple more 3.5 drives too 👀
I've read on the forums, that you should pass through the whole HBA card/sata controller and not the disk by itself. Apparently you lose some SMART functions that are required for ZFS to detect errors and do its self-healing. I've never actually tested this, so just speculating here.
Thanks! Yes, always HBA for sure if you can. I had to do individual disks though because I was using the same controller. 2 for TrueNAS and 2 for Proxmox VMs.
@@TechnoTimTinkers Tim do you have any concern about losing data because you passed through only the drives. If you read anything on the TrueNAS forums they insist this is a formula for disaster. Yet, I have done this exact same thing on my testing machine for a while now and never had a problem. Interested in hearing your point of view on this. Thanks.
Nice build. Been debating building something akin. But would love to include space for 6 + 3.5s maybe 2 2.5 or NVME and room for a 3090. Do all my NAS, AI, plex, etc. on one box and even maybe run a gaming VM on it in down time. i9-9900k still so with that would need room for AIO. Less I upgrade the CPU and explore more cooling options. But for your use case this is not bad at all.
Thanks! I totally know what you mean! Once you start thinking of all of the use cases you start adding more and more. I had to think about this build as if I were 99.9% of the home users who just wanted something good looking, quiet, low power, and would never fill almost 30TB or storage 😂
We need a deep dive on Nano please, sweet pup!
He's such a good boy
I had the same wishes to build in the Terra for my NAS. I love the aesthetic of the Terra, but I will need expandability. My main server has been in a Corsair ATX tower running Proxmox and TrueNAS Scale virtualized. I wanted to separate everything. I went with the Jonsbo N3 and TrueNAS Scale Bare Metal. Jonsbo N3 was the same price as the Terra with expandability for the future. I really didn't want to drop 1k on a 8 bay Synology even though it would have been much less work. The Jonsbo N3 will be solely a NAS and now I'm continuing a separate build for my main server. I went with a Jonsbo C6 for the main server. I enjoy the idea of having everything separate.
Does your Mini ITX motherboard support PCIe Bifurcation? If so, you could make a 'Ultra Fast' storage option using the PCIe 4x4x4x4 Option with (4) NVMe M.2 SSDs.
I really hope Fractal ends up making a Terra+ that can fit at least four 3.5" HDDs. It's such a wonderful design and it'd be perfect for a smaller NAS if it just a had a bit more room.
I hope so too!
interesting experiment anyway that case it's too much small to be used as a NAS. You don't have any space left to mount other hdd.
I will say it. Unraid runs on a usb stick. Then you can use the NVME drive for VM's -- FAST
An ARC A310 seems like a perfect card for that! And a 3d printer 😂
How does setting up the boot drives work? I understand you have two mirrored SSDs, is Proxmox installed on these?
Love the TShirt!
Thanks for the shirt!!!
Got an ncase m2 a few weeks ago, planning a similar build for the next gen intel processors. Time to ditch the raspberry pi.
Have you considered making a DIY JBOD?
@5:10 had same idea for another build but these "simple" peices were being sold on Amazon Canada for $94 for 6 three that idea out the window until i got a buddy with a 3D printer to make a dozen for $10.
you don't need to build out an 800 dollar machine to do this though add some ram to a 9th gen coffee lake box for half the price and have room to expand and add more drives while using more power for a year or two until you build something better - how most people should and will do it - why be extravagant in this economy
Yup. My main Proxmox node I built this year is an Asrock Rack mb and 9700.
Don't you need proper ECC memory for ZFS? I thought standard memory was risky?
Great video, I like the build. Do you have any concern to a virtualizing TrueNas? I have a standalone TrueNas server and then I have another server that I use for virtualization, I went with XCP-NG as my hypervisor. I said it works perfectly fine, but I wonder if I could get a little bit more bang for my buck by consolidating down to one server?
I virtualized TrueNAS for years. I had no issues doing so as long as you pass the controller or individual drives though.
Why do you choose TrueNAS Scale instead of TrueNAS Core if you want to do virtualization and containers via proxmox on a single node host?
Sadly they have effectively discontinued TrueNas Core, it's still going to get security updates for sometime, but that's it, The Register did a good article on this back in march.
@@h.b.5577 Thanks for that information. I ended up using TrueNAS Scale as vm after I tried the TrueNAS lxc unsuccessfully..
Hi there. First of all congratulations for your video. I'm a newbie on this stuff, especially on linux environments, and after tried a few months a working units with Truenas Scale, it seemed too complex to get it working also as a media center using Emby, and also a few other things like qbitorrent, etc. due to the annoying permissions sets... So you triggered me a desire to have a try on this combo Proxmox + Truenas Scale, but i miss some skills on that part of getting the whole hhd drives pass-through available to Truenas. I'm aware that this wasn't the purpose of your video, but can you please help me out? Or maybe have a dedicated video for that? 🙏
Thanks in advance
How much clearance do you have on the CPU cooler side? Could you fit a little taller cooler. Would your same config work in notch 2?
14th gen intel. I see you like to live dangerously.
I promise I updated the firmware and microcode 3 times 😅
How are the Exos disks doing soundwise please? I am building a NAS which will be right next to me and was looking at these disks, but I am afraid they will be quite loud (especially when not idle).
These are pretty quiet if you use some kind of sound dampening like rubber feet or bushings.
More than $1.200 us for a simple NAS? It’s not logic 🤷♂️
Hobby I think 😊
One issue is that your TrueNAS VM can't really utilize Proxmox's extra/unused RAM for the ARC cache. You have to choose a static allocation ratio. Why not just host the ZFS volume(s) from Proxmox directly? For the TrueNAS GUI?
And I've just built a nas with a ryzen 670e motherboard with a 9700x , a really bad idea after undervolting it by 0.1 volts and a maximum 300mv extra undervolting on PBO I can get it down to 50 Watts idle when set to 45 Watts tdp , making my 6000 ram run at full speed using expo uses extra about 10 Watts .
that is with the igpu only and one ssd.
Nice.
How did you manage to get the drives passed directly to truenas. I was under the impression that proxmox always had a layer that hides the bare metal drive and that the only way to directly pass it is with a sad controller. This has been a big pain for me personally given my hardware setup. Would love to know how you do it
If you don’t want to get a separate HBA card, as long as there is a SATA controller that has its own IOMMU group, you can just pass that through to the TrueNAS VM and it should allow TrueNAS to get direct control of the hard drives just like bare metal installations. I have been able to do that on a X99 machine, which has two SATA controllers and one of which has its own IOMMU group. Everything works perfectly in TrueNAS core, including all the disk info, SMART info etc. the only issue I found was the VM will lose control of the SATA controller after resuming from sleep and so far I haven’t find a solution to it. I eventually went back to importing the ZFS array to Proxmox itself and it works perfectly with sleep which allows me to use this machine as a on-demand NAS storage and avoiding the high idle power consumption of X99.
I have also tried what’s shown in this video to pass through the disks to the VM and it seemed to work. Everything appears normal except for one thing that is I cannot get the correct disk info, such as serial number, temperature and SMART info, which makes me feel shaky about it and went the other way around.
@@cyberbass4 o wow nice. Thanks for that. Might give it a try. Unfortunately my case can’t use an hba card because of the custom backplane of the dell workstation
@@Ezza232499 Yeah. Hope you are lucky enough to have such a SATA controller on your motherboard which can save you a lot of trouble. It is actually easier than passing through individual disks and all ZFS related stuff works perfectly, except for the sleep part.
Whats the best way to share the TrueNAS VM storage with the different LCX containers like Proxmox/Jellyfin in this situation? A share in TrueNAS?
That's the route I would go. Create an NFS share and map it.
If your mounting a network share from a TrueNAS VM to the host proxmox. Be aware that you will probably have to manually start and LXC or VM that uses it. As even if you start the TrueNAS VM first. The share won't be ready in time, and the container/VM won't start. How I get around this is running the containers from a VM using autofs or creating the volumes from NFS shares, and forcing (in my case docker) to wait for the mount before starting.
how big should a boot ssd be
Sweet case. $1,281.40 if anyone is wondering.
Jonsbo N1
Is the power consumption with the hdd's running or spun down?
Thank you! Do you think you could build this in the Era 2 and fit 4 3.5 hdd inside?
Good question, not sure. It's a great looking case too! I would just look at the specs and see how much wdith you have next to the PCIe card slot and then see if there is enough depth and width. It's tough to know for sure which is why I went out on a limb when I built this.
@@TechnoTimTinkers I appreciate it. New to servers and home labs. Trying to figure out storage for my photos as a photographer and dad. As well as a media collector
Is it safe to have zfs without ecc memory ?
Yeah as the conclusion at the end of the video, the case is the limiting factor plus the motherboard. Bad choice.
are we trending back to blacked out sff systems again? no rgb 4 life
Hopefully!
@@wojtek-33 It was fun while it lasted, back to RGB! 😅
two editional fans in my livingroom is not an option for me
Using truenas in proxmox without passing disk controller is a bad idea
no ecc :(
Why such an expensive motherboard, when using the cheapest CPU? Or did you get a good deal on it or something?
It was on sale but I went this route because I wanted DDR5 and Mini ITX.
you gotta noc TUA and black out that thang
I think it’s time for you to buy a 3d printer , design custom brackets for hdd’s
am I dumb? How are you using so many vms with a processor only having 4 cores?
CPU over subscription. Pretty safe to do. Memory over subscription on the other hand is not good.
I also have coupled Proxmox and TrueNAS together and just love the benefits (console, VM snapshots, backups). Especially migrating went flawlessly. However, to have disk spindown and SMART I bought a PCIe SATA controller and passed that one through. Only issue I had was that I had to tell Proxmox to NOT try to import the TrueNAS rpool on bootup.
For my Linux ISO ..... 🤣🤣🤣👍
You stole my thermal paste 😠😠
Or just get a Jonsbo n2….