We have 20Mbs/10Mbs cable connection for around 25-30$ for last at least ten years, before it was 10/1 for the same price for ten years. THANK GOD FOR O2 !!! They slowed my life so much
7:26: Just to note: the eBay search results shown there are for the Dell rNDC card, not the stand-alone PCIe version. Unless you have a Dell server with a compatible onboard networking port, don't buy the parts shown on screen here.
Regarding the AT&T BGW320-500/505 we have found a way around not using it at all on AT&T's network. It involves using an external XGS-PON ONT which has a 10G ethernet port. Availability is abysmal and it's been a community effort figuring the device out (big ups to the few people who got deep into it and figured it out), the manufacture either won't or doesn't respond to emails thus far. Thanks for the video Wendell.
Definitely going to be hunting for that on the forums, currently still have the BGW210 and getting it sorted to bypass at least that unit, but being able to pass the ONT too would be awesome if the same approach works, though admittedly less necessary as the BGW210 is the main source of my issues. Thanks for letting us know the options exist :D
Some things to add, the X550 does better at longer cable runs than the aquantia chipset. Also there are router OSes made specifically with low latency in mind like VyOS or tnsr (from netgate the makers of pfsense).
@@mitch7918 vyos is targeted at enterprises and carriers. Those have staff that are comfortable with CLI. The CLI on vyos is pretty easy to work with once you understand the syntaxes and configuration modes. It does struggle with hairpin NAT, so use split DNS for that.
2 months ago, and that specific motherboard is nowhere to be found...any w680 board is $500 I think? that's what my searches have found. What alternatives do you have for that board? how budget are we talking here?
Jeez, keep it up. The tidbit about pcore and ecore on linux vs freebsd was a good nugget of info. Thanks to you and the commenters. The comments about these horrible att routers was great too. Led me down a path. Maybe I can join and figure it out. I do not know how youtube bullseyed this one but I'm here.
Been using a ryzen 3200g in a router box since late 2019. Works fine with 3gbs fiber. For anything faster than 3gbs, you likely need openwrt as I don't think the freebsd can do 10gbs yet - unless they fixed it.
I have a Dell R420 w/ Dual E5-2430 w/ 32GB ECC RAM and X520 2xSFP+ card booting pfsense off a SATA SSD. It has no problem with 2Gbit internet and even tested 10Gbit with it (using another machine acting as my WAN connection). I'm into it for maybe $300 and it is absolutely solid as a rock. I'll trust enterprise hardware a lot more than consumer grade hardware. Am I missing something?
Sliger is a really fantastic company to work with. I had the opportunity to work with Kahlin Sliger in designing a chassis for a medical instrument. The company was so great to work with commercially, that I only ever recommend them personally.
7:30 I had no idea there was a Firmware update to the X550, that “unlocks” 2 & 5gbe. I’ve been hunting for the 700series that they made that supports this. Makes a great bargain, even better. 👍
I get about ~2.5 Gbps L3 performance using OPNsense with 4 cores of a 2680 v4 under ESXi. I tried with 8 cores and it scaled linearly so I'm not convinced you need hardware as good as what's on display for consumer 2.5 or 5 Gbps services unless you're going to run something like Wireguard.
I'm rockin an old Dell PowerEdge R210 II running pfsense. I slapped in a Realtek quad 2.5G network card (I know eww realtek) but it actually works fine if you install the proper drivers. May be upgrading to a R330 soonish. Love your videos, you literally do things I find interesting and I may even go to school for networking next year
this forbidden router makes so much sense in a lot of homelab setups, it's simple, a proper machine running a VM that contains your software router, and has proper hardware to connect to your switch. You can add dns, proxy, nginx, vpn, and everything network on the server, and even Firewall rules! If you don't need 10G you can get a 4 port 2.5G ethernet card for cheap and maybe even ditch the expensive managed switch all together! Assigning vlans for each port and rules to connect everything together. All at a low price and with the benefits of a VM!
Wait, that's a novel idea (to me). My switching needs are minor for non poe. Could I get 2 or 3 2.5g NICs to work like this? Assuming chassis supported it.
@@badharrow as long as the motherboard supports giving the PCIe lanes to the VM that's possible! I do something very similar and works wonders, no need for expensive switches here.
I've been working on a home router project over the last year, got a supermicro 1u 5019D-4C-FN8TP with xeon D-2123IT. Setup Proxmox and created a virtualized PFsense. My biggest issues were the 2.5/5/10gb negotiations. My modem has 2.5gb but had trouble getting things to work, so tried passing through the NICs to Pfsense but found out BSD has trouble with realtek drivers. Got things sorted out now using an intel i225 nic but virtualized and passed to the VM using virtio. My connection is 1.2gb but I can get a consistent 1400mbs out of it and iperf gives me stable 10gb between my homelab servers. I have the t540 nic in my desktop, if I had known about the t550 and firmware to get 2.5/5 that would have made my life alot easier. Thanks for the video!
Also have the 540. Didn't realize until this video it doesn't negotiate 2.5/5. Had I paid attention to that detail a couple of years ago I would have saved myself a whole lot of stress. Looking into 550 options now, that 5 second comment in the video just explained so much for the problems I've been having.
@@beepboopbeepboop190 I ended up getting a TEG-S762 switch for my office to connect to my desktop. Has 2x10gb ports so I don't have to worry about negotiation. 10gb to desktop then 10gb uplink to my homelab, and it has 4x2.5gb ports as well.
@@krj15489 That's a good call. Mine is installed in my firewall/router so one is connected to a 10G switch on the lan side but the other port is connected to my ISP's terminal. I'm 99% sure that's not 10G but I think it might be more than 1G. The nic on the fw's motherboard is 2.5 though so I may just swap to that for the wan.
I’m from Chile, here I have 2 isp connections, one at 800 Mbps and another at 600 Mbps. I can upgrade one of the isp connetion to 2 Gbps, and for that reason I’m building a custom router with one of those super micro motherboard with 6x10G ports that you can find on ebay.
I wouldn't mind going in on a group buy for the motherboard. I've got quite a few alder lake cpus from XOC I was planning on using to replace my current homelab system with (3970x, 256gb ram). These would fit the bill quite well.
No SR-IOV on Aquantia. :( RE: P-cores and E-cores, I think the way to go is pinning VMs to cores and using isolcpus or cpusets to keep the hypervisor from using them.
Gotta be nice to have an Internet connection to do your router justice. :) My own network uses 1000 base T/ Gigabit ethernet. The bottleneck is the cable modem. For the router, I use the Linksys WRT1200AC with the OpenWRT on it. If you can upgrade a consumer router with OpenWRT you won't be disappointed. You can then use NFS and have it also be a light show machine - AND a router!
Hey Wendell, Attempted to go out and find these components and had trouble finding where to buy the motherboard. It appears no one sells it standalone. I went through the gigabyte sites resellers and distributors list and found no motherboards with any of them.
Please do the kickstarter. As I commented last time you did a video on this board, I would buy one right now for almost any cost. It seems like a much better choice then starting a platform on the ASROCK rack x570 boards. Gigabyte needs to direct sell these things to kick up competition in this space.
Great video, Wendell, it really got me thinking about a new router build down the road. AT&T Fiber sounds... less than ideal. We have CenturyLink here (1Gb) and I just hooked up my homebrew opnsense box in place of their Zyxel C4000 modem/router and we were in business. Having to work around subpar ISP hardware is never fun. Cheers!
Just put in 8 Gig internet. I have the equipment to handle it, but I have opposite problems with some of my machines. Download and uploads are not symmetrical, both ways in different machines. I do need to look into building a router, since I have an Asus RT-AX89X that does handle 10 gig, but maybe not well? Your videos are the best!
Hey Wendell, thanks for the great video! I was wondering how would a mikrotik routeros compare to the pfsense/opnsense, since there was a debate that pfsense could have problems pushing 10gbit. rOS level 4 license is around $45 and it comes with everything a home router would need, so kinda curious how would it stack against pfsense in some higher speed routing.
If most of your traffic is GSO/GRO friendly (which is usually the case if you are not like a ISP or data center) even a Atom C3xxx will do 8+Gbps *per core* with routing, firewalling and NAT under plain Linux networking (at least on bare metal.) Of course if you need to do something requiring traffic inspection such weak cores could become very painful very quick, or if you are routing a metric crap-ton of non-TCP traffic (no GRO/GSO, but there was some work to fix that recently at least for UDP, wonder if it has fully landed and enabled for routing yet) Still, perf/$ is prob way better on a W680/Alder lake platform, at least the more sensible SKUs, the headroom can come in handy, and the idle wattage - where its probably going to spend most of its life anyway - does seem pretty decent - go for it :)
I know this is old, but your video is relevant. You can pickup tons of power edge servers right now for around $500 with quad 10gbe a raid controller drac etc and quad xeons. I just picked one up with quad v3's for $300 on Marketplace.
This is my current build project at the moment. Never realized how underpowered consumer routers were. Always went with custom firmware on mine and got the functionality I wanted. Then I learned of how much more potential there is building one yourself. I also plan to build a plex streaming server. Would it be advisable to put in a separate machine or would it be ok combined with the router? I already have it on my NAS server (readynas) and perf sucks when transcoding is needed.
I had plex running on my Synology NAS and it had a tough time transcoding big files as well. Build a media server with a AMD 2700x and it's been a champ. I'm about to drop. 5900x in that box and spin up another VM for this project :) But yah, plex running on a PC is flawless compared to a consumer NAS. Still love my Synology though.
The "forbidden" router: running the router software as a VM while a hypervisor like XCPng actually runs on the bare metal. This would allow you to create another VM for your Plex. However, this configuration is "forbidden" because it has lots of potential for weirdness when the routing software is virtualized.
Good cheap case option for all purpose home server is definitely has to be the node 804. $100 bucks and you can slap 8 3.5 hdd's and a bunch of ssd's. Supports micro-atx boards. virtualize your nas, router, servers do it all in one box.
I bought this same case, the only issue I had with it is the front water cooling IS NOT compatible with custom water loop blocks. It says it supports water cooling but it has to be an AIO that doesn't have the bottom outputs for tubing. My corsair XR7 only fit one way and I had to take out the front USB panel. I don't care since I used a pikvm to control it. And of course you don't need to water cool your router.
I have 2 kids and a wife all using our home router and I'm trying to use my Flex remote Ham radio. Would this help? Beacuse I'm bored of packet loss on my radios audio stream..
A lot of useful information here as always. I would be interested in hearing what other homelab users are doing with > 1Gbe ISP connections (webscraping? actual commercial/public facing websites? etc). For my work almost everything over >1Gbe is constrained to my local network(NAS related). That said it would be interesting to see either a bifurcated or low-idle power solution that could be left on 24x7(e.g. I don't want a 100-300 watt router running a 10pm at night just to watch netflix or browser the web)
Interesting video, inspiring for our Home Labs. How does it handle VPN? Is there an Intel QuickAssist card you can recommend to improve performance? Would you be interested in doing a shootout between a Linux based routing solution versus BSD? For example IPFire vs pfSense; both bare-metal and via Proxmox.
Would a new M2 Mac Mini with 10GIG and a few Thunderbolt to 10Gig adapters / SFP adapters work better? Seems like that's how I would go with it. Its small. Sips power. It does run UNIX flavor. Seems Like that's how I would go with it. Its also SMALL.
I wouldn’t bother, macOS is not made to be a router or firewall. And Linux on M2 is likely not there yet, especially with Thunderbolt support. If you want less tinkering, the Minisforum MS-01 is probably the better fit. And it’s around the same price as the Mac Mini M2, it also has built in SFP+ ports (Intel X710).
I'm planning my first storage server right now mostly focused on expandability since I don't really know how much I have to store. Thanks for your videos on the topic. I'm still worried I'm missing basic things. DIY router is a very interesting topic. I'm wondering if it would make sense to build a forbidden router into the storage server? I was thinking it would be nice to be able to support 40gbps eventually like thunderbolt in various places. Making the server at least as good as direct storage in that regard. Is that practical? I've been thinking used epyc so far but that may not work well for high speed routing.
I love these types of build your own solutions but obtaining a good OS with the features I desire has always shifted me away. After trying pre-build enterprise solutions I shifted to building with Sophos XG, PFsense, Open Sense and others I landed on just purchasing a Firewalla Gold. The price, hardware and most of all the software experience all with my current needs. I hope you review it one day.
I second this I have a Firewalla purple and just ordered the new gold plus after running pfsense for a few years and trying opnsense for half a year I think Firewalla is the perfect balance of being able to tinker with low energy use hardware and solid stability.
I’m just waiting for my first 1gbps Wi-Fi install because I’ve only had 40mbps down and 5mbps upload speed for 20 or so years since I was a kid here in UK.. Wi-Fi cuts out so much and when I eventually build a computer I’m going to invest in a better router for all my streaming box’s and any other stuff I get.. Would you recommend buying a after market router or build one for 1gbps?
I would love a couple of these to throw in my home lab at home here is Australia!! I would go in for the kickstarter group buy that Wendell suggested!!! Does anyone know where i could buy this in Australia??
I'd love to see some comparison between sth like Vyos and Opnsense/pfsense. Would love to know how big the routing difference really is nowadays since it has been pretty big in the past, especially in virtualized environments
Or alternatively as cheaper solutions, you buy an old refurbished i3/i5/i7 (or Ryzen) computer adding a high-speed multi-port NIC card or one of the bevy of firewall appliance mini-computers on Amazon for a couple of hundred bucks (or upcycle an old computer with a decent processor).
If the case is the least expensive component, why is the msrp of that motherboard well over 600 bucks. Also that motherboard is discontinued, and I cannot find one less than 900 bucks? Budget build?? I can afford the cpu at 160 bucks, but holy shnikies????
How does the hypervisor handle the E-cores? I'd think the smartest thing it could do is given each VM a perportional share of the P-core and E-cores access according the number of VCPUs assigned. I doubt any are that smart though. Last thing you'd want is VMs randomly getting all E-cores.
When they say the x540 can not do 2.5 or 5gbe it is technically true but if connected to another 10gbe component depending on drive speed it will do all of the range from .1 to 10gbe
This is basically how I started my internet company in the 1990's, T-1, T-3, and serial multiport cards... Voila, internet company, initially started on a 486 dx2 back then... LOL 😂
I am using a i3 10100 k cpu and pfsense. Might upgrade my nic down the line to a 10gug. But right now I get like 226mb down and 10mb up. Very efficient. Just need to figure out what to do w my wifi router or just switch to ap mode or buy a dedicated AOm
Can you set the bios to have this boot on power loss? You want your router auto rebooting always. It would suck to have to go hit the power button or log in to ipmi, or worse not know you lost power because you’re away and have to wait until you come home to power up your network!
What do you think of the mikrotik ccr2004-something? Basically a router with 2 25g sfp interfaces stuck on a pcie card that exposes another 2 interfaces over said pcie. Stick it in your NAS, hook it up to a switch and job done?
Regarding the X550-T2. Intel's product page claims that 2gb and 5gb are only supported in Linux. Since pfsense is freebsd based doesn't that mean x550 would only negotiate at 1gb or 10gb in pfsense?
I just got a internet upgrade from 1gbps to Fiberlink 2.5 Gbps. My motherboard has a 2.5G LAN port and they gave me a Zyxel AX7501 router.If I pay a one time fee of 100€ I get 10 gbps internet for the same monthly subscription. What do you think,I have a 5800x3d on B550 TOMAHAWK and the only motherboard I found with a 10g port is the X570 creation but used..I don't want to change the processor or the platform... is it worth changing the motherboard or getting a separate network card with 10g network port?
While it's good to see 2.5G ports becoming the norm now, 10GbE ports in motherboards need to really catch up, especially high-end ITX boards. I tried to find an x570 motherboard with 10GbE onboard and the only one I considered was the $1000 Gigabyte board, which I did not get. For now I'd recommend to just buy a plug-in 10GbE card. I have the Asus XG-C100C that I got for $100 USD that works well in my X570 board.
i have laying around a 3200g , B550 2x 4gb 2400 ddr4 .... and 2x Intel PRO 1000 Quad Port which i want to trade it fo Low Profile but 2 or more years ago none of the Router OS Supported WiFi 6 and i didnt have a Low Profile Quad port Ethernet
How necessary is ECC memory for a RAID NAS? I'd like to build a new machine to replace my current one, and wondered if its worth the money to get a motherboard like this one so I have error correction.
One thing I'm considering getting as a router + VM host is a Deciso DEC740. It's a GPU-less AMD Ryzen Embedded V1500B, in a small passive chassis. How does the CPU horsepower in one of those compare to the Alder Lake, for network tasks? I do have my eye on W680 stuff for desktop, though.
+1, I actually emailed their sales channel (through the MW34-SP0 page on their site), but didn't get a response. (To be fair, I'm not representing an enterprise or a system integrator, so it doesn't surprise me too much, but I was hoping....)
@level1techs, Is Intel VPro feature a recommended thing to have for pfsense/opnsense build or not? I searched and saw some people like to have it and others wanted to disable or totally avoid it.
One other thing that I would suggest is that an AMD 7600 using PCIe 5 interfaces should yield better results especially locked all core clocks with a Linux or similar OS stripped to its bare bones.
I would really be interested in the performance of Clear Linux on that kind of platform, since it's curated by Intel, directly. I'm using it as a general purpose hypervisor and docker host on a mere 10W TDP Intel SoC and NICs on Supermicro X11SBA-LN4F. Very nice board for a 100/30M VDSL connection which is all I'm getting in Germany, unfortunately. I run 2 VMs on that with Sophos XG and 3CX. In Docker I'm running Home Assistant and a Unifi WiFi controller. With all that, the CPU is running 30-40% usage. The main constraint of this machine is RAM.
Was there anything done in the Tunables to get greater than 10Gb throuput? I just built a host with the 12400 running OPNsense bare metal. Iperf running on the LAN interface. Only getting 7.8 Gb test results without running multiple streams.
we have 25 Gbit/s Residential in switzerland for 65$ a month but no routers to handle it 🤣🤣
Meanwhile, I’m on about 24mbps/8mbps in Australia in a city 😂
wow, this is stupidity fast for home use! can DIY a x86 router to chug this bandwidth
vyos
We have 20Mbs/10Mbs cable connection for around 25-30$ for last at least ten years, before it was 10/1 for the same price for ten years.
THANK GOD FOR O2 !!! They slowed my life so much
Fiber 7? right
7:26: Just to note: the eBay search results shown there are for the Dell rNDC card, not the stand-alone PCIe version. Unless you have a Dell server with a compatible onboard networking port, don't buy the parts shown on screen here.
Regarding the AT&T BGW320-500/505 we have found a way around not using it at all on AT&T's network. It involves using an external XGS-PON ONT which has a 10G ethernet port. Availability is abysmal and it's been a community effort figuring the device out (big ups to the few people who got deep into it and figured it out), the manufacture either won't or doesn't respond to emails thus far. Thanks for the video Wendell.
where is this being tracked? just signed onto ATT Fiber 1gig
Deets and link to buy? Lol
@@Level1Techs I attempted to reply but I'm not sure if the comment got ate by TH-cam or deleted, didn't post a link but described where it was.
Don't see it even in held. Email me at Wendell at level1techs dot com and I'll make a forum post for it
Definitely going to be hunting for that on the forums, currently still have the BGW210 and getting it sorted to bypass at least that unit, but being able to pass the ONT too would be awesome if the same approach works, though admittedly less necessary as the BGW210 is the main source of my issues. Thanks for letting us know the options exist :D
Some things to add, the X550 does better at longer cable runs than the aquantia chipset. Also there are router OSes made specifically with low latency in mind like VyOS or tnsr (from netgate the makers of pfsense).
Interesting. The aquantia has been fairly flakey in my experience. Maybe I should try a shorter cable run.
Wish VyOS had a web gui, sticking with OPNSense
@@mitch7918 vyos is targeted at enterprises and carriers. Those have staff that are comfortable with CLI.
The CLI on vyos is pretty easy to work with once you understand the syntaxes and configuration modes. It does struggle with hairpin NAT, so use split DNS for that.
What are you talking about VyOS and low-latency? I fail to see what VyOS has that OpenWRT/OPNsense don't.
@@mitch7918 I'd kill for a GUI for VyOS
Omg omg omg omg! I’ve been waiting for this. In my city they are upgrading from 10Gbps to 25Gbps on SFP28
what city is this?
2 months ago, and that specific motherboard is nowhere to be found...any w680 board is $500 I think? that's what my searches have found. What alternatives do you have for that board? how budget are we talking here?
Jeez, keep it up. The tidbit about pcore and ecore on linux vs freebsd was a good nugget of info. Thanks to you and the commenters. The comments about these horrible att routers was great too. Led me down a path. Maybe I can join and figure it out. I do not know how youtube bullseyed this one but I'm here.
Been using a ryzen 3200g in a router box since late 2019. Works fine with 3gbs fiber. For anything faster than 3gbs, you likely need openwrt as I don't think the freebsd can do 10gbs yet - unless they fixed it.
A group buy is an interesting idea. I'd do it.
I have a Dell R420 w/ Dual E5-2430 w/ 32GB ECC RAM and X520 2xSFP+ card booting pfsense off a SATA SSD. It has no problem with 2Gbit internet and even tested 10Gbit with it (using another machine acting as my WAN connection). I'm into it for maybe $300 and it is absolutely solid as a rock. I'll trust enterprise hardware a lot more than consumer grade hardware. Am I missing something?
Wendell and team are like the deep dive tech friends I never had! Thanks for being you!
Sliger is a really fantastic company to work with. I had the opportunity to work with Kahlin Sliger in designing a chassis for a medical instrument. The company was so great to work with commercially, that I only ever recommend them personally.
7:30 I had no idea there was a Firmware update to the X550, that “unlocks” 2 & 5gbe. I’ve been hunting for the 700series that they made that supports this. Makes a great bargain, even better. 👍
I just updated the firmware on my x550 and if I plug it into a 2.5g switch port it drops to 1g.
@@grizredford8407 I wonder if there are sequential firmware updates. Perhaps still needs to be manually negotiated.
I get about ~2.5 Gbps L3 performance using OPNsense with 4 cores of a 2680 v4 under ESXi. I tried with 8 cores and it scaled linearly so I'm not convinced you need hardware as good as what's on display for consumer 2.5 or 5 Gbps services unless you're going to run something like Wireguard.
I'm rockin an old Dell PowerEdge R210 II running pfsense. I slapped in a Realtek quad 2.5G network card (I know eww realtek) but it actually works fine if you install the proper drivers. May be upgrading to a R330 soonish. Love your videos, you literally do things I find interesting and I may even go to school for networking next year
I've found the Wiitek SFP+ to RJ45 copper modules will negotiate 1/2.5/5/10 for cheaper SFP+ network cards and things like the UDM Pro.
Just got an Intel X520-T2 and it pushes the two 10G connections just fine.
THIS is the content we need more of. Thank you so much!
this forbidden router makes so much sense in a lot of homelab setups, it's simple, a proper machine running a VM that contains your software router, and has proper hardware to connect to your switch. You can add dns, proxy, nginx, vpn, and everything network on the server, and even Firewall rules! If you don't need 10G you can get a 4 port 2.5G ethernet card for cheap and maybe even ditch the expensive managed switch all together! Assigning vlans for each port and rules to connect everything together. All at a low price and with the benefits of a VM!
Wait, that's a novel idea (to me). My switching needs are minor for non poe. Could I get 2 or 3 2.5g NICs to work like this? Assuming chassis supported it.
@@badharrow as long as the motherboard supports giving the PCIe lanes to the VM that's possible! I do something very similar and works wonders, no need for expensive switches here.
I've been working on a home router project over the last year, got a supermicro 1u 5019D-4C-FN8TP with xeon D-2123IT. Setup Proxmox and created a virtualized PFsense. My biggest issues were the 2.5/5/10gb negotiations. My modem has 2.5gb but had trouble getting things to work, so tried passing through the NICs to Pfsense but found out BSD has trouble with realtek drivers. Got things sorted out now using an intel i225 nic but virtualized and passed to the VM using virtio. My connection is 1.2gb but I can get a consistent 1400mbs out of it and iperf gives me stable 10gb between my homelab servers. I have the t540 nic in my desktop, if I had known about the t550 and firmware to get 2.5/5 that would have made my life alot easier. Thanks for the video!
Also have the 540. Didn't realize until this video it doesn't negotiate 2.5/5. Had I paid attention to that detail a couple of years ago I would have saved myself a whole lot of stress. Looking into 550 options now, that 5 second comment in the video just explained so much for the problems I've been having.
@@beepboopbeepboop190 I ended up getting a TEG-S762 switch for my office to connect to my desktop. Has 2x10gb ports so I don't have to worry about negotiation. 10gb to desktop then 10gb uplink to my homelab, and it has 4x2.5gb ports as well.
@@krj15489 That's a good call. Mine is installed in my firewall/router so one is connected to a 10G switch on the lan side but the other port is connected to my ISP's terminal. I'm 99% sure that's not 10G but I think it might be more than 1G. The nic on the fw's motherboard is 2.5 though so I may just swap to that for the wan.
I’m from Chile, here I have 2 isp connections, one at 800 Mbps and another at 600 Mbps. I can upgrade one of the isp connetion to 2 Gbps, and for that reason I’m building a custom router with one of those super micro motherboard with 6x10G ports that you can find on ebay.
I wouldn't mind going in on a group buy for the motherboard. I've got quite a few alder lake cpus from XOC I was planning on using to replace my current homelab system with (3970x, 256gb ram). These would fit the bill quite well.
No SR-IOV on Aquantia. :(
RE: P-cores and E-cores, I think the way to go is pinning VMs to cores and using isolcpus or cpusets to keep the hypervisor from using them.
So it is forbidden after all
Gotta be nice to have an Internet connection to do your router justice. :) My own network uses 1000 base T/ Gigabit ethernet. The bottleneck is the cable modem. For the router, I use the Linksys WRT1200AC with the OpenWRT on it. If you can upgrade a consumer router with OpenWRT you won't be disappointed. You can then use NFS and have it also be a light show machine - AND a router!
Definitely would be interested in a group buy on this setup.
I don't know where your intro music comes from, but it is consistently among the best for any YT channel I know.
Hey Wendell, Attempted to go out and find these components and had trouble finding where to buy the motherboard. It appears no one sells it standalone. I went through the gigabyte sites resellers and distributors list and found no motherboards with any of them.
Please do the kickstarter. As I commented last time you did a video on this board, I would buy one right now for almost any cost. It seems like a much better choice then starting a platform on the ASROCK rack x570 boards. Gigabyte needs to direct sell these things to kick up competition in this space.
It seems AsRock Rack are also getting in W680 boards now. But I don't think they're selling them yet, they're listed as "preliminary"?
Great video, Wendell, it really got me thinking about a new router build down the road. AT&T Fiber sounds... less than ideal. We have CenturyLink here (1Gb) and I just hooked up my homebrew opnsense box in place of their Zyxel C4000 modem/router and we were in business. Having to work around subpar ISP hardware is never fun. Cheers!
I like the 4x NVMe and 8x SATA ports on that board. I may actually buy one as an "everything" box.
Imagine in the next 2-3 years when ALL consumer motherboards will have this level of IO.
So exciting, it's like a glimpse of the future :D
Just put in 8 Gig internet. I have the equipment to handle it, but I have opposite problems with some of my machines. Download and uploads are not symmetrical, both ways in different machines. I do need to look into building a router, since I have an Asus RT-AX89X that does handle 10 gig, but maybe not well? Your videos are the best!
Great video,. especially the coverage of the nuances of the 10GbE card.
Hey Wendell, thanks for the great video! I was wondering how would a mikrotik routeros compare to the pfsense/opnsense, since there was a debate that pfsense could have problems pushing 10gbit. rOS level 4 license is around $45 and it comes with everything a home router would need, so kinda curious how would it stack against pfsense in some higher speed routing.
If most of your traffic is GSO/GRO friendly (which is usually the case if you are not like a ISP or data center) even a Atom C3xxx will do 8+Gbps *per core* with routing, firewalling and NAT under plain Linux networking (at least on bare metal.) Of course if you need to do something requiring traffic inspection such weak cores could become very painful very quick, or if you are routing a metric crap-ton of non-TCP traffic (no GRO/GSO, but there was some work to fix that recently at least for UDP, wonder if it has fully landed and enabled for routing yet)
Still, perf/$ is prob way better on a W680/Alder lake platform, at least the more sensible SKUs, the headroom can come in handy, and the idle wattage - where its probably going to spend most of its life anyway - does seem pretty decent - go for it :)
You should look into VPP (w/Linux Control Plane plugin) + DPDK + FRR.
I use the X540-AT2 in my main home workstation and home servers. I use Trendnet TEG-10GECTX and Asus XG-C100C in my other rigs
I know this is old, but your video is relevant. You can pickup tons of power edge servers right now for around $500 with quad 10gbe a raid controller drac etc and quad xeons. I just picked one up with quad v3's for $300 on Marketplace.
My 50mbit connection would like to have a word with this 2,5gbit internet connection out of dreamland.
This is my current build project at the moment. Never realized how underpowered consumer routers were. Always went with custom firmware on mine and got the functionality I wanted. Then I learned of how much more potential there is building one yourself.
I also plan to build a plex streaming server. Would it be advisable to put in a separate machine or would it be ok combined with the router? I already have it on my NAS server (readynas) and perf sucks when transcoding is needed.
I had plex running on my Synology NAS and it had a tough time transcoding big files as well. Build a media server with a AMD 2700x and it's been a champ. I'm about to drop. 5900x in that box and spin up another VM for this project :)
But yah, plex running on a PC is flawless compared to a consumer NAS. Still love my Synology though.
The "forbidden" router: running the router software as a VM while a hypervisor like XCPng actually runs on the bare metal. This would allow you to create another VM for your Plex. However, this configuration is "forbidden" because it has lots of potential for weirdness when the routing software is virtualized.
Good cheap case option for all purpose home server is definitely has to be the node 804. $100 bucks and you can slap 8 3.5 hdd's and a bunch of ssd's. Supports micro-atx boards. virtualize your nas, router, servers do it all in one box.
The group buy for the motherboard is a good idea. I’d consider buying a new router for my University’s ACM chapter if it went through.
I bought this same case, the only issue I had with it is the front water cooling IS NOT compatible with custom water loop blocks. It says it supports water cooling but it has to be an AIO that doesn't have the bottom outputs for tubing. My corsair XR7 only fit one way and I had to take out the front USB panel. I don't care since I used a pikvm to control it. And of course you don't need to water cool your router.
I went for a Topton N305 mini pc, passive cooling, 32GB ram, 2TB nvme, 4 x 2.5Gbit ports, low idle power as seen on STH ;) (obviously no 10Gb though)
I have 2 kids and a wife all using our home router and I'm trying to use my Flex remote Ham radio. Would this help? Beacuse I'm bored of packet loss on my radios audio stream..
A lot of useful information here as always. I would be interested in hearing what other homelab users are doing with > 1Gbe ISP connections (webscraping? actual commercial/public facing websites? etc). For my work almost everything over >1Gbe is constrained to my local network(NAS related). That said it would be interesting to see either a bifurcated or low-idle power solution that could be left on 24x7(e.g. I don't want a 100-300 watt router running a 10pm at night just to watch netflix or browser the web)
I just have it because I can. 5gbps fiber is cheaper from my provider than 100mbps from Spectrum at $60/month.
Interesting video, inspiring for our Home Labs.
How does it handle VPN? Is there an Intel QuickAssist card you can recommend to improve performance?
Would you be interested in doing a shootout between a Linux based routing solution versus BSD? For example IPFire vs pfSense; both bare-metal and via Proxmox.
Group buy for the MW34-SP0 would be great!
Would a new M2 Mac Mini with 10GIG and a few Thunderbolt to 10Gig adapters / SFP adapters work better? Seems like that's how I would go with it. Its small. Sips power. It does run UNIX flavor. Seems Like that's how I would go with it. Its also SMALL.
I wouldn’t bother, macOS is not made to be a router or firewall. And Linux on M2 is likely not there yet, especially with Thunderbolt support.
If you want less tinkering, the Minisforum MS-01 is probably the better fit. And it’s around the same price as the Mac Mini M2, it also has built in SFP+ ports (Intel X710).
Ive been waiting for a video like this, some areas in the UK have access to 3Gbps down
I'm planning my first storage server right now mostly focused on expandability since I don't really know how much I have to store. Thanks for your videos on the topic. I'm still worried I'm missing basic things. DIY router is a very interesting topic. I'm wondering if it would make sense to build a forbidden router into the storage server? I was thinking it would be nice to be able to support 40gbps eventually like thunderbolt in various places. Making the server at least as good as direct storage in that regard. Is that practical? I've been thinking used epyc so far but that may not work well for high speed routing.
It would be cool to have a short or some short video updating this video.
I love these types of build your own solutions but obtaining a good OS with the features I desire has always shifted me away. After trying pre-build enterprise solutions I shifted to building with Sophos XG, PFsense, Open Sense and others I landed on just purchasing a Firewalla Gold. The price, hardware and most of all the software experience all with my current needs. I hope you review it one day.
I second this I have a Firewalla purple and just ordered the new gold plus after running pfsense for a few years and trying opnsense for half a year I think Firewalla is the perfect balance of being able to tinker with low energy use hardware and solid stability.
Yeah I have that exact spectrum router that I have to use for my static ip’s, so it’s modem -> spectrum router -> pfsense
I’m just waiting for my first 1gbps Wi-Fi install because I’ve only had 40mbps down and 5mbps upload speed for 20 or so years since I was a kid here in UK.. Wi-Fi cuts out so much and when I eventually build a computer I’m going to invest in a better router for all my streaming box’s and any other stuff I get..
Would you recommend buying a after market router or build one for 1gbps?
I would love a couple of these to throw in my home lab at home here is Australia!! I would go in for the kickstarter group buy that Wendell suggested!!!
Does anyone know where i could buy this in Australia??
I'd love to see some comparison between sth like Vyos and Opnsense/pfsense. Would love to know how big the routing difference really is nowadays since it has been pretty big in the past, especially in virtualized environments
Dudeeeee, the 13900 on this motherboard would be a nutty home server! VMs for days!
Once again recommended the 12500 because it actually supports ECC and is only a few bucks more.
i really do like these builds just DIY everything
have you done some openwrt?
I wonder do you need this much performance for a over 1 Gigabit router?
Would you say the the processor and ram are more responsible for your performance and if so what other motherboard would you recommend.
Or alternatively as cheaper solutions, you buy an old refurbished i3/i5/i7 (or Ryzen) computer adding a high-speed multi-port NIC card or one of the bevy of firewall appliance mini-computers on Amazon for a couple of hundred bucks (or upcycle an old computer with a decent processor).
If the case is the least expensive component, why is the msrp of that motherboard well over 600 bucks. Also that motherboard is discontinued, and I cannot find one less than 900 bucks? Budget build?? I can afford the cpu at 160 bucks, but holy shnikies????
How does the hypervisor handle the E-cores? I'd think the smartest thing it could do is given each VM a perportional share of the P-core and E-cores access according the number of VCPUs assigned. I doubt any are that smart though. Last thing you'd want is VMs randomly getting all E-cores.
I think the only usecase for E cores is to help with an IPS/IDS like suricata
That’s scary… maybe less demanding stuff like dhcp, dns services
When they say the x540 can not do 2.5 or 5gbe it is technically true but if connected to another 10gbe component depending on drive speed it will do all of the range from .1 to 10gbe
This is basically how I started my internet company in the 1990's, T-1, T-3, and serial multiport cards... Voila, internet company, initially started on a 486 dx2 back then... LOL 😂
Is there any alternative to the Motherboard? Because where I live there are literally none available.
I am officially impressed!
Now that the Minisforum MS-01 is out. The MS-01 might be the better appliance for high speed (> 10 Gbe) routing and firewall.
I am using a i3 10100 k cpu and pfsense. Might upgrade my nic down the line to a 10gug. But right now I get like 226mb down and 10mb up. Very efficient. Just need to figure out what to do w my wifi router or just switch to ap mode or buy a dedicated AOm
Would I follow the same process if I didn't need a router but wanted to build my own managed network switch?
Can you set the bios to have this boot on power loss? You want your router auto rebooting always. It would suck to have to go hit the power button or log in to ipmi, or worse not know you lost power because you’re away and have to wait until you come home to power up your network!
What do you think of the mikrotik ccr2004-something? Basically a router with 2 25g sfp interfaces stuck on a pcie card that exposes another 2 interfaces over said pcie. Stick it in your NAS, hook it up to a switch and job done?
Wendell did you do any VPN Client testing for through put speeds?
Regarding the X550-T2. Intel's product page claims that 2gb and 5gb are only supported in Linux. Since pfsense is freebsd based doesn't that mean x550 would only negotiate at 1gb or 10gb in pfsense?
what is the software at 14:51 ? looks like some sort of NAS OS from the early 2000's. This is something I would be interested in for sure.
Have you ever tried using OpenBSD instead of FreeBSD? I'd love to see benchmarks based on that. (I prefer OBSD's PF verses FBSD PF.
Is there a gui like pfsense? Or something like that?
@@magesnz No GUI based on OpenBSD PF I'm afraid.
I just got a internet upgrade from 1gbps to Fiberlink 2.5 Gbps. My motherboard has a 2.5G LAN port and they gave me a Zyxel AX7501 router.If I pay a one time fee of 100€ I get 10 gbps internet for the same monthly subscription. What do you think,I have a 5800x3d on B550 TOMAHAWK and the only motherboard I found with a 10g port is the X570 creation but used..I don't want to change the processor or the platform... is it worth changing the motherboard or getting a separate network card with 10g network port?
While it's good to see 2.5G ports becoming the norm now, 10GbE ports in motherboards need to really catch up, especially high-end ITX boards. I tried to find an x570 motherboard with 10GbE onboard and the only one I considered was the $1000 Gigabyte board, which I did not get.
For now I'd recommend to just buy a plug-in 10GbE card. I have the Asus XG-C100C that I got for $100 USD that works well in my X570 board.
How much power does it consume on average?
i have laying around a 3200g , B550 2x 4gb 2400 ddr4 .... and 2x Intel PRO 1000 Quad Port which i want to trade it fo Low Profile but 2 or more years ago none of the Router OS Supported WiFi 6 and i didnt have a Low Profile Quad port Ethernet
So you're saying the epyc 7742 I just bought is overkill for a little 8tb Nas
Looking at the MB+12500 with the nas sliger case for a nas build. Just a shame the MB isnt really out yet.
How necessary is ECC memory for a RAID NAS? I'd like to build a new machine to replace my current one, and wondered if its worth the money to get a motherboard like this one so I have error correction.
Intel ark does not show ECC support for i5-12400. Is it working even though it is not listed?
I have a question? You know those cards that give you 4 M.2s from a 16x slot. Will that NIC work on that card? What about the Sata port M.2s?
Cost efficient??? Nextwarehouse has it (motherboard) just above 700$.
Crying in dollar signs
@Level1Techs Would a 5600g/5700g be able to route as well as the 12400 in this scenario?
One thing I'm considering getting as a router + VM host is a Deciso DEC740. It's a GPU-less AMD Ryzen Embedded V1500B, in a small passive chassis. How does the CPU horsepower in one of those compare to the Alder Lake, for network tasks?
I do have my eye on W680 stuff for desktop, though.
I'd really like to get that Gigabyte MW34-SPO shown here. I guess GIgabyte isn't into selling these as I can't find one anywhere.
+1, I actually emailed their sales channel (through the MW34-SP0 page on their site), but didn't get a response. (To be fair, I'm not representing an enterprise or a system integrator, so it doesn't surprise me too much, but I was hoping....)
Same, it was first announced all the way back in the spring and... nothing.
@level1techs, Is Intel VPro feature a recommended thing to have for pfsense/opnsense build or not? I searched and saw some people like to have it and others wanted to disable or totally avoid it.
One other thing that I would suggest is that an AMD 7600 using PCIe 5 interfaces should yield better results especially locked all core clocks with a Linux or similar OS stripped to its bare bones.
wouldnt this build be better equipped with a threadripper since it gives you more pcie lanes on the CPU
That moment where the router you show that is "canned" is my exact router provided by Spectrum hahaha.
Would love to see the forbidden grandchild, some of us don't have more than 1gig but also don't want the isp modem/ont
I would really be interested in the performance of Clear Linux on that kind of platform, since it's curated by Intel, directly.
I'm using it as a general purpose hypervisor and docker host on a mere 10W TDP Intel SoC and NICs on Supermicro X11SBA-LN4F. Very nice board for a 100/30M VDSL connection which is all I'm getting in Germany, unfortunately.
I run 2 VMs on that with Sophos XG and 3CX. In Docker I'm running Home Assistant and a Unifi WiFi controller. With all that, the CPU is running 30-40% usage. The main constraint of this machine is RAM.
so the only way to get this board is to request a review sample from gigabyte?
Unless Wendell starts a group-buy!
I have had 10 for the last 4 years Hopefully this can help get the most out of it lol
Was there anything done in the Tunables to get greater than 10Gb throuput? I just built a host with the 12400 running OPNsense bare metal. Iperf running on the LAN interface. Only getting 7.8 Gb test results without running multiple streams.