Somos Internet CFO here. Forrest is as crazy and visionary as he seems in the video, but it's insane how many times his crazy ideas have proven right and we have made step change improvements as a company. If you want to come work on crazy world class engineering in Colombia, hit us up!
@@christopherd.winnan8701 it's in the beginning of the last chapter 1:02:50, he says they build the routers they give customers, and therefore they make it so that device sends up energy from the customer "up the system", eg up to their gear. Therefore the customer is paying for the electricity to power their ISPs gear
Point-to-point architecture is fine for multi-tenanted buildings where space and power is abundant (and probably free to the telco) in the basement for hosting switching equipment. However, when you need to deliver fibre to dense urban environments, p2p becomes very expensive. You either have to install cables with extremely high fibre count (cable is cheap, but splicing is time consuming = expensive), or you have to install lots of street cabinets with a utility power connection etc. in each neighbourhood. PON architecture isn’t lazy ‘old school’ technology - it’s just not the most optimal design for MDU buildings. FYI - this is coming from a CTO of a telco that operates almost entirely on p2p architecture! As a mainly b2b provider that works for us, but we understand for wider domestic deployment, PON is usually the better choice.
What did you think of his points (heavily translated into my own native "dummy" language) of pulling power from the routers, and/or the "put mini data centers throughout the network" stuff? Wondering if the end to end ownership can address the street cabinets thing
@@1stPrinciplesFM yeah he’s basically talking about what we’d traditionally call the ‘aggregation layer’ in an access network. He’s absolutely right - the more aggregation you can do, the cheaper it will be to build (generally).
@@TelcoGeekI agree with you and mention some of the same in my comments, I spent 15 years at a hyperscaler building very large fiber and transport networks. Many have tried various versions of what is contained in this talk, and they are expensive at scale, insanely so. Eliminating every coupler, patch panel, etc is critical. Beyond that, technology has outstripped the consumers ability to the capacity anyway. And even beyond that peering points and routing paths start to affect performance, backbone design, all these attempts to do deterministic egress to reduce costs affect performance. I’m not sure I even see last mile as that critical in the us except for underserved areas, which benefit greatly from PON and traditional FTTH (which you certainly don’t have to “rip and replace” every 4 years (which isn’t even a typical depreciation length). I’m not sure what this is solving honestly.
@@1stPrinciplesFM All he's talking about is PoE powered customer premises equipment, it's not new he just doesn't understand the stuff he's talking about and seems to be confused as if his engineers are creating new technology. Lots of fiber ONTs are powered this way. The same with fiber ring design with mini data centers we normally call PoPs (points of presence) and making the switches do routing vs. routers, if you are running a basic IPv4 and IPv6 network you can use mid to high end switches to handle the layer 3 routing since the routers bring other services to the table that they are not using nor probably need, just about anyone plugged into an internet exchange is using a layer 3 switch as a collection point for routes, none of this is new. A check of their BGP routes shows they are advertising 3,840 IPs with 27,000 customers? You can bet your life there's layers of NAT going on within the network and all of their IP space is just leased/reassigned space from Cogent. You'd think after 27k customers they could buy a little of their own IP space because they are 100% dependent on one of the worst network carriers on the planet who controls all that IP space. Also the performance required to move packets at 100Gbits is not going to fit on a little circuit board with a LED lamp on top nor is it going to be delivered for $80 without eating massive costs on the frontend for every port sold. Not bashing a company trying to solve a problem or jump on market opportunity for something, but a lot of this doesn't add up and you probably shouldn't have someone who openly admits he didn't know networking a few years ago go on camera and talk about all the networking. Its making us actual network engineers cringe and others are going to rip this video to shreds in the coming days.
PLEASE enter the Las Vegas area. We’re in a Cox stranglehold. They’re charging $200/mo for 1G fiber with data caps. I worked with an Open Fiber Network recently deployed in Colorado Springs and when I asked why Cox pricing was so expensive, the Cox rep said “because we’re the only provider in town…” Hand to God. Everyone in my neighborhood is totally frustrated by their over priced data-caped service.
Insane episode! Kudos to Somos, love it. 2 things became clear: CEO is really not good at communicating clearly. Christian is really not good at understanding networking (I did not either, but ChatGTP helped) :) Still: great convo! Keep up the good work
As someone who built one the largest hyperscaler fiber networks, I'm not sure this guy knows what he is talking about. Many of the "assumptions" he is talking about haven't been around for many years, and he doesn't seem to really understand the depreciation cycle for a FTTH network at all. I get that he wants to make these high level sweeping narrative elements, but many of these are strawmen that he is creating and don't reflect reality, and he is missing big chunks of the capex story for telcoms.
Yeah he’s definitely not telling the full story. Theres something missing. Nothing in Columbia happens without cartel involvement or the us government intervening.
This is actually quite similar as what large ISP do. Ive seen how its deployed. The only diffrence is the interconnect with othe fiber nodes in the field. I do wonder how this can work in a massive scale of the USA. Its one thing to network building together with Vlan fibers. But a diffrent story to cover a few large buildings miles apart and devided by a few neighborhoods. Or even a river or two.
42:00 That ability to control your own destiney is one reason why open source is typically seen as a major factor for successful tech companies. It also helps move companies towards the DevOps model helping address some of the team sprawl for a given thing like mentioned in his anecdotes.
Interesting! Feels like dependency on external open source code would be sorta the opposite of that though? You're reliant on someone else in that scenario
@@1stPrinciplesFM Kind of a middle ground really since you have full rights to fork it, study it, and modify it yourself you can make adjustments as needed by your organization at the speed of development but you can leverage community or commercial support as well. The level of reliance depends on the level of investment the organization wants to put into it.
UTOPIA operates a very similarly architeched network: point-to-point, commodity ethernet hardware, fiber huts (aka mini data centers) in various places around the cities where they operate, etc. 10 gig is available from most of the ISPs that operate on the network...not for $80/month though. I haven't priced it recently, but it was one of the cheapest places in the US for 10 gig Internet access. Faster speeds are possible, but so few people need those kinds of speeds, you have to special order it.
So glad there is a isp embracing modernity, so many network people I have spoken too have said things like "they can do IPv6 after I retire hahaha" or I was on a cruiseship with terrible performance despite being on Starlink, I did some tests and the provider was tunneling the traffic halfway round the world to thier data centre for thier conviniance amd in process adding enough latency as to make it unusable
42:55 this is actually the reason I don’t use isp’s router. They can see all the clients connected to it and their traffic. Even though these guys are decent but I don’t trust any of them. I will just throw udm, local dns forwarder with DoH
Going to look into this. South America and the Caribbean have fiber that's actually shared and the claim data speed isn't even close to 1% of the actual claimed speed. Not saying this is the case with this company but I will need to see it to believe it.
They are in a dense environment and it just sounds like they are deploying switches into buildings, connecting these switches together building to building, and then just giving customers an ethernet handoff from that switch in their building. So this can definitely scale but only under these conditions.
Pretty sad they have faster and cheaper internet then in Texas with Spectrum $80 a month for 600Mbps down 20up. We still have crap internet in most of the US on cable internet fiber is still really rare.
It would be negligible as the hops are seemingly very close, you will start to see increased latency as it travels farther distances between hops, which will generally be between cities, and out onto the internet
It does help when fiber and cable have to compete for customers, the new fiber in my area is trying to lure people who have had high speed on cable for years. Fiber in my area is $55 for 300 Mbps, $65 for 500 Mbps, $80 for 1 Gbps; unlike the cable company these are not guaranteed to go up after an intro window, but time will tell if how stable those are. With the cable company just responding so far by opening a 2 Gbps tier, rather than adjust their prices.
56:22 We have housing plans that will increase localized density massively, maybe not country wide density averages (NEVER trust those!) all over the west as well. Density is community, but why else??? 56:22
Routing vs Switching on CPU vs ASIC was wrong, both can done by hardware. Hyperscalers have as little switching as possible and route as much as possible. Hence CLOS fabrics
@@haitiankid4lyf Nope, Somos has public IP space so they are handing out public IPs to each customer's router. No NAT past the customer's router doing it. I'd hope they aren't just doing a bunch of L2 switches, but rather using L3 switches as routers, which would make this chaining of buildings together much less of an issue even if it's a bunch of hops.
Very well that another company enters the telecommunications market in Colombia, but the Somos Internet service sucks too much and is very bad, and the worst thing is that it is not fiber optics, but ethernet cable. Knowing that the future is fiber optic FTTH.
What's the problem with getting ethernet instead of fiber, for a regular consumer? In a home you're going to convert from fiber to ethernet to plug into your computers/the Wifi access point sooner or later anyway. As a business that runs fiber all the way to its servers in the rack I'm sure you could work out something to rent a plain fiber without the ethernet layer on it from them, too.
46:08 random thought, well repetidive one Ron. You want to stop advertising peak datarate, like 100GBits/s and INSTEAD the price per GB (say 2 cents) during peak and low usage hours ron! Why?!??? We don't lie for one thing hint 46:08
@@danielsalcedo30Usan conexiones mediante cable UTP, por lo cual lo que instalan en el apartamento al final es cable UTP y no fibra óptica como si lo hace Movistar o Claro
What is bad about it? Bandwith too low? Latency through the roof? Service interruptions? Too expensive? "Bad" really doesn't mean a lot, other than that the customers are unhappy for unknown reasons.
Somos Internet CFO here. Forrest is as crazy and visionary as he seems in the video, but it's insane how many times his crazy ideas have proven right and we have made step change improvements as a company. If you want to come work on crazy world class engineering in Colombia, hit us up!
Sent you a req on LNKD lol
Getting your customers to pay you in both money and electricity is wild.
@@JohnDobak How does that work?
@@christopherd.winnan8701 Somewhere in the video he mentions that Somos routers in customers homes send Power Over Ethernet back to their switches.
@@christopherd.winnan8701 it's in the beginning of the last chapter 1:02:50, he says they build the routers they give customers, and therefore they make it so that device sends up energy from the customer "up the system", eg up to their gear.
Therefore the customer is paying for the electricity to power their ISPs gear
Amazing discussion on how to build vertically integrated businesses and handle complexity
Point-to-point architecture is fine for multi-tenanted buildings where space and power is abundant (and probably free to the telco) in the basement for hosting switching equipment. However, when you need to deliver fibre to dense urban environments, p2p becomes very expensive. You either have to install cables with extremely high fibre count (cable is cheap, but splicing is time consuming = expensive), or you have to install lots of street cabinets with a utility power connection etc. in each neighbourhood. PON architecture isn’t lazy ‘old school’ technology - it’s just not the most optimal design for MDU buildings.
FYI - this is coming from a CTO of a telco that operates almost entirely on p2p architecture! As a mainly b2b provider that works for us, but we understand for wider domestic deployment, PON is usually the better choice.
What did you think of his points (heavily translated into my own native "dummy" language) of pulling power from the routers, and/or the "put mini data centers throughout the network" stuff?
Wondering if the end to end ownership can address the street cabinets thing
@@1stPrinciplesFM yeah he’s basically talking about what we’d traditionally call the ‘aggregation layer’ in an access network. He’s absolutely right - the more aggregation you can do, the cheaper it will be to build (generally).
@@TelcoGeekI agree with you and mention some of the same in my comments, I spent 15 years at a hyperscaler building very large fiber and transport networks. Many have tried various versions of what is contained in this talk, and they are expensive at scale, insanely so. Eliminating every coupler, patch panel, etc is critical. Beyond that, technology has outstripped the consumers ability to the capacity anyway. And even beyond that peering points and routing paths start to affect performance, backbone design, all these attempts to do deterministic egress to reduce costs affect performance. I’m not sure I even see last mile as that critical in the us except for underserved areas, which benefit greatly from PON and traditional FTTH (which you certainly don’t have to “rip and replace” every 4 years (which isn’t even a typical depreciation length). I’m not sure what this is solving honestly.
@@1stPrinciplesFM All he's talking about is PoE powered customer premises equipment, it's not new he just doesn't understand the stuff he's talking about and seems to be confused as if his engineers are creating new technology. Lots of fiber ONTs are powered this way. The same with fiber ring design with mini data centers we normally call PoPs (points of presence) and making the switches do routing vs. routers, if you are running a basic IPv4 and IPv6 network you can use mid to high end switches to handle the layer 3 routing since the routers bring other services to the table that they are not using nor probably need, just about anyone plugged into an internet exchange is using a layer 3 switch as a collection point for routes, none of this is new. A check of their BGP routes shows they are advertising 3,840 IPs with 27,000 customers? You can bet your life there's layers of NAT going on within the network and all of their IP space is just leased/reassigned space from Cogent. You'd think after 27k customers they could buy a little of their own IP space because they are 100% dependent on one of the worst network carriers on the planet who controls all that IP space. Also the performance required to move packets at 100Gbits is not going to fit on a little circuit board with a LED lamp on top nor is it going to be delivered for $80 without eating massive costs on the frontend for every port sold. Not bashing a company trying to solve a problem or jump on market opportunity for something, but a lot of this doesn't add up and you probably shouldn't have someone who openly admits he didn't know networking a few years ago go on camera and talk about all the networking. Its making us actual network engineers cringe and others are going to rip this video to shreds in the coming days.
@@jasonjones9888 Well said. Single-homing behind 174 is not great for sure.
woah the lamp router is so fire. im about to start dismantling usgs and throwing them in lamps for an etsy
I wonder if they run LAMP and also run the matter protocol so that it'd be lamp on a lamp controlling a lamp :-)
Interviews in this place are really interesting.
Really.
Interesting.
How interesting
100 Gbps for $80/month is just crazy
Truly. I want it here in the Statea
Cool insights that's exactly how we have designed our FTTB connections, Ethernet to the customer
PLEASE enter the Las Vegas area. We’re in a Cox stranglehold. They’re charging $200/mo for 1G fiber with data caps. I worked with an Open Fiber Network recently deployed in Colorado Springs and when I asked why Cox pricing was so expensive, the Cox rep said “because we’re the only provider in town…” Hand to God. Everyone in my neighborhood is totally frustrated by their over priced data-caped service.
The max we‘re getting in Switzerland is 25Gbits for about US$70/month. Still more than most people can saturate.
Insane episode! Kudos to Somos, love it. 2 things became clear: CEO is really not good at communicating clearly. Christian is really not good at understanding networking (I did not either, but ChatGTP helped) :) Still: great convo! Keep up the good work
@@longrangepodcast hahaha I love the shit sandwich of feedback
Guilty as charged that I got confused, hopefully it helped you understand later!
@@1stPrinciplesFM no offense, I hope :) was super interesting to listen to and dive into some details myself then!
As someone who built one the largest hyperscaler fiber networks, I'm not sure this guy knows what he is talking about. Many of the "assumptions" he is talking about haven't been around for many years, and he doesn't seem to really understand the depreciation cycle for a FTTH network at all. I get that he wants to make these high level sweeping narrative elements, but many of these are strawmen that he is creating and don't reflect reality, and he is missing big chunks of the capex story for telcoms.
I cringed when he was talking about switch vs router, which is a huge misunderstanding he has.
Yeah he’s definitely not telling the full story. Theres something missing. Nothing in Columbia happens without cartel involvement or the us government intervening.
"instead of cgnat, we just do vlans"
This is actually quite similar as what large ISP do. Ive seen how its deployed. The only diffrence is the interconnect with othe fiber nodes in the field. I do wonder how this can work in a massive scale of the USA. Its one thing to network building together with Vlan fibers. But a diffrent story to cover a few large buildings miles apart and devided by a few neighborhoods. Or even a river or two.
Yeah, definitely intended to be for dense locations. Probably you'd have to start in the northeast
Cox Southern Nevada be like “$1200 month, promotional offer! With 1.25TB data cap!”
42:00 That ability to control your own destiney is one reason why open source is typically seen as a major factor for successful tech companies. It also helps move companies towards the DevOps model helping address some of the team sprawl for a given thing like mentioned in his anecdotes.
Interesting! Feels like dependency on external open source code would be sorta the opposite of that though? You're reliant on someone else in that scenario
@@1stPrinciplesFM Kind of a middle ground really since you have full rights to fork it, study it, and modify it yourself you can make adjustments as needed by your organization at the speed of development but you can leverage community or commercial support as well.
The level of reliance depends on the level of investment the organization wants to put into it.
@@AndrewMorris-wz1vq cool perspective, thanks!
We need this in trinidad and tobago like yesterday
USA too
UTOPIA operates a very similarly architeched network: point-to-point, commodity ethernet hardware, fiber huts (aka mini data centers) in various places around the cities where they operate, etc. 10 gig is available from most of the ISPs that operate on the network...not for $80/month though. I haven't priced it recently, but it was one of the cheapest places in the US for 10 gig Internet access. Faster speeds are possible, but so few people need those kinds of speeds, you have to special order it.
Indeed, it’s $120/month for 10Gbps via Utopia
So glad there is a isp embracing modernity, so many network people I have spoken too have said things like "they can do IPv6 after I retire hahaha" or I was on a cruiseship with terrible performance despite being on Starlink, I did some tests and the provider was tunneling the traffic halfway round the world to thier data centre for thier conviniance amd in process adding enough latency as to make it unusable
That's wild. Also not that uncommon with home routing for roaming
Somos software engineer on deck, didn't know about the rings really, but I know soon enough we'll be building Halo rings for life in space 😂
@@jdrt10 I'm down
Interesting talk
Thanks! Was a super fun one to record
You rediscovered NMLI / CMDS from 1990’s.
Yes, that is how iteration works.
Epic
Thanks king
42:55 this is actually the reason I don’t use isp’s router. They can see all the clients connected to it and their traffic. Even though these guys are decent but I don’t trust any of them. I will just throw udm, local dns forwarder with DoH
And more than this company does not allow to put its own router. It's time to use their router, which is too bad.
Going to look into this. South America and the Caribbean have fiber that's actually shared and the claim data speed isn't even close to 1% of the actual claimed speed. Not saying this is the case with this company but I will need to see it to believe it.
You are building a CLOS network across the city? So you are telling me, the slums of Medellín has faster internet access than major US cities?
He’s just connecting buildings to buildings as he goes and eventually land back at a datacenter. Definitely not a CLOS network.
When are you coming to NYC?
It helps when your labour cost is almost 0. Active Ethernet point to point doesn’t scale.
They are in a dense environment and it just sounds like they are deploying switches into buildings, connecting these switches together building to building, and then just giving customers an ethernet handoff from that switch in their building. So this can definitely scale but only under these conditions.
Pretty sad they have faster and cheaper internet then in Texas with Spectrum $80 a month for 600Mbps down 20up. We still have crap internet in most of the US on cable internet fiber is still really rare.
can i get one in castle rock colorado?
Curious what the latency is on a network like this with many hops before you can get out to the internet.
It would be negligible as the hops are seemingly very close, you will start to see increased latency as it travels farther distances between hops, which will generally be between cities, and out onto the internet
48:43 That's what We Will Do as well❤❤❤❤ 48:43 and he will be One of Us soon ron❤😮😊 48:43
No ISP is going to charge $80/month when they can charge $90 or $190 to people willing to pay it.
an isp trying to undercut and onboard/poach costumers would have incentive to offer a lower price.
It does help when fiber and cable have to compete for customers, the new fiber in my area is trying to lure people who have had high speed on cable for years. Fiber in my area is $55 for 300 Mbps, $65 for 500 Mbps, $80 for 1 Gbps; unlike the cable company these are not guaranteed to go up after an intro window, but time will tell if how stable those are. With the cable company just responding so far by opening a 2 Gbps tier, rather than adjust their prices.
and yet ISPs do exactly that. Retail pricing is $130 for Gig, but they'll frequently sell for $30 / $40.
56:22 We have housing plans that will increase localized density massively, maybe not country wide density averages (NEVER trust those!) all over the west as well. Density is community, but why else??? 56:22
Routing vs Switching on CPU vs ASIC was wrong, both can done by hardware. Hyperscalers have as little switching as possible and route as much as possible. Hence CLOS fabrics
How. Fast. Do. These. Guys. Talk???
@@isaaclarson we live at 1.25x speed
@ super cool video/talk regardless. Really hard to follow if you've become relaxed lol.
@@isaaclarson haha fair honestly. the interview I just did today was much more reasonably paced
Why would i need 100 Gb/s in my apartment?
@@christopherd.winnan8701 Why not have everything instantly load if it's the same cost as normal 1 gig internet?
how does nat work in this situation?
Same as it does anywhere else, the customers router is doing NAT
@@ZippyDooDa435 make sense but was wondering if there would be double or triple nat since it seems like it's traversing multiple l2 switch
@@haitiankid4lyf Nope, Somos has public IP space so they are handing out public IPs to each customer's router. No NAT past the customer's router doing it.
I'd hope they aren't just doing a bunch of L2 switches, but rather using L3 switches as routers, which would make this chaining of buildings together much less of an issue even if it's a bunch of hops.
CGNAT. They're only advertising roughly 3300 Public IPs but have roughly 27k customers. CGNAT is inevitable at this scale.
@@Salted_Caramel88 I wouldn't even want the service then.
Very well that another company enters the telecommunications market in Colombia, but the Somos Internet service sucks too much and is very bad, and the worst thing is that it is not fiber optics, but ethernet cable. Knowing that the future is fiber optic FTTH.
Are you a Somos customer?
@1stPrinciplesFM I was a customer until a few months ago.
What's the problem with getting ethernet instead of fiber, for a regular consumer? In a home you're going to convert from fiber to ethernet to plug into your computers/the Wifi access point sooner or later anyway. As a business that runs fiber all the way to its servers in the rack I'm sure you could work out something to rent a plain fiber without the ethernet layer on it from them, too.
46:08 random thought, well repetidive one Ron. You want to stop advertising peak datarate, like 100GBits/s and INSTEAD the price per GB (say 2 cents) during peak and low usage hours ron! Why?!??? We don't lie for one thing hint 46:08
Poor bait at title and admitign in description you lie.
Dumb questions and misconception change the world. Ye like mu psshht ??? Sequence assoc hint haiahah 50:44
This service its bad, i know people how have it and it's Bad!!!
Los que conozes que usan Somos, usan Fibra o coneccion de FiberWave?
@@danielsalcedo30Usan conexiones mediante cable UTP, por lo cual lo que instalan en el apartamento al final es cable UTP y no fibra óptica como si lo hace Movistar o Claro
What is bad about it? Bandwith too low? Latency through the roof? Service interruptions? Too expensive? "Bad" really doesn't mean a lot, other than that the customers are unhappy for unknown reasons.
@@santiagovargasr538 You can do up to 5 GbE on UTP so that in itself is fine.