SaltEMike Reacts to The Citcon Server Meshing Panel
ฝัง
- เผยแพร่เมื่อ 21 ต.ค. 2023
- The Video - WHEN THEY POST IT ON YT
Check out my main channel - / @saltemike
Check out my clips channel -
/ @saltemikeclips
Check Out My Twitch Stream to see this live: / saltemike - เกม
This has to be the best Citcon to date. The very best. They just blow you away in ever panel. The first time ever the ship team really was not front and center. Ships were not a big deal. The actual game foundation was front and center. So good!
Glad to hear they finally made some progress. Let’s hope it stays that way and wasn’t a one time get people back then press new marketing ships kind of thing.
I think what makes this best is its all actual game demonstrations, not projected ideas and its so much of what we have wanted for SO long
Only dissapointment was the ship panel, that's perfection.
Day 2 and they are announcing something about SQ42 at the end. So... LETS GO!!!
@@schlagzahne6741 It wasn't the actual game though was it but a choreographed and stage-managed demonstration.
The lag spike when Benoit transitions from green to red IS the proof of server meshing. Plus running 3 servers and the replication layer on a single system with multiple clients connected, you would expect some latency.
keep in mind that this is done locally, imagine this running on a dedicated datacenter and server,the lag should be gone
@@UnKnown43783exactly this would be getting processed on multiple real servers and not a bunch of basically virtual machines sharing a single set of hardware
@@UnKnown43783 not gone, but lower. Plus i would expect that they would add in transition areas as well if needed
@@shadowlordalphaI wouldn't be so sure about that, this is V1 of the tech so ofc some hiccups will occur but with iteration, further optimization and dedicated servers this should work flawlessly, otherwise it would be jarring ingame with hundred of thousands of players.
@@UnKnown43783sophisticated servers for their task would make a huge difference if they are really good interconnetced im so exited for the future of the tek
Your channel won't die, man. Folks don't watch your channel because you're cynical, they watch bc you are being honest. You're not trying to hype or de-hype anyone.
word
Eminem
I don't watch Olli .... he's like a little baby happy with everything .. every promise .I can't stand this 6-year-old optimism ... Mike seems more reasonable.
@@I3ladefist yeah man, I liked Olli's first videos as they were his first reactions to the game and it was cool and all.. but then I felt too much hype from him and that just doesnt feel good... 6yo optimism is the best definition one could get.
Yea, I avoid Bored channel with a flamethrower so I can come here to get the best of the best.
Ships having their own dedicated server will bring literal meaning to large scale server battles.
I was talking some shit coming into this, given the glacial pace of meaningful updates and the necessary tech coming together for the PU systems to function, but I am so happy to eat my words.
This CitCon hasn't totally allayed all of my trepidations about release dates etc, but the spark of hope is there again and it's so nice to see the team showcasing new tech, managing expectations in a healthy way and finally demoing the vision that all of us want to take part in.
The legacy of boundary-pushing PC games might actually have a new yardstick by which to measure contemporaries. It actually seems possible again. What a wild ride.
It pleases me to see someone actually appreciate just how important that silly little demo was. It's literally the missing foundation corner that this whole house of cards is built on. We've officially arrived at "There's a chance". I think scaling this will be a lot easier than creating it. I bet this time next year we have full OCS and SM.
possibly, but since this is version 1 with static it is currently unable to scale so client side players shouldn't notice any major improvements but server side things should be a lot more stable, at the same time spaital os was the first in 2017 with their demo albeit they failed and I don't know what has happened since, but either way it's not something that was comprehensible 20 years ago.
@@yulfine1688I mean the static part was at least nessesarry to ever want multiple systems and massively helping relative server load. Which I believe to be very important when so much of the experience is dictated by server health.
SC is factually a better game when the servers run better. From menus and transactions going on faster, to NPCs being far faster and smarter (especially FPS ones).
The thought of having two capital ships flying through space fighting each other and each one is its own server is mind boggling.
YES. If they can scale this properly it means that full crews of capital-class ships will run on their own independent servers and can engage in combat with other ships of the same class and there wouldn't be any performance issues.
That's the dream, at least. It's looking closer to where it might be possible soon.
If they really can scale it up it's absolutely massive, it will completely change gaming as a whole. Most games are limited to 100-200 players MAX but with this it will be possible to have thousands of players in the same area. It's one of the biggest breakthroughs made in gaming ever.
It will still have a maximum. Keep in mind the replication layer is a single server receiving, storing and resending the state of all the objects of multiple servers involved.
Something I'm worried about is what happens in non enclosed spaces. The reason why you can stream in and out the red and blue servers is because they can't see each other, what happens in a large outside environment or in empty space if there's hundreds of fighters flying? Does the replication layer have to stream all servers within visual range of each other?
@@PolBlanesCebrianI'm pretty sure anything in space would be proximity based.
If a thousand nutjobs are blowing shit up, you can turn around and leave if it's laggy. Logout in your ship and "hopefully" hop on a different server that's not at war. Hopefully.....
It was definitely a exciting moment.
Now, lets see it in game. Then the dream will be a bit more real
I dont think you can quantify how much hate and BS that team deals with from literally everyone. I am excited for them to have some of this figured out partly. Bc it benefits me, but also for their own sanity.. good for them
yup this is huge. With 10 years into development I was beginning to doubt server meshing would ever become functional. Massively impressive to see they did it. This is not your average online game server.
"I cried"
Yeah... The memes will be legendary about this Citcon in 12 months.
He cried then. We’ll all maybe cry later. Hope it’s tears of relief but not holding my breath for that.
I seriously wish more people understood exactly how freaking MAGICAL this technology is. Sadly, most people won't get it, won't understand, won't notice, or won't care.
Perhaps we need a "SugaRMike" channel as well :)
Just another 10 years.
Mike I've always liked how genuine you are. You make some great content. Keep up the awesome work.
Watched the whole stream with you and Levelcap. Just wanted to say, that you guys were a great combo and if you should repeat that next year, I will drop in again. 👍
Not only is this a milestone but the tech developed, if it were to be licensed to developers for other games, be it starengine or this server meshing, would likely be worth billions.
Mike i watched you live reaction yesterday. And i dont blame you, the emotion of what this panel meant was palpable
Its amazing to watch the hype cycle live. Ill believe it when i see it in game
Yea, and I remember a "jump to Pyro"
When it is deployed to the PU...great. I'm not going to hold my breath.
It's hard to believe they finally made it. And I want to be happy and want to believe in this game, because this is a game with a huge potential. But I'm still somewhat cautiously happy. And I want to see it coming to life. I still consider it took too long time to develop this core feature. But if we have it in the end, I'll be happy.
The only worrying part:
According to him, if you are in green that is next to purple, green is authoritative over entities in green only… BUT green simulates everything in purple. So while green is not authoritative over purple it still has to spend resources communicating with purple, and simulating what happens there.
This is why it is seamless, both green and purple are simulating one another and do a simple authority ‘hand off’ when an entity crosses. It also means green is doing a lot more work than if it was only authoritative/simulating itself.
So a few things:
Latency is bound to be considerably higher interacting between server boundaries at least, i.e. rubber banding while guy X strafes left and right, and will cause freezing at worst.
What makes me optimistic is despite this the server is responsible for a much smaller area. So while it will have extra work simulating areas next to it, the overall area it is simulating is smaller than e.g. the entirety of stanton.
I expect some big teething problems with this but still considerably better than today.
There are a lot of people who don't understand what the "server restart" scene represents. This will not happen during an error of this type. There are always backup servers that can take over, there is no need to restart one manually (this is a demo). This means that on 2 servers (main and backup), one can "transfer" (the replication layer does this constantly during the game) the data to the other and the backup becomes the main server. Thus the main, becoming the new Backup, can be diagnosed and restarted while the players continue to play. Later becoming a new backup. And so on.
All this will not be visible.
hi mike,
at 22:30 paul mentions he’s running everything on the one pc and so it’s a little slow, this would include the server as he’s able to kill/revive it all on the one screen, that’s probably why it takes a few seconds. on a computer that’s only running the client it’s probably wayyy faster at reviving a server
hope that puts your worries at ease :)
either way there will still be some delay and hiccups that's just normal if it was perfect I'd be very concerned
This just makes me happy for us and for them, man what an incredible step forward, people just dont understand this level of complexity from IT side of things. It´s absolutely amazing, and more important seeing how the gaming industry is moving the last five years... I mean yes we are just used to delays and bugs, but seeing how proud they feel on this, being able to feel how slowly performance is becoming better in PU, just respect.
Being a technical guy, this is an amazing thing.
HOWEVER... They had to make everything "physical" as single entities. This makes me think of beam weapons and tracktors. So there must be overlapping between zones. I'm sure they have thought about it though and the lag between them. The question of lag between the servers is very important but, again, they have thought about it.
It’s cool Salt. I hope they really get this done, this time. They have so much practice , practice over years and years of hyping us up, then more years go by. But I will hope with you Salt.
I think the import part in the transition is that the purple and red sever are out of POV from each other because of the corner. So if all that is disappearing is stuff out of POV then we will never notice.
yep its like occlusion culling only render what is on screen and visible
I also noticed, they have server meshing active all the time, so the demo starts out with 1 server controlling multiple zones. So they can scale and optimize server density. If theres an area with an influx of clients, you can have more servers take over more zones vs small number of servers that each serve a large set of zones.
CC was awsome glad I could talk to you guys after Day 2. Regards from xEtarYx
Wow. Three weeks since they got it working, that had to be nerve racking coming up to CitizenCol
Paul Reindell is a freaking legend, people should learn his name, if this tech works out he will have been one of the major players who pushed game development into a true next generation
This is either SC's most important breakthrough or another 2016 CIG sandworm.
best citcon ever. Full stop. The wait has been long but they were right in working every feature on SQ42 first
My concerns are that there is notable desync even with 3 containers running on the same host.
Being on the same host is the best case scenario when it comes to networking.
Theres no traversal of intermediate networking devices like firewalls, routers etc
Now translate this to Cloud based containers in multiple regions and continents and latency will be a big issue.
Also if you look at the resource usage from the tool int he bottom right corner when he spins up the 3 servers, the total RAM consumption is around 30GB.
Now lets assume they have debug logging enabled and have done minimal optimisation to give them worse case.
Still this is for a tiny area footprint with 2 players and a buggy.
Now scale this to the aspirational size of Star Citizen....the infrastructure footprint and cost to operate will be Huge.
I dont think many people realise just how expensive its going to be to run this, which suggests a significant revenue stream will be necessary to maintain this on an ongoing basis.
Needing an ongoing revenue stream will mean both a subscription model will be essential and microtransactions too. Which will inherently make the game unbalanced whale Vs avg Joe.
they're running this from a single system so I am not surprised there will always be delay no matter what you do because theres limit on fast information is able to travel and so on, but on a proper server system it should be mostly minimal outside of larger scale conflicts
On a big picture scale this is just a small baby step in all honesty. I'm glad they now have a working foundation for static servermeshing. But when we're talking about DYNAMIC servermeshing we're literally talking a universe away still. Technology will continue to hold that reality back for many years to come still. But at least they finally have "something" to show finally for our hard earned money right? And SQ42 looks simply amazing now as well. So yeah the dream is still alive but sadly for the Verse side it's still just going to be a lucid dream for a long time to come still. 36:29
Wow!! Mike I can't believe you made time to create a React to this. Hopefully you got some sleep 😝
One thing I noticed a lot of people in chat seemed to get caught up on, is the term server, and having multiple servers. It's indeed multiple servers, but the servers job is only to do the simulation, as in the physics of what is happening. All entities ( that means any object can be moved or interacted with, so basically everything but the ground) is simulated by a server. The information about the simulation is then fed to the replication layer that sends the data to all the other servers.
The reason you can see and shoot something from one server from an other one, is because every thing exist on both servers, at all times, it is only that server A is simulating person A and server B is simulating person B.
In SC even bullets are entities and objects, so when you fire a bullet it will travel though it's server, and then when it gets to the boundary of that server/container it will be handed over to the new server where it will continue as all the information about the bullet is stored in the replication layer, so the new server can pick up the bullet exactly where the other server left of.
All this ends up that for the user there are not really "multiple servers" or containers or instances. A user only connects to a server ( what they call a shard ) and that can consist of anywhere from 1 server to a thousand physical servers all running together to form that server.
And when running this system with the replication layer not tied to the server, and having multiple servers you can indeed have a server per ship. They can also have multiple servers per zone if that is needed, as it's only a matter of assigning what server has the authority of that entity.
(Talked with Paul at the CIG After party for quite a while about the tech and the dude is a wizard, but take my memory and understanding with several grains of salt as I'm just going of my own limited understanding of everything)
As a programmer, I have to say, these developers are extremely capable and have impressed me in many ways. Server meshing is just one of the many things that was thought to be almost impossible until the Star Citizen people solved every one of it. 2024 may be a very interesting year. :)
Relentless optimism guys, we must hold the line😂
Probably the most important feature and panel this past weekend. Hope? Yes! Hyped.. not yet. Reality will set in as the months go bye and Mike will remain salty I have faith lol
How does this work when you have 50-100ms latency with the servers, though? If the time between client and server is non-instantaneous, does it hold up?
implementation without any issues and bugs approx 2037.
That seems optimistic.
Running a tiny environment on a single PC is still a ways from what needs to happen on the PU. But damn it is pretty amazing!
Good that CIG could show a proof-of-concept demo of static server meshing, including something of a failover/failback test for both 3.22 (DGS + separate replication layer) and a comparison with the static server meshing.
- I'm not doing somersaults yet, since separate replication layer is neither in the PU or "standard" PTU yet.
- I expect - even more so than PES - that there will be a raft of issues with both replication layer and static server meshing at scale
- Would be nice to see static server meshing in PTU by end 2nd quarter of 2024 ... as always ... we'll see!
What internet speed is needed to pull this off? What hard drive / Ram speed is needed?
I wonder how far away is each server physically, how close do they need to be for this to work? Do they need to be in the same country? What latency can this work at? Do this again, but I want to see Server 1 in Europe/Africa, Server 2 in South Korea/Japan, Server 3 in Australia. Hmmm I wonder if rays could be ray traced from server to server?
Paul is an incredible engineer. He was also involved with the introduction of planet tech. He is so passionate about this stuff, you can see how proud he is that they pulled this off. And rightfully so.
This reminds me of how nuclear fusion researchers recently achieved ignition for the first time. It doesn't achieve anything on it's own, but it proves that it's actually possible.
@SaltEMike, In relation to your question at around 19:55 in the vid:
The tech you experienced with your mining example is a persistence example. A server crashes and its state is restored when the server is brought back up. What the second layer (the replication layer) does is run the same instance over multiple servers Like: Your on orison, there are for example 2 major zones and 20 ish small zones, the 2 majors being the city and its area's and the small ones are player streamed hangars. These are "meshed" together to behave as a single game session. And multiple servers handle all the load, dependent on the load required and the server config a single server could run anywhere from 1 zone to several thousand.
Now when 1 server dies, the replication data on the replication server remains intact, and your entity (your player) is migrated to another server with the same replication layer as the one your are on. You might experience a hickup of down time, and if the crash is big enough, lets say a physical server broke down rather then a virtual server then new servers will need to be allocated for more data and no existing server might have your particular replication layer active, in that case you could get 10-20 seconds of lag and then pop back in. But I expect they will split single instance loads over multiple physical devices so a single device crash cannot cause a major issue.
What this means is, when your server crashes, your experience of that crash is the same as in the example in the video, your character moving from Purple to Green zone. AKA. you dont even see it happen. = No more 30K, now if the replication server crashes.. (and i expect them to run concurrent replication for that as well) we might get a crash. But a handler server like that has a extremely low load and complexity, allowing it to be highly stable.
What currently however, and i did not see indicated here, happens is that it takes a while for the server to "crash" with the server running slower and more poorly over time and then crashing.
I expect them to put in play pre defined metrics for a servers health that either the replication server checks for or a controller does. So that when a server starts to under-preform you are moved to another server in situ, and that server you where on is rebooted preemptively .
The tech itself is not that different from current day database servers, where a single stack of physical servers behaves as a single server, and any virtual clients, or datasets running on the server exist as shared data. giving it high redundancy. Current server replication on data center level is a replication of the physical servers content to another site or server, replicating it over 2 physical stacks of servers often running dozens of server instances on each.
Never seen it in games like this.
A game like WoW only has a dataset for player locations if your server crashes you are moved to another seamlessly , but any entities on the original server are lost (creeps/bosses etc) It allows them to layer multiple servers over each other to allow for redundancy and more players in a given location at a given time without the server struggling to handle it.
People seem a little concerned about the size of the zones in the example. But the size doesn't matter (in this case). The servers will scale up and down based on demand and the zones will be defined/configured dynamically. Previous plans for server meshing were based on the idea that certain servers covered certain zones, like there could be a server for each planet, each sector of space, maybe a dedicated server would spin up for a huge ship or battle, etc. But in this example of the modern iteration, those zones could be as big or small as you want/need. All three of these zones could be controlled by one server, split between 100 servers, replicated/duplicated to parallel servers (i.e., people in the same place/zone, but on different servers, seeing the same things, interacting with the same environment and with each other), or areas can be completely turned off when there is no demand. Plus, since the replication/PES handles all the persistence and streaming, the AI, dynamic economy and other background simulations/mechanics will be able to run independent of the game servers. An AI miner could be out mining somewhere and if there's nobody there to see it, no server needs to be spun up. It's just a matter of how many physical servers you have and you could theoretically have a million people in the same place at the same time. Not that anyone's PC could handle it....
From what they detailed was managing cost to effectiveness ratio not every time a org uses a javelin can they have their own server for that destroyer and so on it would probably be too costly so the next major step moving from static to dynamic would be finding a balance with the shards for this to happen. That is going to be very difficult to do
@@yulfine1688 Lots of tweaking to do. Not going be a smooth process I'm sure. But still a big step in the right direction.
Will you have your react video for StarEngine reveal?
This is the most important aspect for the SC ‘Verse and we just re-Watched it in real time. Fuck yes 🤙🏻
I believe this would hold up at longer ranges as I dont think its intended to have 3+ servers literally running areas right next to each other.
As of right now, we have individual servers handling the 700k+/1M+ entities of the entire system.
If there was a massive 200 player, multi cap ship battle forming around Ghost Hollow, I'm assuming a server would spin up to handle the traffic of maybe the 60km region around it. That would only be the 200 players, their ships and gear, and like a handful of NPCs.
For larger battles, I'd assume they would further nest spheres of influence so if you got within visual range, you would appear.
Big Deal - Mike great content you are really taking your content to the next level
Also having large fleet battles should be possible (space or planet side or both) without time dilation like in EvE as each ship (and perhaps even compartments in ships) will have their own dedicated containers linked via the server meshing. So having 100 players on a one Idris or Javelin and other Idris/Javelins having their hundreds can all be managed by their own servers dedicated to their clients and only having to track entities that cross into their container. (Missiles, bullets, lasers, other players or vehicles etc).
At that point the only thing holding back the game would be client side rendering of all that going on but at the distances and scales that these sorts of battles will play out at most of the time, I imagine that will also not be a problem.
It s a very long way off (perhaps several years just for the foundational tech here to be optimised and matured enough) let alone actually implementing the final result and testing. But its now actually in sight and far more realistic.
It would be great if the logic could anticipate large fleet battles when huge numbers of entities start to gather and start spinning up fresh servers on the fly, transitioning the clients to these so its in pristine condition for the actual slog it's about to go through.
Ah, I can but hope and wait.
I need the SQ42 fleet but in SC on multiple player sides sides squabbling over contested space in unclaimed systems, with a looming Vanduul invasion just casually happening mid battle as we all react to the chaos and some join up as others exploit the madness for easy kills and loot. An actual epic clusterf**k with lasting ramifications on the in game universe. Imagine the salvage and pirating and ripples through the economy. The UEE Navy showing up to lay the smack down, albeit a little late.
The "magic" is the replication layer. That is what allows the smooth transition between servers. That and super low latency networking. I'd imagine they'll want all the servers colocated on the same network. Curious to see how it will work with clients over Internet. That's probably the biggest remaining hurdle.
I'm fairly certain Star Citizen is hosted on AWS.
Pearl Abyss's Black Desert Online does this and there is a slight stutter when you cross into a new "streamed cell". I'm curious to see if CIG can scale this without hiccups.
Mike, I'm with you, the whole dang, two day show, was full of great presentations but that server meshing demo was the show stealer for me. This is incredibly tough technology to crack and they did it, even if it was only on a small scale for the demo, it still shows they have accomplished something great and I look forward to seeing it scaled up to the PU. Freakin awesome!!!
This is all great, but in RL a lot of the potential issues will be avoided because the boundaries between servers will be in space between planets and/or moons. If you assigned one server per celestial body, you'd scale up the capacity of Stanton by what, 16x. With a little conservative margin, let's say a player cap of 1200+, instead of 100. But it would be easy to split it up even further - say, one server per major LZ, one per orbital station and LaGrange point station. None of those would require much in the way of viewing/shooting across server boundaries.
In other words, what they're showing is worst-case. If they can come up with a good algorithm for redistributing server boundaries, then dynamic meshing will take care of the rest. At that point CIG will have an asset on their hands that they can license to other game companies for hundreds of millions of dollars. And if S42 is good, then CIG will be funded for the foreseeable future. Of course those are two big 'if's.
this is how i assume they will do it. and the transition happens while in QT travel so that it doesnt impact combat and whatnot. hoping thats the case at least.
Star systems should not be connected anyway...other than to receive info on entities passing between them. As they bring in more star systems, it will be actually easier to have larger numbers of concurrent players as they really won't all be in the same 'instance'(direct interaction)...if you get my meaning.
The issue of course is when you get thousands of players in the same place/zone at the same time. The database transactions will go off the charts. Keeping a handle on that is task for 'Server Meshing'...however they do that? That is a great thing about cloud technology is that Amazon, Microsoft, and Google have HUGE datacenters that can handle however much money in VM"s/Networks/PaaS/Kubernetes, etc., etc., that you can throw at them. DECIDING on what technology(s) to use is possibly the hardest part.
@@cygnus6623 Agreed, Pyro and Stanton should not have to be aware of or communicate with each other. They can be in the same 'logical universe' but otherwise can be completely independent.
But the issue you raise is still a valid one, especially for static server meshing. The problem is that things can become unbalanced, i.e. you can end up with a lot of load on one server and not a lot of load on another. But that's the kind of thing that the parent server tech should address. Dynamic server meshing should address the problem by moving the server boundaries, and/or introducing additional servers, to handle the load. So if there are 1000 players in a universe, and they all happen to congregate in one room like the lower level of the Microtech showroom, then the server should adjust to only handle that one room.
Booting up a new server will be much quicker, as the new server only needs to be updated for its immediate surroundings (entities streamed in), not the whole gameworld. The less a server has to keep track of, the quicker it can be started.
What about the clothing simulation in big groups? I'm not a coder but something tells me this won't go well?
Right now the Entire universe and every simulation, every calculation and all 100 players are being handled by One server. Even having just the replication layer and 1 server will be an improvement over what we have now because the workload will be shared instead of all handled by 1 server.
They might be able to prioritize entities near players to get replicated first as the ‘new’ server spools up. That way the process of streaming in entities from the replication layer will seem faster than 10-30 minutes from the player’s point of view?
Server Meshing is what Worlds Adrift needed, but couldn't accomplish with the small team that Developer had. This success is amazing.
Man, I have felt so doomer about this project for years, but this is amazing!!
I have never been so hyped for SC and S42. Best CitCon ever!
After a Server crash there is no need to repopulate, the replicationlayer is still there. A server basicaly just manages authority at that point, so a server switch shoul be very fast
It still needs all dynamic objects and their states replicated to it. I'm sure it will be pretty slow at first but they will find a way to reduce the initial load in.
I see manny comments here regarding lag between servers and etc. with a little bit of planing the will probably calculate where these server edges. I mean I cannot think these will be dynamically set at any points.
I think the idea is that if a server 30Ks lets say, with meshing one of the other servers would take over rendering right away until the server that 30Ked is back up
Are we still getting static server meshing first or they're going for dynamic server meshing?
Static first. That’s what the demo was.
*servermesh is born and looks at saltEmike*
server mesh: "daa daa"
Is there a direct link to those presentation videos for those who missed it?
They haven’t posted them on YT yet and the twitch stream is HOURS long I’ll update when they post ot
Thank you.@@saltemikereacts
I don't think most people realize what they saw in that demo. CIG has done what most thought was impossible. They've found a way to merge individual servers in a way that they essentially become one big seamless megaserver. Never been done before. The way it's described that tech is infinitely scalable. The only real limitation is the number of servers you have. Let me put it in a way that everyone will understand... With this server tech the Oasis from Ready Player One just became possible... Let that sink in.
It's important to understand the difference betwen a server as an application and a server as a piece of hardware. In this demonstration's context, a server is an application. Multiple server applications can run on a single piece of server hardware.
Why was the best parts of the show the shortest parts?
Greybox also you need to see the zones they have to be visually distinguishable. Also see that those three boxes are not the same size. My guess this super important too.
This is WAAAAAY more impressive than that fancy cinematic they released, but obviously need to see how it works when deployed to live to be 100% this is actually achieving what they believe it is achieving. Just, wowee though, extremely impressive tech demo.
This is live. We have been playing with it since 3.18 came out. They are improving on it and refining, but this IS live...PES = 3.18. Listen to the presenter, he says 3.18 right at the start.
This isn’t live, they said they just got it working 3 weeks ago.@@cygnus6623
@@cygnus6623 it's only "live" if we are playing it right now, this isn't even on the public test yet
This is a milestone in gaming.
The only thing I'm wary of with this demo is latency... as in there is none because the client and all three servers are running on the exact same silicon.
So what happens when all the players and the servers are hundreds of miles from each other and they're interacting at the boundary of a server mesh? If there are discrepancies between client and server, or on this case possibly even server and server, will we just get a ton of rubber-banding?
you basically answered it yourself, they ran it from a single system... servers are not single systems limited by just the towers hardware latency will be much more minimal but there will always be delay as information can only travel so fast between destination.
Answer for 20:23 , the slow speed is because currently the server has to stream every single entity in the entire solar system. a 30K with static server meshing will mean that a 30K server in these containers only has to stream in the entities of that server from the replication layer when needed, because the server that 30K'd would be a small size.
IE.. the entire solar system does not 30K, only a small aspect of it does. The rest of the entities are already streamed to their servers.
You explained it very clean
You won't get huge fleet battles until they get to dynamic server meshing. What they have shown is static server meshing. With static server meshing, each server is permanently responsible for a certain volume of 3D space. If there are more than 100 people in that space, the server will bog down. However, if there are too few people in a 3D volume, they are wasting server resources. So they will probably have something like one or two servers per star system or so, because we know that this is what the servers can handle right now. Possibly they do one server per planet to give us a bit better performance than we have right now. In the future they are going to build dynamic server meshing. With dynamic server meshing they adjust the 3D volume that each server handles in real-time so that it is a region that holds 100 people. That means you can have two rooms right next to each other with 100 people in each and the servers will adjust to have one server per room instead of one server per star system. This will allow an unlimited number of people to be as close to each other as they like because the region for each server will dynamically adjust to be small enough to hold only 100 people. This is the point in time when we can have huge space battles with unlimited numbers of players fighting each other.
The difference between this spinning up and a current 30k server, is the current server has to populate the entire Stanton system but with server meshing each planet could be its own server so it only has to spin up that planet and not the entire system at least that’s how I’m understanding it
One thing I would have liked to see is what happens when they crash the green server what happens when someone is in the purple server and tries to transition into the green server while it’s crashed and spooling up, could someone in the purple server still shoot someone in the green server then they load back in to being killed while they are spooling up the server
He mike. Your concern half way through about the server spinning back up. I think you forgot about the Shards. I think once a server crashes a new one will start running the shard. I remember them saying rn shard and server are the same but in the future they wont be the same thing
19:20 The difference between this and that is that you never actually get logged out because you are on the Replication-Layer. So the question is how long will the lag persist (the time in which you are in a zone without a server running that zone). I honestly can't say that as well (obviously since I don't work at CIG 😂) but I will say that two aspect I think will shorten that spin-up time:
1. If we actually have ServerMeshing the whole point of that is that one server should work on less objects so the amount it needs to load in startup will naturally also be decreased
2. I can only speculate but I think the time will shorten at least a bit because right now the Server also INCLUDES the Replication-Layer which for sure also takes at least a bit of time. I can't really tell how much work starting a Server has to do with the Server (the simulation) and how much with the Replication-Layer (the object-tracking) but I can't believe all of the time is only the server and not also a bit the replication-layer.
I hope this explanation was at least somewhat understandable and reasonable. I think this tech is so interesting and it honestly is one of the reasons I still follow StarCitizen (I am not super old as a backer but I have been with the project since 2018)
LASTLY: For anything who doesn't believe this is real here is a simple explanation why this tech is actually possible and works:
With ServerMeshing we are still playing on ONLY ONE SERVER for the entire universe, only that now this server is the Replication-Layer and not the game-server meaning that the server all clients are connected to (which gives the clients information about the ENTIRE UNIVERSE) is only giving database-entries to clients, servers and the entity-graph (database) which is a very easy and lightweight task. The real game-logic which is very heavy and takes most of the performance is now only applied to specific regions and when we are entering that region your physics and world-interaction is calculated by that server BUT we never actually switch any server-connection when transitioning between zones.
THE PLAYER IS ALWAYS CONNECTED TO THE SAME SERVER
VERY LAST: I THINK I KNOW WHY THE STUDDER HAPPENED. The player JUMPED at the exact point of the transition (which he only did that one time and only that time we actually saw a stutter). My prediction is that this is similar to desync. While he transitioned there was a very short time where he did not move (for the replication-layer) and that was transmitted to the client we are looking through. That client then wanted to smooth over that very small stutter so he walked in the same direction BUT in reality he already jumped. Shortly after that the replication-layer gave that information to your client and it now needed to change what it had shown before (walking to jumping). At least something similar to this is I think the reason that lag was there
Let’s go saltemike sunday
talking about the stutter. like its not still very early in the server meshing tech.
This technology will not change at all how fast a server comes back after a 30k. Mike.
Replication layer: it should only need to "propagate" the entities where players are currently at. In theory you could have a very large area running on a single server and it could have players extremely spread out and in areas with tons of entities, however... a good amount of the time it may only need to stream entities to a server where players are, and being able to push additional entities to that server as needed. "Hey server, you have players at X, Y, and Z, here are all those entities. Ask me when they move to new areas."
Basically: just like how a client only needs its local entities streamed to it, so too does the newly activated server only need those fewer entities. It doesn't (shouldn't) need the entire 700,000+ entities of current day servers to be streamed over before it can start handling gameplay.
Paul's gonna go down as the next John Carmack for this shit
I about broke my TV when Server Meshing worked! I was like "LETS GO!!!" because as soon as they put it in PU I think SC will take off!
That may have been the first SC demo that didn't bug out Live lol
In extremely layman terms, because I saw comments in the chat about now quite understanding how it works.
Think of the scene like a giant table. One server. As of now, it's one table, with one server managing the WHOLE table. The server is the table, so if the server eats shit and dies, the table goes with it. You can get up out of your chair and go sit at a new table but then you'd have to start over until they can bring in a new server to manage the other table, which will then pick up where the dead server left off. But too late - you're already sitting at a new table.
This tech, the replication layer, means that the table is sitting somewhere else being managed specifically by one server. That's ALL that server does. It looks at the state of the table. It doesn't do any work. It simply looks at the table and makes sure everything is in place. Now that that's there, you can have, for example, 3 servers sitting at the table. Server 1 manages purple, server 2 manages green, server 3 manages red. The servers sitting in the zones, or "authoritative" as was called, do ALL the work. They do the calculations; they do all the moving and rearranging and playing. That server then tells the replication layer, "Hey, I did all this nonsense. Update". And the replication like is essentially like "bet. Zone 1 had all this done, zone 2 and 3 are as they were." Now imagine all 3 servers doing their stuff and updating. The replication layer is just getting overarching updates to everything that's already been done. The work has already been processed, it's just been sent up to the replication layer to save it's state. So if a server dies, say server 1, 2 and 3's zones are STILL up and running. While a new server spins up and then asks the replication layer WTF server 1 was doing. Replication layer then just sits new server 1 in zone 1's seat. As soon as the server gets set as authoritative in zone 1, it's ready to pick right up where it left off.
The reason why you can just look across servers is because you're looking at the replication layer. You're looking at the WHOLE TABLE. YOU as a player don't notice a thing. The difference is all back end. Now there are entire servers dedicated to just managing 1 zone of the table that you happen to be in. You can SEE the other zones, but if you don't interact with them in any way then they are simply sitting there in a semi-loaded state. If you throw something across server lines, that object will enter the other server, which will realize something crossed into it do all the calculations (physics, coords, etc) and be like "yo rep layer, I got a Pico that just landed at coordinate x, y, z. Mark it." And the replication layer will be like "bet. Marked and updated the table".
When the whole table eats shit, such as, say, all the server die due to a data center outage, the replication layer has this massive repository of numbers and words, if you've ever seen a .dat file like in Escape From Tarkov where your ENTIRE PLAYER PROFILE is literally a notepad file full of words, values and integers which tell you everything about your player - essentially the replication layer has that. A gigantic repository of files tell it where it needs to put everything so that when the data center is either restored or replaced, it immediately loads a new table again, reads the files, repopulates everything and assigns new managers. Normally that takes time but if you have a whole data center outage.... eh, they'll just kick on the service when it's ready.
I agree with you, Mike. I pledged for the PU. I will play SQ42, but I am passionate about the PU. This presentation was monumental.
Honestly phenomenal reactions agreed 1000%
how many time they show somthing and it never comes out
20:18 replication isn’t anything new. By the time they figure this out, they’re likely going to have an active/passive setup where if one crashes, and their health checks fail, it’ll be damned near instant. That’ll be one of the “easiest” things for them to do.
nah it'll never be instant, look at the current state of the server it has at least 1s - 5s delay. It's like playing on 1hz tick server. This cause fps combat to be janky AF, including the AIs
when benwa stuters comming into the server , nothing wrong with that its the first meshing these things can be fixed the better the tech gets, hell yes it looks real to me boys, excited about this.
The reason it took so long for the server to come back after he killed it was simply that it took a while to re-start the server. If there are multiple servers already running and ready to take over the simulation, it won't even take a noticeable time. The reason it takes so long for you to get back to the same shard in the current PU is because it takes so long for the normal server-rotation restart cycle in the current version before the shard is restarted.
I'd like to assume that the server replication system makes a copy of each currently running server to keep on standby, that is then updated every couple minutes. If the active server is 30K'd, the rep system uses the stored image to create a new one.
Totally a guess on my part, but that would seem like the logical ideal reality for us.
that is probably part of it actually, its a more stable and faster way to take information and actively transfer it to a new server or shard something that other games cannot do as quickly.
EVE is probably the only game where they are able to very quickly transfer that data and during larger scale conflicts things still collapse and break
EvE's solution is 'Time Dilation'. For the player the game goes into slow-motion and for every 10mins that elapses in game - it took 2 hours for those players sitting in front of their PC's. Thats why the Big battles are set at a time both parties agree on - and they notify CCP and request a reinforced server shard in 'solar system x'. The fight may start on Friday evening and rage until saturday afternoon or evening for the players in the battle.. but in game time that fight was over in a couple hours.
Me and a couple friends in the turrets of my MSR - with our arclight pistols on hip, P4 on our backs have more polygons than that entire solar system And the ships in EvE where that big battle is happening and with EvE - its point and click or give a command.. orbit 500, click afterburners, click gun..and the game is doing all the flying and calculations.
In SC we actually pilot our ships so a 20 second delay from you turning your pips onto a target or firing your weps wont work - much less a 4 minute delay.@@yulfine1688
Looks to me like servers are just switching authority and that can be done quickly. All servers knew the positions of entities. Meanwhile entities are stored in the database & on the replication layer. Servers process data from the clients & update the replication layer, which updates the databases. So whole system is decoupled. A server crash might be very fast if a backup server can just gain authority & stream in that locale. Then it’s a case of only giving servers authority on what they can process & what they can load in in a reasonable amount of time.
If they separate like they are showing here they'd only need to load where players are. So 30k and back after a few seconds