Valve's "Secret Weapon"
ฝัง
- เผยแพร่เมื่อ 30 เม.ย. 2024
- 🔴 Get bonus content by supporting Game Maker’s Toolkit - gamemakerstoolkit.com/support/ 🔴
Valve is famous for crafting games that are polished and intuitive. But that's no accident: the developer has a near-religious obsession with playtesting. Learn more, in this video.
=== Before you watch ===
Content warning: Blood, Combat
=== Sources ===
[1] Valve reflects on The Orange Box, ten years later | PC Gamer
www.pcgamer.com/valve-reflect...
[2] Developer Commentary | Portal
store.steampowered.com/app/40...
[3] Portal Beta 2005 & More 2006 Content | Logan on TH-cam
• Portal Beta 2005 & Mor...
[4] Beyond the Box | 1UP
Archive: web.archive.org/web/201105240...
[5] Kim Swift - 'Our Journey From Narbacular Drop To Portal' | IGS 2007
• IGS 2007: Kim Swift - ...
[6] Narbacular Drop | Digipen Institute of Technology
www.digipen.edu/showcase/stud...
[7] Valve's Design Process for Creating Half-Life 2 | GDC 2006
www.gdcvault.com/play/1013237...
TH-cam mirror: • Valve's Design Process...
[8] Valve’s philosophy with User Research... | Steve Bromley
www.stevebromley.com/blog/201...
[9] The Science of Playtesting | GameSpot
www.gamespot.com/articles/the...
[10] Opening the Valve | Eurogamer
www.eurogamer.net/i-valve-060606
[11] Valve Is Finally Ready to Talk About Its Games Again... | The Escapist
www.escapistmagazine.com/valv...
[12] Thinking With Portals | Game Developer Magazine
[PDF] ubm-twvideo01.s3.amazonaws.co...
[13] 126 - Josh Weier Interview | Kiwi Talkz
www.listennotes.com/podcasts/...
[14] The Cabal: Valve's Design Process For Creating Half-Life | Game Developer Magazine
Mirror: www.hl-inside.ru/magazines/gd...
[15] Handbook For New Employees | Valve
[PDF] cdn.akamai.steamstatic.com/ap...
[16] The 100 greatest video games of all time... | GQ
www.gq-magazine.co.uk/article...
[17] The Final Hours Of Half-Life 2 | Gamespot
www.gamespot.com/articles/the...
[18] Why Are Valve's Games So Polished? | IGN
www.ign.com/articles/2009/03/...
[19] Half-Life 2: Episode One Update Released | Steam
store.steampowered.com/oldnew...
[20] How fidgeting playtesters convinced Valve... | PC Gamer
www.pcgamer.com/how-fidgeting...
[21] Christine Phelan - Creating a Nightmare... | Konsoll 2021
• Konsoll 2021: Christin...
[22] Integrating Narrative into Game Design... | GDC 2008
• Integrating Narrative ...
[23] RPS Interview: Portal's Kim Swift... | Rock Paper Shotgun
www.rockpapershotgun.com/rps-...
[24] Meet the minds behind Half-Life 2 | PC Gamer
www.pcgamer.com/meet-the-mind...
[25] The Final Hours of Portal 2 | Steam
store.steampowered.com/app/10...
=== Chapters ===
00:00 - Intro
01:25 - Playtesting Portal
05:47 - Valve's History
08:38 - Playtesting Tips
15:43 - Conclusion
=== Games Shown ===
Portal (2017)
Super Mario Odyssey (2017)
Far Cry 3 (2012)
Ratchet & Clank: Rift Apart (2021)
Portal 2 (2011)
Star Wars Jedi: Survivor (2023)
Narbacular Drop (2005)
Half-Life 2 (2004)
skate. (Unreleased)
Left 4 Dead (2008)
Half-Life 2: Episode One (2006)
Half-Life Alyx (2020)
God of War Ragnarök (2022)
Untitled Magnet Game (Unreleased)
Superliminal (2019)
Viewfinder (2023)
=== Credits ===
Music provided by Epidemic Sound - www.epidemicsound.com/referra... (Referral Link)
Narbacular Drop - Full Walkthrough | COG in the Game
• Narbacular Drop - Full...
Konsoll 2021: Christine Phelan - Creating a Nightmare in Half-Life: Alyx | Konsoll
• Konsoll 2021: Christin...
F-Stop (After-Image) - "lab_intro_scripted2" | fstop
• F-Stop (After-Image) -...
Portal 2: F-STOP PROJECT CAPTURE - Full Walkthrough | Bolloxed
• Portal 2: F-STOP PROJE...
Exposure by LunchHouse Software | LunchHouse Software
• Video
Mirror: www.playground.ru/portal/news...
=== Subtitles ===
Contribute translated subtitles - amara.org/videos/MyGYKu72hcPs/ - เกม
I discovered the importance of playtesting while working on my very own game. You can follow my development journey by watching the series "Developing" - th-cam.com/play/PLc38fcMFcV_uH3OK4sTa4bf-UXGk2NW2n.html
real
Knowing all of this, why did Artifact fail?
Imagine the anxiety when gabe stopped the presentation, then the whiplash when they all got jobs at valve, an absolute dream
"Whiplash" is such a strange word for me, since there's no real equivalent for it in my first language. So thanks for applying it here, giving me another chance of finally wrapping my head around it. (The movie "Whiplash" did very little in that regard for me, but that might have been on me.)
@@lonestarr1490 it's when you are in a car and stop suddenly, your body may be held in place, but your head whips forward,
@@lonestarr1490 whiplash is the very sudden changing of direction either physically or mentally. It's a specific kind of shock.
One would think that a job at valve is heaven, but wheeew. The unwritten part is that expertise and performance are largely irrelevant. Your productivity is controlled by what is allowed by the survivor show-modeled organizational structure. And that one’s career being the same as a contestant on survivor is extremely unhealthy for anyone but managers and their friends. But there are no managers? There absolutely are. Humans will always organize in a hierarchy, and the same is true at valve. Success is being in the right social circle, nothing else.
I just imagine him putting up his hand saying "That's enough." with the whole team just looking at each other confused before he continues with "You all start on Monday."
Kind of funny how Portal, a game all about testing and test chambers, would never have been as succesful as it was without its extreme amounts of player testing.
Y'know what they say, "Art imitates reality"
"Let's do another test... For Science!"
Beep bop... I'm the Philosophy Bot. Here, have a quote:
"For a man to conquer himself is the first and noblest of all victories"
~ Plato
I'm convinced they only made Portal the way it is (focus around testing) because they enjoyed watchig players playtesting their game and making it perfect much like what GLaDOS does in the game.
Ultrameta!
My brother actually found out “players never look up” the hard way - as a player. There’s a temple in Twilight Princess where one of the last puzzles is simply looking up - super easy in hindsight, but it took him over an hour to solve it.
People never look up either
There's a prankster that hung $100 at a college campus free for grabs and it took the students a few hours to see and claim it
Can you elaborate please?
But @@Klipschrf35 is right - people never look up, even in real life. Because why would they? Above us is nothing than a blank sky. There were never real predators coming from above. Humans don't need to look up.
Which temple? I’ve played LoZTP a few times but it’s been awhile.
Water temple miniboss?
This type of development requires a very special style of leadership that doesn't punish their employees when they get negative feedback. If tests are done every week, and every week you hear what doesn't work with your current approach, your boss needs to be fully behind you to consider it a chance for improvement and not a "trick" to point out all of your mistakes.
Fully agree, a lot of leadership actually don’t like this type of stuff since they can’t hide their emotions when they know this is just part of the process since negative feedback create stress and usually make someone vulnerable and people will feel bad often.
@@pencilcheck Mostly insecure people who shouldn't be leaders. But yeah, humans are an anxious species.
@@drakesilmore3760 you are describing almost 99.9 of the current leadership today.
Shame that that version of Valve died after 2013. Since their pivot to free to play the company is defined by its incredible employee turnover.
Not just insecurity, stupidity and trauma projection. Most bosses will just scream at the devs, "I thought you said it was ready" and even when the devs are like "dude you gave us 3 days this is a 4 year dev cycle, we told you this would happen" the manager will hold that against them the rest of their careers, and say the devs have bad attitudes, and if they hired some new people the new people would "do it right". (The new people are Indian and get paid a fraction of what the old crew got paid.)
Playtesting is a Compass, not a Map. It is best used to tell if you are going in the right direction, not what that direction is.
That's a very good way of putting it!
Thanks. As a hobbiest game dev that spends nowhere near the amount of time I want on making the hundreds of incompatible and half formed ideas in my head into a real game, I spend a fair amount of time thinking about this stuff.
my new headcanon is that the "science" done in portal is just in-universe playtesting
Facts
And Glados is the game developers watching the playtesters
It would probably explain why both GlaDos and Wheatley can't help you solve tests. That seems like a sly nod to Valve's game designers painstakingly watching playtesters and can't help them 😂
@@TerranceBhS the euphoria from someone solving a test is just satisfaction that they got your puzzle
I guarantee that several of Glados' mean-spirited quips originated from playtester notes saying something similar. And then they developed her personality in that exact direction because the players LOVED it even more, each week, as the writing iterated through the same process.
Being one of those playtesting supervisors who remembered writing similar notes, and then having them show up as dialogue in the actual builds of the game, must have been a surreal but cathartic experience.
This “antagonist” concept is exactly why a lot of portal-inspired first person puzzle games fall flat and feel lifeless.
seriously I haven't found a single one that is any good :(
The Talos Principle is a game where the (somewhat secondary) story (with an antagonist) started to interest me a lot and become the main reason to keep playing for me
What are the odds I mention The Talos Principle and then immediately see the announcement to its sequel afterwards LETSGOOO
@@joshuam2263 I was actually the opposite: I loved the puzzles, but the story made me too sad to keep playing. That's a hell of an accomplishment!
@@joshuam2263 wait WHAAAAT ?
I have huge respect for the level designers of Portal
I just wish they would make a portal3 or a half life portal combined game where freeman and chel have to work together
Still impressive today 😄
@@IdentifiantE.S7 minutes ago
@@si2foo tbf, didn't Portal's story already end? I can understand Half Life 3 because of the cliffhanger, but Portal actually resolved its story. I don't really see how they could do much else. And a Half Life-Portal crossover just feels too fan service fan fiction-y for me.
@@cloakedhornets6515 pretty sure there's some theory on how they connect since they both open portals to other dimensions
These Valve guys are pretty good, maybe they should make some more games
I know 😫
Right? This whole video makes me sad.
They've probably been working on CS2 after releasing Alyx, you know.
Yeah, like that Half-life series
@@Mate_Antal_Zoltan half life alyx is probably amazing but it’s also kind of lame that their biggest release recently isn’t able to be played by 99% of people
One of the best playtesting stories out there is when Shigeru Miyamoto's son first picked up and played Super Mario 64. Miyamoto was a little concerned with the fact that he wasn't showing interest in going into the paintings of Peach's Castle to go after the power stars, but spent over half an hour just running around and jumping and trying to run up and slide down slopes, walls, and attempt wall kick jumps outside Peach's Castle.
It ironically was a good sign, because as his son and so many other children pointed out in the post-playtest interviews for Super Mario 64, they *LOVED* how well Mario controlled. Miyamoto and his team spending a year focusing on Mario's controls, statistics, and animations really paid off if the simple act of running and jumping around in a 3D game could be fun.
I still remember my dad bringing the game home for an early Christmas present. He's not a gamer in any way, but when he saw someone playing the game on one of those Wal-Mart demo setups he thought it looked incredible. At 8 years old I was right around that perfect age where I was starting to develop the dexterity to play the game. For many years after that it was like a family tradition to bring the game out on Christmas and show off my skills and how they had improved over the year.
Oh man haha, I did the same thing as a kid! I genuinely have more memories about running around outside the castle than any actual map.
@@Cyax0kThat’s so cool man 😄
I heard Miyamoto called his son stupid, when he didn't like the feedback he was giving during feedback
Isn't one of the quirks of their design process that they basically won't work on content until they've stuck their platformer protagonist in a featureless room and polished the movement until it's fun even with nothing to interact with?
For most of the first year of Portal's development, the "actual game" *was* the tutorial. The final game's sequence of test chambers were all initially concepted as being the tutorial section for a much larger game that they scaled back on significantly.
Ohh that explains why it always felt exactly as that lol. Except for the last proper chamber 18 which has every possible mechanic crammed together in a convoluted way, as if they were compensating for the rest of the game that never was :D
@@Tomlacko Chamber 18 is really the only real "graduation chamber" in the game (well, maybe chamber 19 as well.) Even the escape sequence is a tutorial, with escape_00 being a moving panel tutorial, escape_01 being a rocket turret tutorial, and escape_02 being... well the boss fight.
I replayed Portal a few months back and had forgotten you don't even get to place both portals until like two thirds of the way into the game. Which was surreal but it makes perfect sense if the whole game is effectively just a dolled-up antepiece for the final boss.
Yeah, to me it always felt like Portal is a tech demo for Portal 2, but calling it the tutorial for Portal 2 also feels right
@@annakrawczuk5221Except they designed Portal 2 to be playable by people who haven't played the first game. And introduced far more mechanics, which require time to teach. So Portal 2 is also very tutorial-y
Pretty fitting that the developers of Portal also loves testing ;)
Seems like Valve realized this too since they love using Portal to showcase their new tech like VR and the Steam Deck.
It's basically their mascot nowadays
Valve seems to be trying to become Reverse Sega in that they've gone from just focusing on games and creating cool mascots in the process to trying to make hardware while Sega made hardware but is now just a games publisher and developer that has a cool mascot.
@@SimuLord in their defense, it is a good business strategy from an economic standpoint, making videogames is expensive and also very very risky economically these days, especially for AAA studios like Valve. So why risk the cost of making a new game when all they have to do is sit back relax and just take a cut from games people published on Steam and get a crap ton of money.
@@robbieaulia6462 Oh, absolutely. When (not if, when) the AAA industry goes full 1983 and crashes hard (just look at the failure of tons of live-service games; a big recession is going to be a tipping point in the near future), Valve will do just fine because there will still be indie games and Valve will get 30 percent no matter how big the studios making the games people play after the big corporations see their unsustainable investments blow up.
@@robbieaulia6462
Who becomes a game developer to not make games?
I just disagree with this take being the default "wow valve were so clever" moment. I think it's sad they lost their spirit, and it's a cautionary tale not one of some money focused triumph.
They lost to the suits.
@@SimuLord you know thats literally not true? just cus they have not released many games since then does not mean there was none in development, there was literally a lfd3 in development and got really far before it got canceled
When Valve was busy developing CS2 which is coming out soon, they invited bunch of community figures and ex-pros to playtest it at Valve HQ. These people later said that there were dozens of Valve employees watching them play live in the same room. And all the time they were asked feedback and sometimes changes based on that were implemented the same day. This video gave me some background to that story.
SPUNJ also mentioned that the Valve employees would listen and watch, but if you asked them a question, they wouldn't really answer. That alligns perfectly with what was said at 11:10. Really interesting stuff, and knowing the masterpieces Valve has released in the past, this makes me very excited for CS2
Yeah that’s pretty standard procedure for user experience testing in general, try not to influence the result too much with your input. Customers playing at home won’t have access to an expert developer, so you need to pretend your playtesters don’t either.
CS2 sadly feels like they catered too much to those players lol
Valve’s developer commentary is fascinating. Hearing about how they changed the puzzles to make it so players didn’t get stuck but also not easy enough for it to be a cake-walk (pun not intended) is really cool and gives a lot of insight into what it’s like to develop a game.
If only they still developed games.
They're releasing Counter-Strike 2 this summer! It's not really a new game, but they rebuilt CS:GO from the ground up to modernize it and enable its developers to make better updates over the next 10 years.
Well half life 3 should not be that far away, right? *RIGHT?!*
@@EthicalAllele isn't counter strike 2 already out?
Yeah they always blow my mind with their problem solving
@@IndicatedGoodLife I hope so lol
14:03 This is especially shown in the climax of Portal 2.
Realistically, "shoot the moon" is the easiest puzzle solution of all time. It's basically the only interactive option on the screen. But seeing everything fall apart around you, tension at the highest point, and calling back to the source of portal surfaces; that made the climax incredibly satisfying without being hard.
I only just now connected that the ground up moon rocks mixed into a paste is what makes portal surfaces portalable.
@@FlamingZelda3 Bro how
@@FlamingZelda3 I mean, you are literally told that in the game, by Cave Johnson himself
@@LazyBuddyBan Yeah, that made it really cool!
@@LazyBuddyBan hence why my comment was worth making. because something that should've been common knowledge after beating the game twice only just recently clicked.
This game has the most adorable sentry guns ever made.
It's actually kinda mad when that sentence makes people go: yes, yes indeed. You're absolutely right.
Who on earth decided to make sentry guns of all things, cute and lovable? And make you feel sorry when taking them out?
I fear Gabe adds something fishy to the office coffee maker every morning 😆
"Don't drink the coffee! They put something in it! To make ya forget! I don't even remember how I got here!"
Sentry guns were designed to be sold for consumer households according to Aperture
@@fojisan2398 Me when Gaben asks to pick up the can
@@tecinplace1316 It takes a special kind of mind to design those for civilian use instead of military lol. Whoever came up with the concept of Cave Johnson is a genius.
Cute? I found their flat tone and their kind invitations so they can kill you to be warnings of absolute psychopathy. Nothing is more terrifying than playing the game on your laptop at night, and hearing the dying sentrybot say, "I don't blame you..." because they want to devote their dying energy into making you feel guilty whilst pretending to do the opposite.
Maybe it's because I associate them with the 102 Dalmatians game's Water Gun Teddy Bear. "We're best friends! Let's play a game! You need a hug!" they say, but once you are in range, they rip out machine gun fire while yelling, "Yeeeeeeehaaaaaaaaw!"
We'll never stop talking about Portal, and that's beautiful. What a masterpiece.
glad you enjoyed it!
@@AlastairGames are you implying you were one of the people who made portal??
@@albingrahn5576 no, I'm just happy for them!
@@AlastairGames thats nice :)
Mark, I cannot understate how much better your already phenomenally-good videos have gotten since you started making your magnet game.
I am a game designer and graduate of coincidentally relevant DigiPen. The moment you started working on your own project with the goal of not just creating for fun but "finishing" something, I started to recognize the genuine insights of development experience infect your videos. In the past, your videos have always been extremely well researched, well produced, and informative. Something to share with my friends who aren't in the industry but are interested in the topics at hand.
Now, it feels like your scripts are more... substantive is the wrong word. Accurate? Relatable, maybe. They feel like more than just talking points extracted from GDC talks, interviews, and postmortem articles. I wince and groan with empathy at the genuine struggles you face in your Developing series because I and everyone else I know who works in games has hit those same roadblocks. And the non-Developing videos feel like their topics and scripts reach so much deeper into the fundamentals of game design than perhaps they did in the past. A video about the single player level design can be really engaging, but it's not as critical to the job as Playtesting or Prototyping is. I have felt sometimes that topics may have come across more simplified or shallow than I think they could've been, even for a youtube audience full of non-developers. I felt less satisfied about some videos. I no longer feel that way.
You've been killing it recently. Keep it up.
I planned to go for digipen but circumstances changed, can i talk to you! I am going into game design, and want to make more contacts
Testing early is also really important because people are much more willing to criticise something that doesn't look like it's received a ton of love and if it looks like it would be easy to make changes to it at its current stage. This doesn't only work with games mind you, it also works with websites, apps, and pretty much anything you can think of. In university I learned about wireframe designs in web design, where everything looks so barebones you sometimes are even just giving people things written on paper, it really helps the iteration process.
The best (and occasionally the most soul crushing) thing you can do after submitting your game to a jam is checking out streamers who are playing games from it. You can usually submit your game to their playlist and discover all the things that could go wrong, in realtime!
How can you sign up to playtest those submissions to the jam?
@@cantflyforshit Once it is submitted to itch anyone can playtest it. But they might resubmit a final version before the time is up, in which case it seems the old commentary is replaced.
If you want to playtest for a jam that's already over, many of them stay available on itch. I recently played games from GMTC's 2022 competition.
14:00 I thought this was incredible when it happened.
The entire time you're trained on clear delineated puzzles, with black and white panels on walls. I died in the fire a few times despite not having any hiccups playing the main game; the swap from slow cerebral puzzles with clear black and white panels, to a 5 second decision in a flaming garbage pit, felt like one of the best thematic shifts in a game I've ever experienced.
Everything after that _felt_ harder, not because the puzzles were more difficult, but because from exactly that point, on to the end of the game, you were trying to find a solution to a problem that you weren't meant to solve (Yes Valve had a solution in mind, but the theme was that GLaDOS no longer personally placed white tiles to lead you to a solution). All of the garbage cluttered the solutions significantly, and one time I stacked boxes to escape into an air duct rather than the correct solution which was to have the rocket-firing robot destroy the glass. But it matches the thematic shift.
Portal 2 continued right off the bat with that theme.
I've been watching your videos for years and this might be your best work imo. You don't just "still have it", you have improved like crazy. Genuenly thank you for making videos like this.
Thanks!
I realise this comment will get buried, but GMTK is such a great channel/game jam organiser/studio. It’s the only channel that’s felt relevant at every step of learning game dev, from inspiring me to get into it in the first place, to giving me something to aspire to when I started the absolute torture that is trying to learn coding, to giving me experience in one of the GMTK game jams (can’t remember which one), and now as I get ready to release my first game this channel still feels relevant. Thank you guys, and see at GMTKGJ 2023.
Lovely comment, thank you! Best of luck with your game
What's your game? How can I follow it?
@@coffin7904 I've got a couple that I'm doing dual development on, and while both of them have their core game loops I want to get them slightly polished before I take to social media. I'll give you a shout when I'm at that stage, if that's the kinda thing you're interested in.
A note for the very end, Viewfinder is insane. The amount of head spinning you initially got from Portal is there immediately in the demo. Everyone should try that if they even have a passing interest in the Portal series.
I'm developing my own TTRPG, and it's kind of astounding and reassuring that I've come up with a playtest-focused philosophy very similar to Valve's. Being able to play the game twice a week gives me so much information on what systems need to be changed or rules need to be added. This is even more true when I'm playing under someone else as the GM.
If i was playtesting your comment i would be annoyed at not being able to figure out what ttrpg stands for without googling. Gm, isnt that a game moderator? 😏
Just a small rant. Read a 600 page book at work last week, filled with random acronyms i had to repeatedly google as i kept mixing up and forgetting them.
@@emmatitova2154 TTRPG = tabletop role-playing game
GM = Game Master (as in Dungeon Master, but not specific to Dungeons and Dragons)
@@emmatitova2154 Ya know fair enough, I throw around the acronyms so much I forget that people might not know what they mean lol.
@@mtfoxtrot5296 Also fair to talk to your audience. This is a GMTK video; the chances that a person reading your comment here knows what TTRPG stands for are pretty damn high.
This must be the secret to their amazing writing too. They always have something for every possible thing you can do
Interesting to hear that Portal’s final boss design changed from an ultimate puzzle finale to a more story-driven finale. That’s exactly the change I’m thinking about for my game
What game are you making?
What if valve made games?
The world isn't ready for that
What if heavy update?
Specially the ones that end in the number 3
The world is not longer what it was before.
Reality is gonna end itself
It's cool how a games style can build from playtesting and feedback, like Portal being clean and sterile to make puzzles more clear. Or for example Sonic 1's Green Hill Zone having checkered and striped patterns, to make running feel faster as the patterns fly by.
"You could almost think of the player as another designer." That is the best description of what my game design philosophy since studying it in college. A textbook I had emphasized and stressed the importance of regular playtesting, and I've stayed true to that philosophy whenever working on a game. Knowing that Valve does the same explains why I love their games so damn much.
Step one: teach the player c++.
Having a little story while the credits play was brilliant.
Hi GMTK from Vietnam!
I have been a long time subscriber to you. I really love your channel and deep analysis of video game mechanics. You have inspired me to think more deeply about game designs and how much effort goes into making sure games convey the right experiences.
I have recently started a small channel to try to analyze game mechanics for Vietnamese. I am no where near as good or knowledgeable as you, but I am trying!
Thank you so much for your amazing videos, and for inspiring me. I wish you the very best!
Thanks for the kind words, good luck with your channel
Good luck
Yup, playtesting is super important. I was surprised myself how quickly people broke the game I was making and didn't pay attention to things that I thought were obvious. I remember putting a sign that explained how to play, but everyone kept missing it because it just blended with the background.
That was an amazing watch! Comin from the QA world myself I can fully understand their philosophy and it shows in their games.
Funny how you called the work in progress maps with the orange walls ugly, I personally love seeing old builds of finished games and the Hammer engine always feels clean with those height maps written all over the place, it has a strange almost liminal vibe.
Keep up the creativity!
I love that you include detailed sources in the descriptions! Please keep it up :D
Can't believe that a video about the two of my favorite games came out on my birthday. I'm not a game developer but your content is such an entertainment. Massive thank you
I love GLaDOS as a character, to the point that of all the characters I myself have come up with as a writer, the one inspired by her is one of my favorites. And heck, funny enough, I did actually have an experience "playtesting" her, where I found that what I was trying to go for with the personality didn't work. I ended up switching gears a bit, and I'm reasonably proud of where it's since ended up going.
As a person who is interested in writing interesting characters, would it be okay to ask how you exactly "play test" a character?
@@ConcavePgons have people read that character’s dialogue?
@@ConcavePgons Chuck them in a scenario and let your fingers dissociate feom your body so you watch them reacting according to their personality, in text form, live. A good character has escaped your control of them and will do what they do, and all you can do is try to organize them by controlling what the environment throes at them.
@@ConcavePgons playing them in a roleplaying-oriented TTRPG might also work
She gave me a bit of insight in what it might be like to be a woman.
Never had other men call me fat or sth.
7:46 That's not the only reason, it didn't add much to puzzle making and acted weirdly with portals on walls
I appreciate the addition of having content right until the end of the video. I always feel bad clicking off vids when Patreons are being shouted out but I cant sit through random usernames for a minute or more. Nice work.
It's so great to return to these videos - so packed with entertaining information - more to learn even on rewatching.
I strongly believe that this video is one of those who contains the most valuable piece of advice a developer could ever get.
Great video as per usual!
If I can humbly bring a little something I learned being a user researcher who runs playtest at a big studio, I think there's two important elements that might add to what you are saying.
1) One of the biggest issue reducing the usefulness of playtests is the culture/vision the devs have on it. Sadly, devs sometime develop an antagonistic relationship with GUR (Game User Research) due to the fact that most of what will come out of it is gonna be critical. We often refer to our job as having to tell someone its baby is ugly. Ego is at the core of creation and we often see defensivness as a primal response to the results of a playtest. What I see from your video is that Valve seems to have created a great culture around iterating where people don't see "bad reviews" from playtesters as an attack but rather as a problem to solve. In my experience its pretty rare to see this and it is quite commandable.
We also live with pressures for playtests to be marketing tools and it also hinders our process.
2) On the tip 4 "Designer should run playtests", I might push back here, not because I think they shoudln't be involved and that user researchers should be the only people working on this, but rather because I think designers should participate (as observers) but not be responsible of 1) the methodology employed, 2) running the actual playtest and 3) analyzing the data. These three steps demand a specific scientific background to not include biases and draw wrong conclusion who might harm the project. It's a really fine line to navigate and we often see devs who quickly observe without context jump to conclusion due to an anecdotal evidence.
All of this to say that the scientific method applied to make the best use of playtests is often conterintuitive and having specialist with research background is definitely a must if you have the means to pay such an expense.
Again thanks for your work, we often pass around your videos at our job (just so you know :) )!
Being in the process of re-designing a game ourselves, for the DBZ Budokai 4 Project, I can't thank you enough for all these informations.
Also, I can testify how much playtesting is needed in order to advance further in a Project: we got like a dozen of playtesters and they found out a lot of issues my friend and I would've never realized to begin with.
Playtest your games, dudes. It's all worth it.
This was really interesting to view as someone who predominantly plays CS:GO. As valve are currently play testing CS2, this added a lot of insight to their “limited access” approach to a beta. Great video!
I like how some games have used Early Access as a way to get a lot of playtest information for design changes, like Hades did. Though not as early in development as Valve would like, Supergiant's approach to Early Access yielded great results in Hades, and this will likely be the case for Hades 2's Early Access later this year.
However, unlike Valve, they seem to base their early access impressions on freeform feedback in their discord server. This is wildly different from the sciency approach of silent observation, and definitely brings eskewed results, but Supergiant's team handled it very well.
They're my favourite game studio.
Yea I think different methods of gathering feedback isn't as important as Mark's final tip which is the devs themselves need to have a clear vision of what they want in the game and then use playtesting to validate it.
I've always wondered how Valve managed to avoid their games from feeling watered-down despite other games using similar testing methods can come out feeling very *"committee-designed"*...
Turns out blindly incorporating player feedback isn't enough.
14:38 WTF I didn't realize it was you the entire video LMAO no wonder I'm loving this video so much. Omg you're literally the best youtuber ever
Great vid Mark, and tks for the fstop footage, I was looking for that for ages.
i found this super interesting because as a web developer if you asked me what the most important principles to usability testing are in my field i would tell you (almost word for word) the same things
turns out when you want to make sure your design or vision are actually working and not frustrating your users, there's only a few good ways to get that feedback
I work in company software development, and I find the hardest part is to actually find testers… especially clients
The entire thing really reminds me of what I've read about Agile development in the realm of software development.
That being: Part of it is to regularly sit down with the client/end users every two weeks or so, and show them the progress you've made, presumably via live demonstration of new features.
The feedback you get from that then informs you about how you're going to spend the next two weeks: new features, tweaks 'n' whatnot. Rinse and repeat.
That said, I've never seen it in practice. My experience so far is: You sit in a room for months on end, working on half-remembered information told to you by your boss, it's shipped out the door one dreary day, and you never meet the client.
Oh, and Valve's "cabals" also ring true for Agile: Multidisciplinary teams, basically.
The 'Agile' model is a little different to that... it's not about constantly engaging the client/end user in the feedback loop, it's about having a feedback/testing loop _between the teams working on the project_ which enables regular iterative improvements.
It's effective in software development because you can regularly test your code to check that it performs as expected, doesn't have downstream impacts, etc and then make more frequent, smaller changes based on everyone's results.
Your client/users are involved in regular testing & feedback, but far less frequently than the daily/half-weekly/weekly "buzz" meetings between colleagues (which are intended to get everyone on the same page, sharing ideas, tracking progress & determining next steps).
Speaking from experience, the Agile method of project management doesn't translate well into other industries for this exact reason... few have the ability to constantly test & iterate at the rate that software engineers can. When you're dealing with physical products or service industries you need a different (but still critically important) feedback & change management approach... usually with a longer change cycle.
@@medea27 I... am confused by your reply, on how to constructively reply, and I am just generally confused.
My mental curveball is the notion of "having a feedback/testing loop between the teams working on the project".
I am unaware of Agile being used as a discipline for inter-team coöperation on a project (except for the likes of Scrum@Scale, but then only more fleetingly than Scrum itself or Agile).
In fact, what ballpark number do you have in mind when talking about the total number of people working on a project? Mine was 8 +/- 3.
It just occurred to me, you could be thinking of projects that require ~100 people, which would then indeed be split up into many small teams.
I wholeheartedly agree with your third paragraph, and that was my main point: I'm surprised by the amount of end-user testing Valve does, and I draw parallels between this behaviour and that single component of Agile development; of very regular communication with the customer.
This video is a godsend! Im currently stuck with playtesting my game, and this is a best advice i could've asked for!
As you were talking about Glados boss fight, i was mixing it in my head with Wheatley boss fight at the end of portal 2.
And I was thinking, if the ending isnt just the test for if people were actually giving attention to the story, that they are moon stones that you shoot portals at
Not just that. I love the subtle details in that fight - I went in with no spoilers, and at that climatic scene, I was just shooting randomly. I hit the moon, and everything instantly clicked as the hints let me understand what happened.
But in the actual fight itself, I noticed (after re-playing) that to solve the previous puzzle, the player needs to have a portal at the center of the room. Afterwards, no matter which mouse button used, you never erase that portal. The moon is also the only bright object in the middle of the screen, making it an instinctive target to aim at. Wheatley is even screaming about the moon! All in all, I got the right answer because the game basically put me on rails, but I FELT like it was all me/my luck.
Yep when I saw the moon it clicked instantly what they wanted me to do and my jaw dropped in disbelief before I even fired a portal.
Originally, "shoot a portal to the moon" was a secret ending at the middle of the game but playtesting showed that pretty much no one could find it out so it was scrapped. It was only later in the development when faced with a dillema of how to end the game someone remembered this scrapped moon ending and suggested to re-implement it as an actual end.
@@Rhekke Are you just going to forget the Grey Goo that one sprays on a surface to make it portal-placement-ready?
So many streamers seem to completely miss the significance of the moon-rock-dust in the white paint (and its effect on Cave Johnson) because they are so busy being personalities, instead of playing the game.
Probably the best video on agile software development, without mentioning the term 'agile software development' even once.
What crazy idea to release this video before the GMTK Game Jam! It would surely have its fun influence on the Jam! I will be so excited to see many quirky games!
I love this "behind the scenes" type video youve been on
the devs over at id software and the creators of age of empires 2 were also obssessed with playtesting. Sandy, one of the level designers of Doom 2 and Quake, talks a lot about this on his youtube. It's a resource I highly recommend watching, not only as a historical testemony but also because he explains it so well.
I love this kind of history and information! Great stuff! This also helps me out as a software developer to think about the subtleties of human experiences that you might not get from a simple "feedback form"
Great video, buddy! I love learning about the behind the scenes of things that interest me like Valve/Portal.
excellent video! thank you so much for putting this together. love the dev trivia and practical insights ❤
Honestly I think weekly play testing makes a lot of sense - especially since most software development has moved towards Agile.
That opening reminds me of my favorite sequence from an old episode of "Beavis and Butt-head". They're watching the video for the Beastie Boys' "Sabotage", which is shot in the style of a classic '70s movie trailer or TV intro.
Beavis says "This video's gonna RULE when it comes out!"
Butt-head: "This _is_ the video, dumbass."
Beavis: "No way! This is just the preview!"
I cannot fathom the idea that your showcasing just a college project to one of the game developer's ceo in the world, and he just stops you in the middle of your presentation, and offers you a job to work on the game at their company and all of your friends who worked on it as well...
That must be the peak highlight of their career.
Mark, I am so impressed with how helpful and informative your videos are. Thank you for producing these! It's obvious that you put A LOT of preparation and work into them. I used to work at a well-known video game company *many* years ago and I've always been fascinated with the craft required to produce a well-made game. Your videos de-mystify so much of the theory and process that goes into that craft. Well done! You've just gained one more subscriber.
This was my literal experience with portal: "What a cool tutorial, and a great segue from tutorial to game by making me reach the surface. I can't wait to apply this portal stuff combined with weapons here in the actual - ..." credits roll.
the tips from the video:
Tip1: - test as early as possible (9:23)
Tip2: - test often (10:17)
Tip3: - shut up and watch (11:07)
Tip4: - designers should run play teats (12:08)
Tip5: - get the right people (12:53)
Tip6: - challenge your assumptions (13:43)
I was one of the smooth-brains who didn't realize the rocket turret breaking the window in Portal was a hint that it could also break glass tubes that had cubes in them. I still got into the vent... by stacking hard drives and other junk items. Oof.
TH-cam just started recommending portal videos to me and now this gem drops with perfect timing
Valve rarely make games but when they do the game is perfect. It's a totally different feel playing a Valve game, they're just so satisfying
Portal 1 & 2 is an absolute masterpiece. The mechanics are easy to pick up, as the levels get harder they don't feel frustrating yet are challenging and feel rewarding to solve. I think this a the best game you can show to a beginner that's new to video games. I also love how big the portal universe feels because of the hidden lore sprinkled throughout the game with the secret rooms, easter eggs, and pre-recorded Cave Johnson messages, GLaDos backstory etc. It makes the game a little more eerie and really pulls you in because you if you pay close attention you notice there use to be actual people within the aperature labs but it is now abandoned even though on the surface it just looks like a mere puzzle game with a bunch of rooms, some robots and one human test subject.
That swing at GoW! :D i love it, and its so true. At the point they shipped it in, not having the puzzles would be just the same...
Excellent video Mark! Valve will always have a special place in my heart for their commitment to making fantastic games and your video highlighted many things I didn't even know about their process.
Valve sounds like such a great company to work at.
Valve is very unique and does alot of things right in my eyes. I hope they prevail through the ages and not give up on there own path.
I've been waiting for one of these essay videos from you :')
15:25 This reminds me of something I've praised Axiom Verge for - although as a metroidvania the core gameplay loop involves a lot of finding new places, meeting exotic lifeforms, and killing them, Trace spends a good deal of his dialogue trying to ask questions first and not shoot at all until he understands the situation from all sides. He starts every boss fight trying to talk his way out of it, and a good deal of his conversations with friendly (?) NPCs go the same way. So it feels *incredibly* well-earned when he finally concludes he's heard enough.
Play testing is so important to Valve that the plot of portal is literally that you are doing testing
Legends says Gaben still playtesting HL3
What made Portal 1 and 2 such great games is that, despite being puzzle games, I was never once stuck on a level, neither did I breeze through, not even once did I have to turn to google for an answer, because I got sick of a puzzle and wanted to move on quick.
It has a really fantastic gameplay loop, and great boss fights that use everything you've learned during the game.
I work with someone who used to work at valve - he's in the credits for most of my favourite games from Portal to Half Life Alyx. He has shared many awesome nuggets with us and one of the things was how valve ACTUALLY playtests and I'll just be keeping that to myself.
I loved this. I really hope more game development companies can implement this creation method so we stop seeing games that are utter disasters like Redfall coming out.
Great summary of something I didn't really see much: Valve! However, I'd like to stress one important point: While I believe designers might get more from running the playtests themselves, there are multiple reasons why this might be a bad idea (learned at my university and from following how studies are done today).
Firstly, it may impact the moral negatively. That's why you should have at least one person between you and potentially hurtful feedback. Especially if you run playtests online, with possibly anonymous players.
Secondly, it may impact the actual performance and behavior of the players. If they speak with the designers who are directly responsible for the level, they might say something different or think something is "not that bad of a deal" and leave it out. That's why in most playtests (and software tests in general) you have the standard phrase at the beginning of "I'm not a part of the design team." You actually have to stress you have no direct "investment" in this playtest.
On the other hand, designers might not get to see the actual recording of the playtests and I'm not sure that's a good thing (especially if it's muted). But that is a slightly different topic, I think.
You were talking about F-Stop at the end, and I was a little heartbroken. Then you mentioned Liminal, which I've played all the way through already, and it's glorious, and Viewfinder, which I bought recently and can't wait to play. Thanks for the suggestions! There aren't enough Portal-like games in the world.
Thanks a lot for this video. I didn't know this is how Valve does it. Without knowing it, I did much of that myself on my current game - I religiously watched every streamer and youtuber and found SO much valuable feedback just watching other people play (and being unable to interrupt and correct them).
This sort of explains why it takes Valve a long time to make a game. This is a good thing, taking the time to make sure everything is polished to a mirror shine thanks to feedback and talent is what makes game studios like Valve great
Wow definitely learned something new this video.
Had no idea Valve made games!
A novelist once said “while you write, many problems arise, but what matters is only that the solutions be beautiful.” or something like that. So relevant to the creation process.
Excelent video as always. And I personally enjoyed the game suggestions in the end. Should become a habit
Subnautica is a great example of how effective playtesting can be. They initially intended the game to be a relaxing adventure through an alien world where you could scan and read about different species, but during the playtesting they noticed everyone was genuinely terrified for 90% of the game so they leaned into it and added aggressive leviathans to contend with, which are now synonymous with Subnautica's identity.
One caveat to keep in mind with that "Test Often" advice is that Valve has more money than god and can afford to slow down its production to a crawl to playtest everything continually. Testing your game every week makes it incredibly difficult for a team to work on big features that take way longer than a week to prototype and implement. It puts enormous amounts of pressure on devs to bugfix at the same time as prototyping. It eats up a good amount of time each week as well to recruit playtesters, schedule them, watch them play, have meetings about the session, etc. So that frequency is not viable if you're a small team working on tight deadlines.
However it's necessary, otherwise you may end up wasting time on mechanics and features, that will be removed later in development. As main cause of exceeding deadlines is lack of precise directions and goals.
@@ceu160193 No other company than Valve tests that frequently. I wouldn't call it necessary. My point is it's only viable under a strict set of circumstances.
@@Selestrielle Because other companies consider it acceptable, if they have to scrap pretty much finished mechanics and features.
@@ceu160193 They really don't. You have no idea what you're talking about.
Absolutely loved this video, I'm very much into Game development and this video came up on my autoplay. The music, your voice, all of the interesting things you bring up, I love it. I'll be subscribing for sure.
I've been looking for a juicy video about the different types of RPGs (e.g., action, jrpg vs non-jrpg?, tactical?), and what even is an RPG in the first place. Something deep. I believe in you.
Valve's secret weapon:
"We don't fucking make games anymore."
That's Valve's secret weapon.
Or update them.
HLA
@@EmptyMag a pointless tech demo for 1% of 1% who have VR gear.
Call me when it's made into a proper game that the full fanbase of HL can enjoy (note if I sound irritated, it's not at you, it's at Valve).
@@thedungeondelver It is a proper game
@@EmptyMag I respectfully disagree.
Please don't reference the "Offical handbook for employees" as it was manufactured by an employee to give the company a good image. Its content is mostly a sweetened lie for good PR. There is a reason it was "'accidently leaked", and it's because it's not real.
Do you have any sources for this?
One of the most gratifying and interesting little bits of playtesting I witnessed - I was working on some Doom 2 maps I was making as a hobby. Stuck them online for people to try, and people on the Doomworld forums sent me demo recordings of them playing through it.
In one room, I had constructed a sofa. I'd gone to great lengths to make sure the sofa was just big enough and just high enough that it was "sittable"; that the player could move onto it, and for a moment feel like they were sitting on the sofa. It served no purpose whatsoever - didn't unlock any secrets, there was no ammo on it. It was just there, in front of a TV. I put it there for set dressing, and I put it there because I remembered all the times I'd played games, and seen a sofa or chair, and due to dubious collision boxes, had been unable to "sit" on it.
First demo recording I watch, the player looks at the sofa, and sits on it, and watches the TV.
At that moment I realised that we were speaking the same language - that through my level design, I had successfully communicated something and the player had instinctively understood. And it made me realise that this is what I should seek in all my level design and, now that I'm working on my first game, all my game design as a whole. The environments, the way things are presented... it's a language. It's no different to writing a novel; you are trying to communicate something to the reader/player clearly, accurately, and in such a way that they enjoy the experience. You have to be "on the same page"; you can't resent players for doing the unexpected, and you can't blame players when they don't understand what you were trying to tell them. And ultimately the only way you can develop this common tongue, is by seeing how people react to your creations.
It's the same as a conversation - you can only really know that someone has understood what you're telling them, by listening to them in turn; by seeing their facial expressions and body language, and hearing what they thought you meant.
Fantastic video! Testing is key in software but playtesting, getting the right feel is even more so in games.
This reliance on playtester feedback can bite them in the ass though sometimes. Namely how they neutered the Director AI in the L4D games. Particularly the dynamic level pathing in L4D2. When initially designing the sequel they had a very robust AI pathing system where most levels the director could pick and choose from 3-4 paths through the level based on how well the team was doing(short for struggling teams, longer for pro teams). But apparently, lots of play testers found this very confusing on repeated play throughs so they scrapped almost the whole thing. Only remnants of it are in three levels that can change, but are not dynamic(it is randomly chosen when the map loads).
As somebody who played a fair bit on L4D2 I find the fact they removed something entirely based on feedback frustrating. They should have left it as an optional feature or even only usable by the AI on higher difficulties. shame really how that played out.
Similarly, the only reason they retconned the original ending to _Portal_ so that Chell doesn't really get to escape, is because when they were working on the sequel, their dumbass testers didn't remember that ending and were confused as to why they weren't playing as Chell anymore.
The sad thing is that Valve doesn't make more games.
As always, awesome video. I home some developers see your videos and can improve.
It was smart to have more information while the patreon credits roll, well done