This is just my job for the next 20 years, I work in a city government district and we are currently going through 20+ year old code that has only had code slapped onto it. most of the code was from contractors who no longer exist. Its made me realize im a lot better at this job than I ever thought, and that I still have way more to learn.
@@Icewallowcome012 yes and no, I have my personal laptop at work for not work thing and the hours are good because government jobs are very strict on overtime. its better than most jobs, but any full time job will eat up free time.
Thank you so much for videos like this Tim- as someone with a long career in software development, it never ceases to be frustrating how little consideration the average consumer gives to the intensely hard work that goes into making "good" software, especially something with the complexity of a CRPG.
As someone no longer in software dev proper, moved to data science. I respect the hell out of people who do ui and ux. That stuff is so, abstract for a lack of better words. I have a lot of problems with many uis in games but wholly admit I couldn't do it any better.
I'm significantly more frustrated with management who don't seem to grasp the concept of complex things taking time to build, and the only way an "MVP" can be delivered in two weeks is if we'll throw everything out afterwards and start from scratch. The nightmare of my day job dev is even more bleak when I compare it to my personal game where I didn't need to write any code in half a year at this point.
The line about programmers using “logic” to back up things that are really just feelings hit close to home. Programmers sometimes like to think that we are as logical as the computers we work with, but unless your name is commander Data, we’re just as human as everyone else.
@@burgundian-peanuts What? No, I've asked the people I'm talking about for their IQ... Admittedly it's a self reported scale, but it's a random casual observation, I don't need strongly-verifiable proof...
Yup! Again and again I've realized that to ignore human psychology and emotions when solving problems means you are literally ignoring more than half the problem statement It's the Dunning Kruger effect in action to think solely logic will prevail in making a project or task successful. We build things for humans, we are humans, we aren't state machines with clean easy to describe states and transitions
Thanks, Tim. I appreciate that you took the time to unpack some of the feelings-based motivations for programmer-types. Writing software is, by-and-large, a creative act. Programmer-types can act like artist-types more often than non-programmer/non-artist types might assume. In a world where programming is less-and-less understood, I hope that this video will help someone treat a programmer with more understanding.
In my previous job a common reason to go refactoring was usually to make something easier to understand so that we can find security issues. The second most common reason was to get time spent executing down because the code was executing in a factory environment and seconds mattered.
For me the refactor is "maintainability", which is making it more easier to understand and modify, minimize if clauses and that just make bugs and security issues more visible, and gives performance boost because there is less branching and even more important it makes it more visible that software is doing something stupid, so lots of stuff can be just removed. When started programming, I have silly idea of optimization so went from BASIC to deep end, writing assembly and learning what happens on every clock cycle and writing own schedulers and interruption hooks. So I learned things from low level. But I know now that it just better to make application (or game) feature at time and make it work, and then refactor it as maintainable as possible because it gives so much high level performance boosts. Like having stuff working asynchronously so UI is not blocked or minimizing network or disk usage. optimization choosing better algorithms on loops is so easy to do when software works. Understanding that had caused that I haven't experienced performance to be ever problem on stuff what I made or I'm the tech guy making decisions. But working on teams where code is messed up and has started to grow wrong direction on very start, like poorly written unit tests that takes hours to run, software doing simple things can be super slow when problems are in high level.
I have had one occasion where someone, despite my numerous comments explaining parts of the code accounting for edge cases, got the idea of refactoring to… remove all of them. “They are not needed, these cases will never happen, your over-carefulness is the reason the performances are bad.” Turns out, the day after they made their commit, a bunch of bugs get ticketed for that system. ALL the edge cases were present in the game, and his refactoring let them crash everything, AND he didn’t test with the actual data we had… *sigh* in the end, I went back to fix it all and to look at how it could be optimized… and the ACTUAL problem was that the server backend had a bug that made it ignore the actual config to use a hardcoded limit of 500 items in the database and we had hit that limit… (Funnily, later, we ended up hitting the actual hard limit of the database, and I ended up needing to create a “pagination” system to manage that monster… and an indexing system. Yeah! Recreating SQL features in house! /s)
@ Small team, review needed just one reviewer then, a junior that blindly believed the other, more senior programmer, and a timing that meant junior was in a rush and didn’t ask me if it made sense… after that we implemented rules that needed 2 reviewers, AND if you were modifying someone else’s code, they needed to be included in the review process. So the team learnt from it at least. :P
This is probably a top-10 most interesting CoG for me. As a career game tester I’ve heard engineers talk about refactoring many times, but haven’t gotten a window in how, why, and why not to do it. Re: spaghettification, there’s currently a vicious cycle where the person who wrote initial code has been laid off, which means there might be no knowledge of why specific code was written as it was, leading to later code that is spaghetti’d on. Then the new person leaves and the next person doesn’t k ke why the initial spaghetti was made or how the new spaghetti is meant to work, so they add more spaghetti.
"spaghettification" implies the people being laid off are getting sucked into a black hole, and the code is the last imprint of their existence in the company and that's what gets stretched beyond recognition...
Depends on a lot of things. Good code is readable and maintainable. But sometimes code needs to do something particularly complex, work with some arcane nasty library that has a lot of quirks, needs to be extremely optimized (optimization is often the death of readability and maintainability). Also depends on if the developer bothers to document and comment their code (sadly most don't). But most code written in the world is pretty basic application-level code that doesn't do anything particular complex and should be readable and maintainable. Game development - well that's an entirely different beast though since optimization is very important in games.
@@paddyotterness That's not a use of the adapter or facade patterns I'm familiar with. Never seen them used to hide bad code. They are used to make something more generic to increase reusability. The library Tim wrote to handle all DOS calls that was later replaced with Windows equivelants is a good example of the adapter pattern. I am currently designing a facade for a state save & load system so that I can swap out the kind of source/target without the code having to care.
I couldn't have gotten to a Tim Cain refactoring video at a better time. I just finished the Vulkan tutorial, and now I've got to refactor the code, which was written primarily with the tutorial in mind. That means things like reconstructing a struct entirely within a performance-critical loop, even though only one member of the struct changes each iteration; or using the same name for a variable that's instantiated multiple times in different scopes. Some things, I just don't like the way they did it, it's not how I would write it. I have to go through and decide what I should fix and what isn't worth it, and this video helped me think through it a bit. I love your programming videos, please make more!
This was a very helpful video for me. I've been working on a game project since August and can feel the impending need to refactor my input handling. However, each time I consider doing it, I realize that it's working just fine right now and that there are better things to do. Glad to know that I might have the right idea of focusing on more productive things until that refactor NEEDS to be done.
I usually plan in a rewrite from the start. So I can quickly write prototypes of systems that can be iterated on quickly, but then rewriting the code cleanly when assembling the actual game architecture from those separately developed systems. The biggest enemy is complexity in the end. So there is a value in cleanly written code. But cleanly written at the right time.
@@vast634 While I do try to write my systems independently and then connect them later, I'm sure there are many ways I could improve, especially since this is my first project with real complexity. I'm always looking to improve; so, I'd be interested if you have any resources that demonstrate this approach.
@ Its my personal approach. But prototyping is generally a pretty standard approach in game dev. Having the system isolated to its own small sandbox reduces complexity a lot, and allows faster iteration. Refactoring later is much easier once you know the procedures and constraints of this game system.
One thing that crops up: Sometimes the original implementation might have some domain specific logic/structure that is there for a reason but appears hacky to an outside observer
Yes! I wish more devs had more initiative to comment the code with some reasoning. I often see "hacky" code without any comment on the code or on GIT or on Jira, etc. Or, worse, comment explaining the code in natural language (something like "this for-loop adds one to each element").
@@ecosta I have found some funny comments in my old code (talking 25-30 years ago). One comment was "uh oh, I probably shouldn't be doing this". Another one was "COME BACK AND FIX THIS BEFORE SHIPPING!".
@@ecosta Ouch, you hit one of my sore spots. 🙂 I will indeed put natural language comments in my code. The code is the machine-ish readable instructions, and as such a translation of the human language description of the code's purpose. The translation is not one-to-one though. The computer only care about the instructions, so, as you alluded to, the 'reasoning' is stripped away. On the other hand, the computer code cares deeply about the 'how exactly' of the algorithm, where the humans often don't (so, 'something' gets added in the translation process as well). I might put a lengthy code comment such as "The input array contains uneven integers, and to make them even, this for-loop adds one to each element". I might even put still further prose there, with the story about how we used to subtract one, but ran into problems with the "1"-valued elements, and solved the thing by deciding to go with the "adds one to each element" solution instead. At the end of the day, the problem is, that while we probably agree on a lot of comments, there will be plenty with widely varying preferences between developers.
@@CainOnGames Back in university, we had to use UDK for our game project (so it was around the time of Unreal Engine 3 if I remember correctly), and I was working on some code that was overwriting the engine, and thus I was reading the source code to understand the original better… and there was a big 50ish lines section commented out with “removed for E3 2003 demo, it crash, need to un comment and fix later” that was never un commented nor fixed. XD (I think it was reimplemented in a different class.) That… was eye opening for young me… and because of it, I have always made sure to extensively keep track of anything I do like this, by keeping notes yes, but also by always having “TODO” at the start of these comment, and periodically doing a search through all the source code for it, usually at least once before commits.
Thanks! I am watching this x2 speed and making notes. I have in a moment a meeting where I need to explain why we need to refactor a lot of code :D Wish me good luck!
@@CainOnGames Please do not optimize your talking speed prematurely! :) Those, who aren't great in English still can understand you and we, who can and like to, can use the already implemented speed option in the video player! Besides, you have a great voice for explanation. Soothing enough to listen to, but not boring so you don't fall asleep while listening to.
I think after a certain level of experience, you get a bit of a feeling for when code requires refactoring, but my rule of the thumb is, if it ain't broke, don't fix it. You can always make something more extensible, faster, etc, but do you need it to be? Is it a bottleneck? Is it important for the project, or are we spending time on unnecessary refactoring instead of doing something that will push the project nearer the finishing line?
Are you a manager? You sound like a manager. Managers often don't want to spend time on things they consider unnecessary or unimportant, even though they tend to think the core things that make software work are unnecessary or unimportant. This sounds like "YAGNI" which is one of the dumbest concepts I've come across. This perspective is largely responsible for the accumulation of technical debt in projects and, over time, technical debt KILLS PROJECTS. Many projects tend to accumulate technical debt over time until they get so bad that a complete rewrite becomes the only way to move forward and that isn't' a good thing. Spending a bit of extra time here and there cleaning things up and doing them the right way reduces technical debt and improves the quality of the software reducing the likelihood that a rewrite will be required in the future. This is very much short-term vs long-term thinking.
Thoughtful and interesting talk. However, its worth pointing out there's a distinction between Refactoring and Rewriting. A lot of your presentation was really about the latter (and your points in that regard are highly astute). Mixing the two issues is common and unfortunately the distinction is rapidly getting forgotten. To be clear - Refactoring has some important properties. - It is done *continually* - it is not a distinct project phase. You should never make a conscious decision about it, its just a code writing habit. The old Red Green Refactor cycle. - It is very, very small. "Refactoring" projects or a "large scale" refactoring are bordering on oxymorons - it should never have got to this stage; you are now "rewriting". - It is never done without automated tests in place (so avoiding the issue of bugs, mostly). - It only changes the structure of the code, never its function. You are mostly eliminating duplication, making your code readable and adhering to whatever best practices are common for your paradigm. - It is not done with performance or re-use in mind - those may be side benefits, but they are not the primary goal of refactoring. In fact, as you allude to, the best *first* condition for code to be in is well organised - this makes potential optimisation much easier IF its needed. Its a continual, constant activity that prevents code sprawling. Also, its important to recognise that almost all of us write software in a commercial context. Having well-structure code should never be a "discussion" with non-developers in the first place. It puts quality/best practice in direct conflict with commercial pressure - this does not need to happen. Using the "quality" lever when delivering projects should come with a triple rated health warning :) Its better kept set to "high". And, human nature being what it is, if refactoring is not a continual, integrated habit, by and large it will not get done sufficiently to have any real impact - and big rewrites become the norm. All this works *only* because, the red-green-refactor cycle allows you to work more quickly, and thus any perceived "overhead" is more than traded off. Kr Great vids btw...
Cain, i admire a lot your job and your attitude to share knowlege. Even tough im not a game dev, i use your advices to be a better product designer/product manager and i even recommend your channel when i teach at universities here in Brazil I just want you to know that you are helping to change the world with your videos.
Great video! Was really nice to see that perspective on refactoring. One of the biggest things for me in it was that sometimes "going too clever" might hurt more than help, that made me relax a little about how i like to write code. Thank you!
Great video, and really thorough! Love the point about being able to adequately pitch a potential refactor, and how that should show if its needed or not. I've been a software engineer for twenty years now and I've seen people (including myself) fall into the trap of chasing the myth of the perfect piece of code. Not being able to demonstrate benefit to someone can lead to a bit of erosion of trust too, sometimes to the point I've seen pushback on any refactoring at all because there's been no past benefit. Not a situation you ever want to be in!
As someone currently refactoring a huge project I wrote for work months ago, this video is so relevant to me 😂 It was mostly finished, bug free, optimized code. However, due to the time constraints it was made under, it was awful to maintain and extend when necessary. So to improve the "human interaction" factor of it, I've been spending a lot of time working with it.
Just spent two hours this morning because someone changed an enum, whose values were arguably poorly chosen (true = 0, false = 1, unknown = 2). This resulted in friends attacking friends instead of enemies on my branch... The benefit of the refactor is dubious, the need isn't there, it could have been done better. Overall: Probably 3 or 4 hours lost between the guy doing it and my wasting time on it. There are times when it is absolutely necessary to refactor. There are times where it's a luxury you shouldn't indulge in. I completely agree with your points.
Never had any reason to doubt you but it’s so obvious that you really are the hands on guy. Can hear you getting kinda angry about the historical refactoring nonsense. Love it !
Really interesting to hear your thoughts on refactoring. For me, the conversation about refactoring has to be combined with testing. If you don't have the tests how can you know the refactor doesn't make things worse, like you talk about 13 minutes in. I was able to refactor my almost my entire game for all the reasons you talk about with the confidence that my tests would keep me on track. Without that I doubt I would have finished the project or at least introduced bugs that I'd keep finding years into development.
I see your perspective, though I think it's partially due to often working with others (in my professional day job, I never refactor unless I'm rewriting something already), but as a sole developer, I think by nature it's a boon people shouldn't fear, as game design systems tend to overlap in ways that are difficult to foresee, especially for smooth-brains like me. (e.g. equipment touches the player, shops, enemy drops, treasure chests, abilities, combat damage, etc.) It helps so much to have things modularized in such a way that at any given point, if I need to pop the hood open, everything I need is there and I don't have to spend too much time remembering how things connect. More often than not, it's worth taking the time to do so, as it makes progress so much easier, and in development I much prefer repeated code over abstracted functions.
Definitely agree on having to avoid getting too clever with the code! Reminds me of Kernighan’s Law: "Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."
Makes me think of EverQuest. I just logged into a server that launched in 1999. I recently watched an interview with Jchan. She spoke in quite detail how bad the situation was. It obviously more than a refector. It's flooded with bugs but if it isn't game breaking it just isn't worth risking.
It’s an interesting point on moving code to a library so it’s more “modular”. I’ve recently investigated whether Lambda Layers in AWS would be beneficial for my team but the resulting repositories would be too tightly coupled as a result. Difficult trade off to manage for sure!
In application programming and TDD, refactoring is just the final step of the normal development cycle. That's why they say: "The moment you stop refactoring is the moment your code becomes legacy"
Yeah, I started a solo game project barely understanding how to program, and have refactored it over and over like a ship of Theseus. Lots of things are better now, but I do experience all the problems you talked about, and sometimes, I have to admit that I'm only refactoring some portion because I thought of a more "clever" way to solve a problem and felt compelled to implement it, even if it wasn't strictly necessary. I come into this as an artist-programmer though, not as a programmer-programmer, so I embrace the chaos. :D
For me refactoring is a daily thing: I write all my code by refactoring things and injecting new features into the code while doing that. This way the code stays clean and easy to read and it's "easy" to add any kind of new features into the code base when nothing is sacred and any part of code can be changed/touched almost at any time of the project.
As a programmer, I'd like to add a reason why programmers want to refactor: They are embarrassed of other programmers seeing their spaghetti code and judging them. To be fair, they do.
I want to take a minute and highlight what Tim was saying about code reusability. Being good at writing reusable code is a skill and it takes an investment of time and effort to get good at doing it. But this is probably the #1 most useful skill a developer can possess and I highly encourage all developers to spend more time thinking about and improving this skill. Sadly most developers put little though into writing reusable code. I've seen so many development groups where they have numerous developers all doing similar things and don't have any reusable code, everyone is rolling their own unique implementations for the same problems everyone else is also solving. Not only is that a waste of time to write all those various implementations but each has to be tested and maintained separately and often none of the developers are familiar with the other implementations. Tim's example of the single library containing all of the DOS calls making it easy to port the game to Windows is a great example. The thing a lot of developers get wrong is they think they will refactor the code later to make it reusable or they will add-in reusability later if its "warranted". This rarely works out. It takes considerably more time and effort to refactor code later to be reusable often overshadowing the benefit of doing so. Note that Tim says the first thing he did after leaving was to write TIG. Building something like that early rather than trying to do it later is a huge advantage. Writing reusable code is a skill and a change of perspective. A lot of developers will find it difficult at first because it requires thinking differently, but getting good at this really pays off long-term. It takes me literally zero extra time to code things as reusable because I've spent so many years writing reusable code that its my natural go-to perspective. It would actually take me longer to write code that wasn't reusable because I don't even think that way anymore. Reusable code that only ever gets used once is actually fine. Non-reusable code that gets copy & pasted or reimplemented numerous times is spaghetti code. I say this to illustrate what is possible and why I highly suggest developers invest time into getting good at this. You can seriously multiply your performance over time with reusable code and when you get good at it you can multiply the performance of whole teams. Its a very valuable skill. FYI /soapbox
"Tim's example of the single library containing all of the DOS calls making it easy to port the game to Windows is a great example." That was right thing to do it. That was also part of code that was easily recognizable to be reusable. There are others too like handling translations, image/audio/video decoding, subtitles, date/time format, persisting data, memory management etc. We know that they are reusable and need to be handled platform independently so this kind of stuff need thinking about interfaces at an early stage how these should be used. Project specific stuff however is not always clear in beginning what are all requirements.
@gruntaxeman3740 Not really. I've seen lots of software that embeds those types of things all throughout the code instead of centralizing them into a library. Oh, are you a newer dev? Those types of things are often abstracted into libraries NOW. But back when Tim wrote that library that was absolutely not the case. If I recall correctly from his video he got pushback on building that library. Its sad but many developers are their own worst enemies when it comes to code reuse. Often code needs to be deemed special or worthy or surpassing some magic threshold to qualify for bei g reusable. All of which is a terrible way to look at it. All code that can be written as reusable should be written as reusable. With experience it takes very little (or no) additional time to do so. Its rarely obvious in advance what code would be useful to reuse 6 months or 5 years from now. The best way to win is not to play, learn to design and write everything for reusability. I have excessive and extensive experience to back this perspective up btw. I've drastically improved my own performace and the performance of teams numerous times using this perspective. But it does require a change in perspective and practice to get good at designing and writing code for reusability.
@gruntaxeman3740 oh, regarding your last sentance. Reusable code is often more modular and flexible than non-reusable code. So adapting to changing requirements is actually a lot easier when your code is written that way. I can't even count the number of times I've had requirements change significantly that one would expect to be a huge issue but I was able to easily accomodate due to reusable code. In fact, cha ging requirements was one of the thi gs that pushed me to get good at reusable code. Like I said above, its a different perspective.
I've written software past 34 years. Today we have stuff in libraries, 34 years ago we didn't. Back then even the C standard library wasn't fully suitable to reuse in games. Today almost everything reusable is found from libraries and engines and problem is that junior developers add new dependencies to stuff that require 10 lines of code. And then software maintenance goes hard when those dependencies change. Tolerance to add dependencies is lower when making quickly something that works and later throw away but when creating _product_ what is maintained for decades, dependencies are evil and need to be limited. Games are good example. We still expect that game written in 90s can be run natively, on Steam deck. That requires maintenance and that is game developer job keep product working. That maintenance part is what makes difference and inexperienced developers don't understand what dependencies are ok to add. One practice to handle this is framework where is almost all reusable code, and maintenance is moving code base to newer framework versions next 25+ years as there is changes regularly. That is well working practice when creating service what is maintained. And when writing software that lasts 50+ years, that requires different angle to write software. I know how that is done.
@ I absolutely agree with you on developers (sadly not only junior) pulling in lots of dependances often for things that are very simple to code directly. That is garbage. A LOT of open source libraries are garbage and are either not designed, are not reliable or have insanely poor performance. Or they feel the need to update frequently and break existing code. The best reusable code is written in-house specifically to address the needs of that team/group. Over time such code grows to form libraries that offer an array of pre-built and tested capabilities that can be quickly and easily leveraged. Unfortunately there are few software developers who have put the time and effort into getting good at designing and writing reusable code. Writing software that is expected to last a long time without significant changes is also something many developers never think about. I've seen far too many projects need code changes and be redeployed just to deal with a small change in business environment or schedules. I try to write all software to be configuration driven and in the case of business software provide the users with the tools to perform foreseeable maintenance or correction operations. Results in software that works for long periods without needing changes. I try to teach this to other devs wherever I work but there are quite a lot of devs that are closed minded or just do the minimum. P.S. I've been coding for about 45 years.
Strange because had i seen this a month ago or so I would have almost no idea what your talking about. But I recently decided to do a project in Lua as my first language/coding experience. Once i got the code working I began to noticed repeated functions and the like. So i wondered how I could consolidate stuff. Opened up youtube to look up some videos. Boom here is this video hahaha. Perfect timing.
One thing that's immediately clear from the comments is that "code refactoring" can mean somewhat different things to different people… I like the definition at the start of the "Code Refactoring" page on Wikipedia (which to my reading, lines up pretty much 1:1 with what Tim says in the video): "In computer programming and software design, code refactoring is the process of restructuring existing source code-changing the factoring-without changing its external behavior. Refactoring is intended to improve the design, structure, and/or implementation of the software (its non-functional attributes), while preserving its functionality. Potential advantages of refactoring may include improved code readability and reduced complexity; these can improve the source code's maintainability and create a simpler, cleaner, or more expressive internal architecture or object model to improve extensibility. Another potential goal for refactoring is improved performance; software engineers face an ongoing challenge to write programs that perform faster or use less memory. Typically, refactoring applies a series of standardized basic micro-refactorings, each of which is (usually) a tiny change in a computer program's source code that either preserves the behavior of the software, or at least does not modify its conformance to functional requirements. Many development environments provide automated support for performing the mechanical aspects of these basic refactorings. If done well, code refactoring may help software developers discover and fix hidden or dormant bugs or vulnerabilities in the system by simplifying the underlying logic and eliminating unnecessary levels of complexity. If done poorly, it may fail the requirement that external functionality not be changed, and may thus introduce new bugs." I also particularly like this, from the same page, a little further down: "Refactoring is usually motivated by noticing a code smell. For example, the method at hand may be very long, or it may be a near duplicate of another nearby method. Once recognized, such problems can be addressed by refactoring the source code, or transforming it into a new form that behaves the same as before but that no longer "smells". […] Failure to perform refactoring can result in accumulating technical debt; on the other hand, refactoring is one of the primary means of repaying technical debt." Would you agree, Tim? Did your code ever seem smelly? 😊
knowing when and how to refactor is a skill that comes with time and experience. I personally wouldnt consider myself great at it but I always try to get low hanging fruits based on simple rules I picked up. Often its enough to have a final look at what you've written in total and then you see how you could combine or simplify a task. If its used a lot, it will be noticeable. Most of the time it doesnt require 200IQ hacks.
I'm refactoring my game Nodrog's Fortress for my enhanced edition release this summer, and your thoughts about where extra cycles might be available really helped. Also, finding spaghetti code and figuring out a better path to execute has been a fun nightmare.
I had a colleague who was obsessive about code refactoring (No, I wasn't entirely sure what the term meant until I watched this video), and he made some really amazing stuff. Of course, that stuff was all he made... I regret that we didn't get to complete our project (simply because the majority of the work needed hadn't been completed), but I did learn a lot from what he was doing (which, of course, was learning in his own right). Also, side note: I'm fanatical about garbage collection. It's amazing how many programmers haven't heard of it, and don't understand how to cater to it (i.e. avoiding memory leaks).
Coming from a slightly different perspective, as a sysadmin I often need to write bits of code for one thing or another. Often I'll need to do something "quick and dirty" to get the job done, with the intention of refactoring it later for eventual and inevitable re-use. Sometimes that happens, but all too often it doesn't. Something always seems to come up that needs more urgent attention. What I've learned over the years is to *at the very least* put comments on everything, even if it seems silly. Refactoring can come later if there's time, but without those comments you'll never remember why you needed to process something in a very particular way that seems like (or may in fact be) a hack.
My team's programmer has gotten a lot better about avoiding needless refactoring, we've settled on taking note of areas for improvement to do them better next time, and then refactoring only as required.
As a full stack developer I follow the adage 'always be refactoring'. Since the code I work on is constantly being maintained over years, it is a necessity to keep things from getting too confusing and out of hand. This is because the overall system/requirements have changed so much over time. Anyways, I find it interesting how refactoring would be done differently/less frequently in other fields.
My question for Tim: Does test-driven development have a place within games development? Assuming that Tim doesn't pick up the question from here, I'll put it to the comments: TDD in web development seems to be both looked on immensely favourably whilst also frequently being poorly understood. I see people in web development who criticise TDD and they are invariably citing examples of either problems they do not know how to effectively mock, or else describing solutions that TDD exists to highlight their flaws. In games development, I spoke to one fellow hobbyist who stated that though he used TDD in web development it was inherently impossible in gameDev, and that really only end-to-end testing could be used. At the time I accepted his advice. Since then I recently got a game 90% of the way to MVP only to discover the remaining 10% of bugs would be near impossible to find, and what I in fact had was a prototype. I rebuilt it using TDD and got a great deal of value out of doing it that way. Now I am able to find bugs much, much quicker and every time I come to a problem spot it is invariably because I stepped off the TDD pathway and into the wild. What thoughts do others have?
Problem with tests is maintaining tests. And truth is, UI code that handle layouts, it is not worth to write tests because they live so much. Instead it is better idea to optimize code maintainability and have type systems or domain specific languages. There are parts of game code that are not just logic stuff, making code that controls how character jumps and how it _feels_ . Maintaining those as tests are just overkill. Test rooms works better and manually testing. But backend and just logic part, it would very dumb to not write tests and ignore proper TDD benefits and that is true in game development too.
Earlier today I was going through my rebuild, and your opening line was, at least superficially, true. Tests built earlier in the rebuild were no longer applicable. The question came as to whether or not to maintain them, and I chose not to at this stage for much the reasons you say. But I would stress, that the purpose of TDD - and the main misunderstanding about it - is not about having good test coverage, it's about designing from the perspective of tests. In my instance, even though the individual tests weren't no longer providing good coverage, building from them just meant the code was very nicely decoupled and easy to bugfix. I have never tried TDD in the context of the sort of feel-based content you describe. I admit that I find it hard to envisage, although then again I spend almost all of my gameDev in strictly turn-based systems so I'm already at a disadvantage with that.
Yes, TDD is not about coverage. It is about thinking requirements and writing test for the requirement. It is not unit testing, TDD works on.. "module level". I can strongly recommend TDD when requirements are known. It is that in reality requirements are not always known and developer need sometimes explore how to make stuff working. Coverage is important on refactoring and maintaining code, when changes can be made easily and it shows immediately if something is not right.
I don't understand how this isn't the same as just testing if the code works after compiling it, in regards to game design? I mean, I recently wrote a little plugin for my browser where I had an ideal situation it would work in, which I used as a basis for testing if it worked, sure, but... I don't see how that's different from loading up a sandbox level in a game and testing a feature. Is it just, specifically, coming up with a test first, with the test being the goal? E.g. there's a box in a distance, when a ball is hit from a certain point by a certain object, it should hit the box and the box should be deleted?
It's very important to separate refactoring from rewriting. Refactoring is a mechanical activity that shouldn't introduce any logical changes, only structural ones. It's the kind of thing that can and should be done with the assistance of tools. The moment the changes stop being structural and become logical, then you're in the realm of rewriting, which is where things get more dicey.
That is refactoring, but altering how the code is implemented to make it more flexible, more modular and to allow the addition of new features is also refactoring but isn't a "mechanic activity" as you describe. Some refactoring is simple enough that it can be done by automated tools, but that's not the limit of what refactoring is.
I would argue refactoring that can be done with automated tools is not the kind of refactoring we are talking about here. The key insight is that most projects have ill defined requirements and actually through the process of creating the project, you discover all of the requirements. Usually large refactoring is about redesigning a project using hindsight as a guide. Feature requirements you didn't expect but had to be hastily bolted on can be now be supported as a first class citizen. This requires a robust test suite to be able to guarantee that during your refactoring you do not regress features you know are important but aren't the focus of improvement of the refactor
My approach is to try to make the new module have as little influence on the rest of the code as possible and create some bounds in which I can write the worst code imaginable. The first iteration will always be bad no matter what, so it makes sense to try to decouple it so the module won't force extra work later. Refactoring comes along with bugfixes/optimizations/writing code that relies on that module, because it introduces new context and makes it clearer what you need to do with refactoring. The time spent on refactoring this way can be accounted as time spent on fixing the actual bug/writing the new module, so it's also a way to hide the refactoring/tech debt work from the management :) In the end, the tech debt is mostly gone and the next iterations take much less time. If we have time to spend on making the project correct and robust. The gamedev specific thing is that you have less time and resources to work on these things, so it might not be possible to do proper refactoring in some cases. The code has capacity to have a certain amount of quick dirty fixes before it's a mess, sometimes it makes senes to just throw in a flag and an if and forget about it. The problem is when it becomes a developer culture, I'm currently working on a project when the whole chain of developers starting from Unreal Engine devs (or the C++ commitee...) has failed to come up with sane methods of writing networking-related code (readily available in almost every other language's standard library), trivial changes take literal months. I'm tired, send help. But I agree with you, good refactoring often requires extra context, otherwise it's mostly a whiteboard masturbation. I sometimes hear critique of Unity ("the developers didn't make a single game, therefore they don't have the experience to make the engine good") or Unreal Engine ("they only made Fortnite and that's why the engine is only good for Fortnite and not as a general-purpose game engine"). This observation might be more fundamental than just refactoring.
In the case of Unity/Unreal, they are somewhat held by backwards compatibility. You just cannot refactor your API if you have customers who would also have to rewrite all their code if you changed yours. This results in Unity throwing all kinds of variant render pipelines while keeping the old one for instance.
A fourth reason for code factoring is because major feature is required and the existing code is to unwieldy and/or unclear for anything other then minor bug fixes. And the cost of a entirely new code that produces the same or more correct answers not only costs millions and will take a decade along with the validation required by the customer costs millions but the budget is much less.
I should take your advice on pushing the refactoring further. I am not yet prioritizing game development yet, because I have other hobbies, so usually I do something in the project for 2 weeks, then I leave it be for 2 months😅. The problem for me is that when I come back to the project, I take a look at the code to familiarize myself with it again and notice that there is something "wrong" in how I'm doing things. But that occurs, because I still learn better coding practices during those 2 months when I'm away. Maybe someday I'll forgive my earlier self and move on to the next system😅
In my experience the most common reason for refactoring code is to add new features & capabilities to an existing, functional system. In fact its often difficult to get approval to spend time refactoring code just to make it cleaner, more optimized or more reusable without being able to tie that work (and thus those hours) to a new feature or capability. This somewhat depends on the industry the code is for. Most commercial and enterprise systems (sadly) don't really care about performance or optimization anymore. Which is why you'll be interacting with some employee of a company and they'll have to apologize for how long their computer is taking to do something (something that should probably be nearly instant except that most code is poorly written). In game development, however, optimization is much more important and its one of the few niches where performance is still tracked and management cares about it.
One related question I have is about the prevalence (and usefulness) of automated testing, continuous integration and other standard "quality assurance" processes. Is this something people are doing and if so, how is it similar to/different from the practices seen in other domains? Is even worth doing for games? I'm not a professional game developer but I've heard very mixed opinions on it so far. I imagine having to deal with frequent design changes might make this a nuanced discussion. In my experience, there's many areas in game development that are exceedingly difficult to test, such as rendering code or interactions with platform-specific APIs (e.g., console SDKs). On the other hand, a lot of utility code or libraries which are more general might be easy enough to test that the benefits outweigh the cost?
I've never been on big game production, but currently smaller one, and I'm sure that these kind of processes are widely used, but...... Like I mentioned here in comments, automated tests are not always good idea on UI side like vector based games or UI layouts. Automated builds for every platform are for sure useful. Test rooms and testmaps are used a lot in vector games. Simultaneously it feels like the gaming industry is very backward in some respects. Like, there are still game titles that don't run natively on Steam deck/Linux, and I found it very odd. Building phase in software is not hard. One huge difference in my understanding is play testing. Like, put people play game, record it and learn from that "is it fun" or "does players stuck on somewhere too much" I hope Tim will someday make video where he goes through some practices about this.
In the overall tech industry? Yes, people are doing CI and unit-testing (and TDD/BDD (test driven development / behavior driven development) to a much lesser extent). Though, a lot of companies say they are doing it, but fail to understand how to unit test most effectively. Or just don't have a contradictory organization structure to support it (like having a QA team separate from the dev teams). Do game developers do it? Not so much afaict. I believe the game development industry has less people doing it and not that many people discussing it at all. One such obstacle is the importance of UI in video games as one person already suggested. UI is difficult to test. You can't unit test UI as easily because you'll just find that you're testing the underlying framework or tool it's built on (which the devs for that tool should've already done), and it's difficult to write integration tests with it, but you can do some high-level snapshot tests to automate it a bit better, but it still could involve manual intervention, so it's not perfect. You should also decouple from the UI, making sure to unit test the APIs/data it needs to use rather than how it gets displayed when unit testing. The goal isn't 100% coverage, but building trust between the different parts of your technologies. In things like rendering, you might want to test the underlying rendering/graphics engine, to ensure the math is being done right. The trick is remaining decoupled and making your tests decoupled and not dive too deep while providing enough confidence to let you understand what your changes accomplish.
@@gruntaxeman3740 I think some notable games that might get close to having good tests are those with great tooling. Warcraft 3 for example had its amazing editor tools. I doubt that it was bug-free, I'm not sure that any game has been bug-free, but I do believe that it's a step in the right direction of testing just enough. If you focus on testing the tools, you might be able to gain a lot of confidence in the underlying game itself.
It’s nice when coding teachers will give u extra points for being crafty with how u sort your projects. Sometimes it can be like 1 step forward 2 back but I swear going back and re learning stuff only solidifies the ground u walk on. Once you learn about classes and objects it can be really nice u can store so much data n basically make entire organized systems that run on a few lines. Where yah I think having clean code is nice but it’s almost not what you think clean code is just like writing a word like appleSauce vs AppleSauce to save your finger the strain n yah just show u still divided a word for the reader 9/10 it’s like good punctuation n grammar in writing where u put the curly bracket } is another one. However some of it is about how fast you can make things run optimization is name of the game.
This is very familiar to the advice given in "The Mythical Man-Month" by Frederick Brooks almost 50 years ago! My feeling is that even though there has been change in large software design since then the broad stumbling blocks from early computing still happen today? Would you agree with this?
One thing I've been doing recently is refactoring (or factoring) a lot of the code in my projects to try and consolidate all the potentially buggy code in one big quarantine zone. Maybe that'd fall under maintainability but I like to think of it as proactive bug hunting.
Completely off topic: What was your favorite dark ambient music back when you were super into that? There’s soo much of it now, but in the 90s-early 00s it was much more niche. Was checking out a live stream where you said the music for Fallout was dark ambient because you handed the composer a stack of dark ambient CDs; as a composer for media, I’m just curious what albums those were.
Hi Tim. Excellent video and I agree, but I like to add and clarify something from my own experience: 1. Refactoring for performance. Just don't do it if there is no reason. Or if refactoring for optimization is done it should be very last phase of development. 2. Refactoring for maintainability. You can't refactor enough. Refactoring for maintainability improves _almost everything_ . Performance, portability, memory usage, readability... everything is improved. And that matters a lot because most of time, developer is reading and understanding code, not writing it. Code should be treated in way that when there is mess, then clean it. Formal way to describe "refactoring for maintainability" is about minimizing "if" clauses and improving how easy it is to read and understand. But there is one important thing where premature refactoring for maintainability doesn't work: There are more than single dev, (or tightly working team) and they are repeating stuff. That thing should NOT be refactored because developers usually don't really know exact requirements what they need, and sharing code causes that they need to adapt that later so the shared code easily is conflicting with requirements. When all developers / teams have done all features ready, that repeating stuff from each module can be refactored away when developers truly understand all requirements that shared piece of code must handle. 3. Refactoring for reusability That late stage refactoring of common repeating code is good candidate for that. Sure the when designing architecture some low level stuff can be actually see early phase that it can be reused. --- Also refactoring should not almost never break the code. Related to code maintainability and polishing it, it should have also proper tests for every feature before doing on those late stage refactorings. If refactoring cause new bug, then just add test that triggered it so it never pops up so changing code pieces to more efficient or sharing those across modules don't break things. I'm aware that UI or similar this is not always good idea, and maintaining tests may require much more work than maintaining just code.
I have to disagree on some of that. Refactoring for maintainability rarely improves performance, often it degrades performance. Depends on the details and the degree of optimization, but optimized code tends to be harder to read and maintain where code optimized for readability often performs slower than less readable alternatives. Also portability, when that is a factor, often leads to less readable and maintainable code since it may be necessary to use ifdefs/conditionals or similar to have specific code for various platforms. Most software doesn't need to be portable though and in those cases portability should be completely ignored. I don't understand what you're saying about "repeating stuff", are you talking about code reuse? Code reuse is the #1 best thing a developer / team can do to improve their performance (them accomplish more in a period of time) and improve the quality and reliability of code. Code reuse should be done early when possible, not later. Trying to refactor code to be reusable after the fact is extremely time consuming and error prone and often people say they are going to do this but never do. Spending a bit of time to think about and plan the code before writing it such that it is designed to be reusable and modular is a huge boon to projects. I've personally 5x the output of entire teams on multiple occasions through good code reuse. But writing good reusable code is a skill and it takes time and effort to get good at that skill. Sadly few developers put much effort into getting good at this. However, it is quite possibly the most valuable skill a developer can possess as it will greatly increase their productivity and the quality of what they deliver.
Performance issues are almost always caused if software is doing something stupid. And when refactor code to be easy as possible to read and maintain, it is easier to spot all those parts in code that are doing something stupid. It is not then changing some loop to work more efficiently. Then it is about removing whole code away. If there is code that can be identified early phase to be reusable when designing architecture and see that is cross cutting whole project, just gather requirements for that, write tests to them and make interface and simplest version that work. Then mark it with comment that this need to be written on proper algorithm later. Optimization is done later. Code reuse is for sure best thing what team / developer can do to improve performance but I mean situation when there are multiple teams or some reason team is not "tight" on communication. Like team blue is doing feature A and team red is doing feature B, and they can share stuff. Problem is that teams probably don't know exactly all requirements so if they refactor prematurely common code away, they very easily lead up to situation that team red is making tiny change to it so it behavior can cover everything module A. And then the behavior causes bug on module B what team blue is working. Teams may also end up to do some messy hacks to make their features working. Proper way is that both teams complete their module and they know exactly requirements on the code that can be shared. Solo dev or closely working team can reuse code better but having other team working on other country, that is not same thing. It is also known that software architecture resembles organizational structure. They are so equal that designing software architecture is also making decision what teams and people do what. That is also reason why it is very important to define interfaces between modules in way that they don't change. What I mean is that writing code is easy and simple. What is hard is to know what to code. Requirements are not always clearly known and there may be exploration to make it work. Developer just don't get exact specification what to do because the exact specification is the code what developer is writing. Refactoring working code when features are known and proper tests are written is actually simple. Many architecture level decision are better to do right in beginning. There should be a lot of thought involved.
@gruntaxeman3740 Strongly disagree that the "PROPER way is thay both teams complete their modu,e and they know exactly requirements on code that can be shared". That is completely backwards. Though lots of developers think that way which is why I regularly 5x most developers. Doing code reuse after the fact is extremely wasteful. Writing reusable code is a skill that takes time to get good at. Once mastered it is fairly easy to design code, interfaces, libraries and frameworks that are highly reusable without most of the effort you suggest. The best practice isn't to spend time trying to identify cross-cutting concerns, the best plan is to write most code as reusable. This is the biggest mistake most devs make, they think code needs to surpass some magic threshold to be written as reusable. Well over 50% of all my code is reusable, always.
"That is completely backwards" It is, but it is faster if all requirements are not clear what is very common. Remember that software developer job is to understand requirements from stakeholder and write it to formal language that computer can understand, and requirements do very often clarify during development. So if there some module written by outsourced team from other country, time is very easily wasted on communication, coordination and later realize that previously written /common module didn't work on both features. And the important thing: writing code is fast. Debugging, trying to understand some spaghetti or debugging, that takes time. So when requirements are not clear, just let both teams work independently and when features work, common stuff can be later refactored. You are talking like writing code is slow and that is big effort but it is not. About 30% of time goes to understand requirements. Implementing something what is known is fast.
"Well over 50% of all my code is reusable, always." I've written currently backend to one game, and when excluding database, language runtime, e2e tests and OS.. 99,12% of code is assembled using prefabricated, reusable components from framework. So yes, this is using framework approach for maintenance. All game code in backend is very project specific so they are not reusable.
Love to see dev stories that are similar to my non-gaming career! My two cents: I think code refactoring as a reader-targeted optimisation. Then, as with all optimisations, you have to ask if it is worthy - just like Tim described!
Some advice from my schooling that I've kept around over the years: the only way to write code with no bugs is to not write any code at all. i.e. every line of code written has the potential to have a bug in it.
Great video Tim, thanks for sharing. I always wondered what engineers meant by this. I have noticed on many projects there tends to be friction between some disciplines.. do you have any tips or advice on how artists and engineers can better work together? Does having tech artists usually alleviate this?
I have a related question: I'm applying to internships right now-for the purposes of projects on a portfolio, is it wise to refactor code, specifically to reduce spaghetti code, even if it doesn't make sense to refactor it in practice for that project? (i.e. spaghetti was added because features were added, but the code works fine). I would maybe assume its helpful because of readability but curious what you'd think.
Wanting to refactor when you can't is like wearing a coarse wool straight jacket on a hot humid day while attending the world lawnmowing championships and you have moderate to severe hay-fever.
This is why I try and follow SOLID principles (especially single responsibility principle) as early as possible. If your code is properly separated and put behind simple interfaces, it's much easier to refactor it without affecting other parts of your code base. Get your interfaces right early, and refactoring the actual implementation later becomes a non-issue.
The biggest problem I've seen with large refactors or re-writes is people running headlong into Second System Syndrome. Programmers will often think about the new code, without thinking about how they will replace the old code with the new code, or what the customer experiences in the interim. I've toyed with the idea of not _just_ refactoring code, or making version 2 of a product, but also writing a program that sits above both versions of the product, checks what features are implemented in the new version, and dynamically switches customers from the old version to the new, depending on what features they actually configure. Unfortunately, it's something I could only do as a hobby myself, since when I explain the difficulty of doing it, people either prefer to just have one product and stall feature deployment for months, or decide that a refactor is altogether too much work and stick with an older product. Of course, I'm not proposing a refactor for no reason: usually, the reason I'm recommending it in the first place is because the current code base causes developers to take days or weeks on tasks that I know they can do in hours.
Maybe too broad a question but I wonder how your approach to game dev changes if you're doing it specifically to learn and improve your skills (Be it programming or designing) versus creating a product at a company! Love your videos :)
You might enjoy my videos about making a little “toy” game for fun. This is the final video: Toy Post Mortem th-cam.com/video/kVV_2Yy_aXY/w-d-xo.html Its description contains links to the rest.
the problem with "Code Refactoring" is that it doesn't have a clear definition on what it is actually about. i mean it as in is it: - abstracting the code? - making the code DRY? - restructuring the entire feature/code setup including all classes? - making the code "Clean"? - cleaning the just written code up so that the variable and function names adher the agreed syntax and adding comments? (many in this comment section see that as refactoring) and so on. what i read way too often about refactoring is that the goal is to abstract and DRY the code to make it more readable and reusable. the problem is that it both is often WAY too early abstraction (and unnecessary) and the attempt to make it DRY leads to very big if else chains or switch cases to handle the different places that aren't supposed to use the same function but do. how much code are we changing in a Code Refactoring? is it only a function? is it a class? is it a feature? is it the entire file structure and what code is in what file? is it going through existing code and removing inbetween step variables that don't really need to exist (expect for ease of debugging). does that already count as refactoring? does refactoring mean you need to write the new code version from scratch with a new approach? on that node. i haven't found a clear definition of "spaghetie code" either. if cross file calls count as "spaghettie" then 95% of game dev code is inherently spaghettie code. it's near impossible to write the code in a way that doesn't require that you need to go through multiple files if you change a function name of a class object that get's used more often or interacted with in multiple places.
For most, it is fairly clear. I feel you're overthinking the purpose of refactoring. You certainly mention some important parts like DRY and restructuring, but focus too much on large changes. Many also do it iteratively, so you wouldn't do an entire class in one fell swoop. Here is a bullet point list that describes what is a refactoring and how it differs from other changes you make in code. Feature Added: Description: Introducing new functionality. Impact on Tests: Requires new test cases for the added functionality. Likelihood of Breaking Changes: High, especially if changes affect existing APIs or dependencies. Feature Removed: Description: Eliminating existing functionality. Impact on Tests: Requires removal or updates of related tests. Likelihood of Breaking Changes: High, as dependent code or integrations may break. Optimization: Description: Improving performance or resource usage. Impact on Tests: Existing tests should still pass; may require adding performance tests. Likelihood of Breaking Changes: Low, as long as external behavior is preserved. Refactoring: Description: Restructuring code internally without changing behavior. Impact on Tests: Existing tests should still pass; no new tests needed. Likelihood of Breaking Changes: Very Low, provided functionality remains unchanged.
@retagainez I understand that there can be a "clear" definition but everytime I hear someone is "refactoring" they are talking about different things and scales. That is what my comment mainly is about. Just "I need to refactor this" doesn't tell anything about the planned actions the dev wants to do because it could mean 10 minutes of work or 3+ weeks of work. Most of the time just changing a functions internals to be cleaner isn't seen as refactoring but just as cleaning or optimizing the layout. Too often refactoring is seen as complete rewrite and days to months of work when you talk to the managers. If it only takes a few minutes or an hour then the managers likely will never hear about it.
@@dovos8572 I blame that on poor knowledge of the definition of the word "refactoring" and the history behind it. Even this video misuses it. Cain talks about optimization, not refactoring. Also, refactoring shouldn't be a task on it's own. It should just be a part of writing tests, writing code to pass those tests, and keeping things tidy once it's done. It should be a part of ordinary coding. If you have some work that will take 3+ weeks of work, that's more of an issue about the code being difficult to change. It COULD be difficult to change because the team doesn't refactor or test their code at all. It's difficult to refactor because it's not designed well (because it doesn't get tested). Note the emphasis on code design, and more importantly test design, designing good tests so that your code may follow.
Refactoring code is a tricky subject. Never refactoring leads to both messy code and a tendency to "gold plate" your code - trying to make it perfect the first time out. Constantly refactoring is a waste of time. Here's my general rules. First, don't worry about getting your first pass of the code perfect. However, follow the boy scout rule. Every time you touch the code - a new feature, a bug, etc - make it a little better. Does the code not work for the new feature? Refactor it. Was the source of the bug a design flaw? Refactor that design flaw out. I find this to be a good middle ground. It avoids gold plating, it avoids constant refactors, and it avoids your code turning into complete trash. In my experience, as long as you're not shipping in the next couple of months, it speeds you up rather than slows you down.
Do not underestimate code cleanness. I abandoned my first project (a card game), because over the time I improved a lot, and couldn't stand to deal with the entire code base. Not that I was so bad at it, but my style changed a lot, and I stopped using bad patterns (yes there are bad patterns). My current big project is very well coded. I do refactors solely to improve the logic, especially when it's old code, or when it needs to be reworked. For me quality code is very important. It's not even just about performance (although that's important to me too), but I'd like to not hate my code and be proud of my work.
Understandable but its important to be careful about this. People can fall into traps where they spend a lot more time being very precise and clean with their code and not end up delivering anything because it took so much time. Balance is needed in such things. I'm a professional software architect and I very much like well design, modular clean code but sometimes less that perfect code is good enough and not worth spending more time on.
When I don't know ahead of time exactly what code I'm going to write, so it involves some degree of improvising, I'm *very much* guilty of the sin of immediately wanting to refactor once I get it to work properly.
On importand consideration in your sorted array insert example that you didnt mention is the average and maximum lengths of the array. Bubble sort and insertion sort are faster than quicksort for very short arrays. This is tge kind of thing that lots of newer computer science grads might not think about because how much emphasis CS puts on asymptotic time and space. For small n frequently something dead simple with bad asymptotic growth is much, much faster.
Hi Tim! Do you agree that colleagues are not friends? In my opinion, people can become friends, but only after they no longer work, study, or depend on one another in any way. Where do you think the line is drawn between friends and colleagues? Do you think it’s a good idea to work with friends? If it is, then how do you manage potential conflicts or maintain professionalism in such relationships?
I don't know about American work culture but that's definitely not true in Europe. Most of the friends of my colleagues, are other colleagues. It's natural.
I became friends with number of colleagues during my years (I work in IT for more than 25 years). I have a friend whom I started working with very early and multiple times we asked each other to come to work together at our next company multiple times when we left. Besides the usual social engagements like parties and holidays together, etc. I made zero friends when I worked in the US though (and I was there for 10 years). It's a US thing I think, here, in Europe, we are friendlier.
Your question: Do you think it’s a good idea to work with friends? The answer is yes if you consider your computer a friend. In most cases it (or he) is very reliable. It is there for you 24 hours a day, 7 days a week. Most of the time he doesn't complain and if he does, he has a good reason for it. He has no big demands on you, answers all your questions, does not shout at you and always plays with you if you want to play a game. The only thing it lacks is an endoskeleton.
Hi Tim. Would more people enjoy shows that repeat the same dialogue again for the rest of your evening because the protagonist constantly died? Speaking on single player open world type rpg campaigns, Do gamers have such lack of awareness they need a game over screen to inform them? Can simply halting progress and being outmatched serve as your tell? Should game designers deliberately make obtuse game mechanics to get players who look up guides to make more posts in a hack attempt to increase engagement in the gaming community?
Do you know where the Living One is supposed to have come from? I know a lot must be left to the players imagination but they seem to come from some completely different continent. Sorry if you've been asked this before.
This is the stuff of my damned living-nightmares as a line-of-business software-engineer. So much Spaghettification from how much Scope-Creep my non-programmer bosses keep piling-on to projects that were engineered for a specific and defined set of requirements, which ends up resulting in having to make large-scale refactors of both code and _database schema_ on a regular basis. Bonus Points when something is implemented *to the letter* of the requirements-documentation I was given, and then don't like the thing that does exactly what they claimed they wanted, either during QA, or even better, *after* _they themselves_ approved it for Prod.
Customer often have vision or concept. Then when asked "how authentication should work or what user roles should have and how UI should look on different places" they don't know anything. To actually launch business it needs that person who actually interpret that vision and make something that works and give value.
Eh sometimes but not always. The best most optimal systems are architected from the beginning to be modular and anticipate the later needs of the project. This is effectively what a good, knowledgeable software architect should be doing for a project. But like many things this is a specific skill that requires time and experience to get good at and many developers never even try to get good at this. Possibly why there are so few good software architects.
@@Me__Myself__and__I I don't think those things are *necessarily* at odds. I just think the vast majority of programmers tend to design more flexibility than will ever be called for, and overestimate the cost of changing small rigid systems, and underestimate the cost of changing large overly complex modular systems when their modularity doesn't actually match the way the problem changed. I will also say that in games, you rarely know what changes will actually be necessary. And the parts you know for certain tend to be part of what most developers call an engine. Games are also incredibly high performance pieces of code and abstraction tends to have real cost, so you simply cannot abstract everything. Which all adds up to real benefits of solving problems as small as they actually exist, until they are actually known to be bigger problems. And when you build that small, if you were wrong, throwing out tiny solutions is trivial and didn't waste any time.
@ Don't agree. Most programmers don't build any flexibility or modularity into their code, the problem isn't devs building too much. The rest of that 1st paragraph is a skills issue, that just means those devs don't have enough experience designing good modularity. I've been building modular systems from my own reusable code for many years now. On every single project its more than paid off. Most systems gain tech debt and cruft over time as they are modified and changed eventually leading to a require re-write. None of my systems have had that problem. Even systems I didn't design but took over and slowly converted into being modular improved and become more stable with less tech debt over time. I literally have decades of experience showing that this approach works very well. But it does require skill and knowledge that takes time to develop - but only if the dev is actually trying to get better at reusability and modularity. Lastly the game engine thing, absolutely not. Much of what is very useful to build using reusable and modular code is not included in game engines. An engine pretty much just takes care of the presentation layer and the input. Engines don't include game mechanics, characters, skills, items, inventories, NPC AI, etc.
Yeah, (many/most) programmers like to live in the technical bubble, where you don't have to think of production or product realities. Also, probably don't want to think there is anything to refine in their methods, because they are basically Spock.
Most of the programmers don't write game code. Best practice for business code is to do absolute minimum amount of features that brings value to customer, and just make that minimum amount of code to good. If it brings value, software has future and more features can be added. And possible this is the most tricky part in game development, that game is often "single release" software, where tons of features should be done and working on release. That is also the problem on game development because games are often poorly maintained. Related to refactoring and polishing, I'm skewed to direction that while we have now digital distribution and easy way to update, game can be released when all of the features are complete and there are proper test suites that allows easy refactoring. But In my point of view, that should not development. Game code can be refactored after that, continue polishing and do optimizations, add features like graphics card specific stuff. But when refactorings that require all knowledge from people are done, game can be moved to "maintenance mode" when there is single dev keeping it working small effort, like moving it to newer language versions, newer runtime versions, porting to new OS/platforms, newer engine versions.. Maintenance should not stop for release because all code in the game will rot away, transforming it garbage if no one is taking care. I believe that game shelf life is somewhat increased so older game releases can generate revenue, but in my understanding most of the sales do happen on first 6-12 months. I think game developers could think ways to keep product on peoples mind longer, like keeping noise on making releases to new platforms, or some cases think game in way that it has sequels based on very same code.
Perfectionism can be its own masochistic hell, trying to make the best thing ever without first profiling to see if it's necessary. Especially with all the critical testing to make sure it works against deadlines with all the time it takes to rewrite. Part of vision is seeing the real priorities. At the very least always back up in case the older version is better lol maybe easier said than done when dysfunctional pride and time is attached.... don't waste time in the first place.
Thank you for the video it was interesting. 4:40 These pause menus where work is supposed to be done only work well if it is a single player game. However, if you want your game to be multiplayer friendly from the start, you usually cannot pause the world calculation. So if a player opens a GUI inventory in multiplayer, for example, the game world cannot be paused. It must continue to run. Could make another video about parallel programming and concurrency? 15:21 🖖👽
@@LDiCesare I'm not talking about pure multiplayer games, but rather games that are for single player and multiplayer. For those, you'll want to have as similar a code base as possible, because redundancy only costs work and money and there's a risk that you announce multiplayer as feature for a game, but then only finish the single player part and then have to say that you can't do the multiplayer part because it would be too much work and the code would have to be changed too much. And in this case, that means that you run the server on the client computer when the player wants to start a single player game. Thus client and server code is running on the same machine. If you want to avoid OS specific task switches, you could run both in the same process, but these are details that I don't want to go into here. The only thing that is relevant here is that the code from both then runs on the same machine.
I disagree on the "don't refactor once you're done" stance. I do it almost systematically and often ask my team to have this step as part of their process. You do a first draft that is exploratory, you encounter bugs you would have not thought of before, you see issues with your logic, and you end up with a great knowledge of the problem you are solving. You've also defined a testing process for your code (usually using unit tests but I don't see you mentionning it so I'm guessing they are not used during the projects you've worked with which seems plausible considering when some were done). You can know do something to the quality standards you expect to deliver without anything left from the exploratory phase. I would also argue that refactoring is a skill that has to be trained. I've seen developers that can't refactor well and some that could refactor a huge part of the codebase efficiently without any issues. The more you refactor, the more you get a grasp on the consequences of refactoring, the more you can refactor or get a feel on the refactorability of a piece of code.
Is someone paying you to refactor the codebase after you've shipped? 😊 (Bear in mind Tim's talking directly about games development, which generally aren't very long-lived products like an operating system, an office suite, etc…)
"Is someone paying you to refactor the codebase after you've shipped?" Should. Code will rot to garbage if not maintained. When code is purely in maintenance mode, this is very trivial: 1. Identify change points 2. Find test points 3. Break dependencies 4. Write tests 5. Make changes and refactor This is the basic routine to move code base to newer OS, newer runtime, newer language version, newer engine version, shovelling of some garbage library and so on.
This is just my job for the next 20 years, I work in a city government district and we are currently going through 20+ year old code that has only had code slapped onto it. most of the code was from contractors who no longer exist. Its made me realize im a lot better at this job than I ever thought, and that I still have way more to learn.
Does that job eat up a lot of freetime? Just curious
Good luck, champ!
@@Icewallowcome012 yes and no, I have my personal laptop at work for not work thing and the hours are good because government jobs are very strict on overtime. its better than most jobs, but any full time job will eat up free time.
I would write a proper comment, but I have to finish refactoring this code first...
Refactor this comment please.
// TODO : Comment this comment
@@ouiVEVO 😂
Unless you are refactoring it to self-document!
Thank you so much for videos like this Tim- as someone with a long career in software development, it never ceases to be frustrating how little consideration the average consumer gives to the intensely hard work that goes into making "good" software, especially something with the complexity of a CRPG.
As someone no longer in software dev proper, moved to data science. I respect the hell out of people who do ui and ux. That stuff is so, abstract for a lack of better words. I have a lot of problems with many uis in games but wholly admit I couldn't do it any better.
I'm significantly more frustrated with management who don't seem to grasp the concept of complex things taking time to build, and the only way an "MVP" can be delivered in two weeks is if we'll throw everything out afterwards and start from scratch.
The nightmare of my day job dev is even more bleak when I compare it to my personal game where I didn't need to write any code in half a year at this point.
The line about programmers using “logic” to back up things that are really just feelings hit close to home. Programmers sometimes like to think that we are as logical as the computers we work with, but unless your name is commander Data, we’re just as human as everyone else.
I think there's an IQ range (I'd say 115-135) that I've seen this rationalization very frequently in.
@@GhassanPL So you can evaluate IQ without administering a professional test? That's awesome!
@@burgundian-peanuts What? No, I've asked the people I'm talking about for their IQ... Admittedly it's a self reported scale, but it's a random casual observation, I don't need strongly-verifiable proof...
Yup! Again and again I've realized that to ignore human psychology and emotions when solving problems means you are literally ignoring more than half the problem statement
It's the Dunning Kruger effect in action to think solely logic will prevail in making a project or task successful. We build things for humans, we are humans, we aren't state machines with clean easy to describe states and transitions
This isn't a programmer thing IMO, everyone does that at all times
Thanks, Tim. I appreciate that you took the time to unpack some of the feelings-based motivations for programmer-types. Writing software is, by-and-large, a creative act. Programmer-types can act like artist-types more often than non-programmer/non-artist types might assume. In a world where programming is less-and-less understood, I hope that this video will help someone treat a programmer with more understanding.
In my previous job a common reason to go refactoring was usually to make something easier to understand so that we can find security issues. The second most common reason was to get time spent executing down because the code was executing in a factory environment and seconds mattered.
For me the refactor is "maintainability", which is making it more easier to understand and modify, minimize if clauses and that just make bugs and security issues more visible, and gives performance boost because there is less branching and even more important it makes it more visible that software is doing something stupid, so lots of stuff can be just removed.
When started programming, I have silly idea of optimization so went from BASIC to deep end, writing assembly and learning what happens on every clock cycle and writing own schedulers and interruption hooks. So I learned things from low level.
But I know now that it just better to make application (or game) feature at time and make it work, and then refactor it as maintainable as possible because it gives so much high level performance boosts. Like having stuff working asynchronously so UI is not blocked or minimizing network or disk usage. optimization choosing better algorithms on loops is so easy to do when software works.
Understanding that had caused that I haven't experienced performance to be ever problem on stuff what I made or I'm the tech guy making decisions. But working on teams where code is messed up and has started to grow wrong direction on very start, like poorly written unit tests that takes hours to run, software doing simple things can be super slow when problems are in high level.
I have had one occasion where someone, despite my numerous comments explaining parts of the code accounting for edge cases, got the idea of refactoring to… remove all of them. “They are not needed, these cases will never happen, your over-carefulness is the reason the performances are bad.” Turns out, the day after they made their commit, a bunch of bugs get ticketed for that system. ALL the edge cases were present in the game, and his refactoring let them crash everything, AND he didn’t test with the actual data we had… *sigh* in the end, I went back to fix it all and to look at how it could be optimized… and the ACTUAL problem was that the server backend had a bug that made it ignore the actual config to use a hardcoded limit of 500 items in the database and we had hit that limit… (Funnily, later, we ended up hitting the actual hard limit of the database, and I ended up needing to create a “pagination” system to manage that monster… and an indexing system. Yeah! Recreating SQL features in house! /s)
How did that get past code review??? That's wild
@ Small team, review needed just one reviewer then, a junior that blindly believed the other, more senior programmer, and a timing that meant junior was in a rush and didn’t ask me if it made sense… after that we implemented rules that needed 2 reviewers, AND if you were modifying someone else’s code, they needed to be included in the review process. So the team learnt from it at least. :P
This is probably a top-10 most interesting CoG for me. As a career game tester I’ve heard engineers talk about refactoring many times, but haven’t gotten a window in how, why, and why not to do it.
Re: spaghettification, there’s currently a vicious cycle where the person who wrote initial code has been laid off, which means there might be no knowledge of why specific code was written as it was, leading to later code that is spaghetti’d on. Then the new person leaves and the next person doesn’t k ke why the initial spaghetti was made or how the new spaghetti is meant to work, so they add more spaghetti.
"spaghettification" implies the people being laid off are getting sucked into a black hole, and the code is the last imprint of their existence in the company and that's what gets stretched beyond recognition...
@@minerman60101 lol. True, that is the common use of that term.
Depends on a lot of things. Good code is readable and maintainable. But sometimes code needs to do something particularly complex, work with some arcane nasty library that has a lot of quirks, needs to be extremely optimized (optimization is often the death of readability and maintainability). Also depends on if the developer bothers to document and comment their code (sadly most don't). But most code written in the world is pretty basic application-level code that doesn't do anything particular complex and should be readable and maintainable. Game development - well that's an entirely different beast though since optimization is very important in games.
@@minerman60101 I knew I'd heard that term somewhere. I'm not mad about this outcome.
@@paddyotterness That's not a use of the adapter or facade patterns I'm familiar with. Never seen them used to hide bad code. They are used to make something more generic to increase reusability. The library Tim wrote to handle all DOS calls that was later replaced with Windows equivelants is a good example of the adapter pattern. I am currently designing a facade for a state save & load system so that I can swap out the kind of source/target without the code having to care.
This is a top tier Tim Cain video
Tim "Deckard " Cain
"Stay awhile and listen!" xD
I couldn't have gotten to a Tim Cain refactoring video at a better time. I just finished the Vulkan tutorial, and now I've got to refactor the code, which was written primarily with the tutorial in mind. That means things like reconstructing a struct entirely within a performance-critical loop, even though only one member of the struct changes each iteration; or using the same name for a variable that's instantiated multiple times in different scopes. Some things, I just don't like the way they did it, it's not how I would write it.
I have to go through and decide what I should fix and what isn't worth it, and this video helped me think through it a bit.
I love your programming videos, please make more!
This was a very helpful video for me.
I've been working on a game project since August and can feel the impending need to refactor my input handling.
However, each time I consider doing it, I realize that it's working just fine right now and that there are better things to do.
Glad to know that I might have the right idea of focusing on more productive things until that refactor NEEDS to be done.
I usually plan in a rewrite from the start. So I can quickly write prototypes of systems that can be iterated on quickly, but then rewriting the code cleanly when assembling the actual game architecture from those separately developed systems. The biggest enemy is complexity in the end. So there is a value in cleanly written code. But cleanly written at the right time.
@@vast634 While I do try to write my systems independently and then connect them later, I'm sure there are many ways I could improve, especially since this is my first project with real complexity.
I'm always looking to improve; so, I'd be interested if you have any resources that demonstrate this approach.
@ Its my personal approach. But prototyping is generally a pretty standard approach in game dev.
Having the system isolated to its own small sandbox reduces complexity a lot, and allows faster iteration. Refactoring later is much easier once you know the procedures and constraints of this game system.
One thing that crops up: Sometimes the original implementation might have some domain specific logic/structure that is there for a reason but appears hacky to an outside observer
Agreed. That's a great place in the code for a comment!
Yes! I wish more devs had more initiative to comment the code with some reasoning. I often see "hacky" code without any comment on the code or on GIT or on Jira, etc. Or, worse, comment explaining the code in natural language (something like "this for-loop adds one to each element").
@@ecosta I have found some funny comments in my old code (talking 25-30 years ago). One comment was "uh oh, I probably shouldn't be doing this". Another one was "COME BACK AND FIX THIS BEFORE SHIPPING!".
@@ecosta Ouch, you hit one of my sore spots. 🙂
I will indeed put natural language comments in my code. The code is the machine-ish readable instructions, and as such a translation of the human language description of the code's purpose. The translation is not one-to-one though. The computer only care about the instructions, so, as you alluded to, the 'reasoning' is stripped away. On the other hand, the computer code cares deeply about the 'how exactly' of the algorithm, where the humans often don't (so, 'something' gets added in the translation process as well).
I might put a lengthy code comment such as "The input array contains uneven integers, and to make them even, this for-loop adds one to each element". I might even put still further prose there, with the story about how we used to subtract one, but ran into problems with the "1"-valued elements, and solved the thing by deciding to go with the "adds one to each element" solution instead.
At the end of the day, the problem is, that while we probably agree on a lot of comments, there will be plenty with widely varying preferences between developers.
@@CainOnGames Back in university, we had to use UDK for our game project (so it was around the time of Unreal Engine 3 if I remember correctly), and I was working on some code that was overwriting the engine, and thus I was reading the source code to understand the original better… and there was a big 50ish lines section commented out with “removed for E3 2003 demo, it crash, need to un comment and fix later” that was never un commented nor fixed. XD (I think it was reimplemented in a different class.)
That… was eye opening for young me… and because of it, I have always made sure to extensively keep track of anything I do like this, by keeping notes yes, but also by always having “TODO” at the start of these comment, and periodically doing a search through all the source code for it, usually at least once before commits.
This channel is awesome. Appreciate all the wisdom and knowledge transfer Tim!!
Thanks! I am watching this x2 speed and making notes. I have in a moment a meeting where I need to explain why we need to refactor a lot of code :D Wish me good luck!
Good luck! I will try to talk faster on future code videos! :)
@@CainOnGames Please do not optimize your talking speed prematurely! :) Those, who aren't great in English still can understand you and we, who can and like to, can use the already implemented speed option in the video player! Besides, you have a great voice for explanation. Soothing enough to listen to, but not boring so you don't fall asleep while listening to.
@@CainOnGames No need! Next time I'll start earlier or move the meeting :)
listening to videos at higher than 1.0x should be a war crime
^ fortunately we're not at war
I think after a certain level of experience, you get a bit of a feeling for when code requires refactoring, but my rule of the thumb is, if it ain't broke, don't fix it. You can always make something more extensible, faster, etc, but do you need it to be? Is it a bottleneck? Is it important for the project, or are we spending time on unnecessary refactoring instead of doing something that will push the project nearer the finishing line?
I worked like that and it turned me into a master-chef and connoisseur of bug-free, feature-complete, unintelligible spaghetti 😅
Are you a manager? You sound like a manager. Managers often don't want to spend time on things they consider unnecessary or unimportant, even though they tend to think the core things that make software work are unnecessary or unimportant. This sounds like "YAGNI" which is one of the dumbest concepts I've come across. This perspective is largely responsible for the accumulation of technical debt in projects and, over time, technical debt KILLS PROJECTS. Many projects tend to accumulate technical debt over time until they get so bad that a complete rewrite becomes the only way to move forward and that isn't' a good thing. Spending a bit of extra time here and there cleaning things up and doing them the right way reduces technical debt and improves the quality of the software reducing the likelihood that a rewrite will be required in the future. This is very much short-term vs long-term thinking.
Yeah this works until you have to change the code (which in games is pretty often)
Thoughtful and interesting talk.
However, its worth pointing out there's a distinction between Refactoring and Rewriting.
A lot of your presentation was really about the latter (and your points in that regard are highly astute).
Mixing the two issues is common and unfortunately the distinction is rapidly getting forgotten.
To be clear - Refactoring has some important properties.
- It is done *continually* - it is not a distinct project phase. You should never make a conscious decision about it, its just a code writing habit. The old Red Green Refactor cycle.
- It is very, very small. "Refactoring" projects or a "large scale" refactoring are bordering on oxymorons - it should never have got to this stage; you are now "rewriting".
- It is never done without automated tests in place (so avoiding the issue of bugs, mostly).
- It only changes the structure of the code, never its function. You are mostly eliminating duplication, making your code readable and adhering to whatever best practices are common for your paradigm.
- It is not done with performance or re-use in mind - those may be side benefits, but they are not the primary goal of refactoring. In fact, as you allude to, the best *first* condition for code to be in is well organised - this makes potential optimisation much easier IF its needed.
Its a continual, constant activity that prevents code sprawling.
Also, its important to recognise that almost all of us write software in a commercial context.
Having well-structure code should never be a "discussion" with non-developers in the first place.
It puts quality/best practice in direct conflict with commercial pressure - this does not need to happen. Using the "quality" lever when delivering projects should come with a triple rated health warning :) Its better kept set to "high".
And, human nature being what it is, if refactoring is not a continual, integrated habit, by and large it will not get done sufficiently to have any real impact - and big rewrites become the norm.
All this works *only* because, the red-green-refactor cycle allows you to work more quickly, and thus any perceived "overhead" is more than traded off.
Kr
Great vids btw...
Cain, i admire a lot your job and your attitude to share knowlege. Even tough im not a game dev, i use your advices to be a better product designer/product manager and i even recommend your channel when i teach at universities here in Brazil
I just want you to know that you are helping to change the world with your videos.
Great video! Was really nice to see that perspective on refactoring.
One of the biggest things for me in it was that sometimes "going too clever" might hurt more than help, that made me relax a little about how i like to write code. Thank you!
I value your content and enthusiasm, Tim. Keep up the good work.
Great video, and really thorough! Love the point about being able to adequately pitch a potential refactor, and how that should show if its needed or not. I've been a software engineer for twenty years now and I've seen people (including myself) fall into the trap of chasing the myth of the perfect piece of code. Not being able to demonstrate benefit to someone can lead to a bit of erosion of trust too, sometimes to the point I've seen pushback on any refactoring at all because there's been no past benefit. Not a situation you ever want to be in!
I like to start my day with like 20 minutes of refactoring just to get in the programming mood and get warmed up.
As someone currently refactoring a huge project I wrote for work months ago, this video is so relevant to me 😂 It was mostly finished, bug free, optimized code. However, due to the time constraints it was made under, it was awful to maintain and extend when necessary. So to improve the "human interaction" factor of it, I've been spending a lot of time working with it.
Just spent two hours this morning because someone changed an enum, whose values were arguably poorly chosen (true = 0, false = 1, unknown = 2). This resulted in friends attacking friends instead of enemies on my branch... The benefit of the refactor is dubious, the need isn't there, it could have been done better. Overall: Probably 3 or 4 hours lost between the guy doing it and my wasting time on it.
There are times when it is absolutely necessary to refactor. There are times where it's a luxury you shouldn't indulge in. I completely agree with your points.
false = 1 is definitely poorly choosen.
Never had any reason to doubt you but it’s so obvious that you really are the hands on guy. Can hear you getting kinda angry about the historical refactoring nonsense. Love it !
Really interesting to hear your thoughts on refactoring. For me, the conversation about refactoring has to be combined with testing. If you don't have the tests how can you know the refactor doesn't make things worse, like you talk about 13 minutes in. I was able to refactor my almost my entire game for all the reasons you talk about with the confidence that my tests would keep me on track. Without that I doubt I would have finished the project or at least introduced bugs that I'd keep finding years into development.
I see your perspective, though I think it's partially due to often working with others (in my professional day job, I never refactor unless I'm rewriting something already), but as a sole developer, I think by nature it's a boon people shouldn't fear, as game design systems tend to overlap in ways that are difficult to foresee, especially for smooth-brains like me. (e.g. equipment touches the player, shops, enemy drops, treasure chests, abilities, combat damage, etc.)
It helps so much to have things modularized in such a way that at any given point, if I need to pop the hood open, everything I need is there and I don't have to spend too much time remembering how things connect. More often than not, it's worth taking the time to do so, as it makes progress so much easier, and in development I much prefer repeated code over abstracted functions.
Definitely agree on having to avoid getting too clever with the code! Reminds me of Kernighan’s Law:
"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it."
Makes me think of EverQuest. I just logged into a server that launched in 1999. I recently watched an interview with Jchan. She spoke in quite detail how bad the situation was. It obviously more than a refector. It's flooded with bugs but if it isn't game breaking it just isn't worth risking.
It’s an interesting point on moving code to a library so it’s more “modular”. I’ve recently investigated whether Lambda Layers in AWS would be beneficial for my team but the resulting repositories would be too tightly coupled as a result. Difficult trade off to manage for sure!
Loved the perspective Tim.
In application programming and TDD, refactoring is just the final step of the normal development cycle. That's why they say: "The moment you stop refactoring is the moment your code becomes legacy"
This video was a great watch! Thanks!
Yeah, I started a solo game project barely understanding how to program, and have refactored it over and over like a ship of Theseus. Lots of things are better now, but I do experience all the problems you talked about, and sometimes, I have to admit that I'm only refactoring some portion because I thought of a more "clever" way to solve a problem and felt compelled to implement it, even if it wasn't strictly necessary. I come into this as an artist-programmer though, not as a programmer-programmer, so I embrace the chaos. :D
Great video! Thank you!
For me refactoring is a daily thing: I write all my code by refactoring things and injecting new features into the code while doing that. This way the code stays clean and easy to read and it's "easy" to add any kind of new features into the code base when nothing is sacred and any part of code can be changed/touched almost at any time of the project.
As a programmer, I'd like to add a reason why programmers want to refactor: They are embarrassed of other programmers seeing their spaghetti code and judging them. To be fair, they do.
Unless you're a C++ dev then you grandstand over those that call it spaghetti code
@rusi6219 What do you mean?
I want to take a minute and highlight what Tim was saying about code reusability. Being good at writing reusable code is a skill and it takes an investment of time and effort to get good at doing it. But this is probably the #1 most useful skill a developer can possess and I highly encourage all developers to spend more time thinking about and improving this skill.
Sadly most developers put little though into writing reusable code. I've seen so many development groups where they have numerous developers all doing similar things and don't have any reusable code, everyone is rolling their own unique implementations for the same problems everyone else is also solving. Not only is that a waste of time to write all those various implementations but each has to be tested and maintained separately and often none of the developers are familiar with the other implementations.
Tim's example of the single library containing all of the DOS calls making it easy to port the game to Windows is a great example. The thing a lot of developers get wrong is they think they will refactor the code later to make it reusable or they will add-in reusability later if its "warranted". This rarely works out. It takes considerably more time and effort to refactor code later to be reusable often overshadowing the benefit of doing so. Note that Tim says the first thing he did after leaving was to write TIG. Building something like that early rather than trying to do it later is a huge advantage.
Writing reusable code is a skill and a change of perspective. A lot of developers will find it difficult at first because it requires thinking differently, but getting good at this really pays off long-term. It takes me literally zero extra time to code things as reusable because I've spent so many years writing reusable code that its my natural go-to perspective. It would actually take me longer to write code that wasn't reusable because I don't even think that way anymore. Reusable code that only ever gets used once is actually fine. Non-reusable code that gets copy & pasted or reimplemented numerous times is spaghetti code. I say this to illustrate what is possible and why I highly suggest developers invest time into getting good at this. You can seriously multiply your performance over time with reusable code and when you get good at it you can multiply the performance of whole teams. Its a very valuable skill. FYI
/soapbox
"Tim's example of the single library containing all of the DOS calls making it easy to port the game to Windows is a great example."
That was right thing to do it. That was also part of code that was easily recognizable to be reusable. There are others too like handling translations, image/audio/video decoding, subtitles, date/time format, persisting data, memory management etc. We know that they are reusable and need to be handled platform independently so this kind of stuff need thinking about interfaces at an early stage how these should be used.
Project specific stuff however is not always clear in beginning what are all requirements.
@gruntaxeman3740 Not really. I've seen lots of software that embeds those types of things all throughout the code instead of centralizing them into a library. Oh, are you a newer dev? Those types of things are often abstracted into libraries NOW. But back when Tim wrote that library that was absolutely not the case. If I recall correctly from his video he got pushback on building that library. Its sad but many developers are their own worst enemies when it comes to code reuse. Often code needs to be deemed special or worthy or surpassing some magic threshold to qualify for bei g reusable. All of which is a terrible way to look at it.
All code that can be written as reusable should be written as reusable. With experience it takes very little (or no) additional time to do so. Its rarely obvious in advance what code would be useful to reuse 6 months or 5 years from now. The best way to win is not to play, learn to design and write everything for reusability. I have excessive and extensive experience to back this perspective up btw. I've drastically improved my own performace and the performance of teams numerous times using this perspective. But it does require a change in perspective and practice to get good at designing and writing code for reusability.
@gruntaxeman3740 oh, regarding your last sentance. Reusable code is often more modular and flexible than non-reusable code. So adapting to changing requirements is actually a lot easier when your code is written that way. I can't even count the number of times I've had requirements change significantly that one would expect to be a huge issue but I was able to easily accomodate due to reusable code. In fact, cha ging requirements was one of the thi gs that pushed me to get good at reusable code. Like I said above, its a different perspective.
I've written software past 34 years. Today we have stuff in libraries, 34 years ago we didn't. Back then even the C standard library wasn't fully suitable to reuse in games.
Today almost everything reusable is found from libraries and engines and problem is that junior developers add new dependencies to stuff that require 10 lines of code. And then software maintenance goes hard when those dependencies change. Tolerance to add dependencies is lower when making quickly something that works and later throw away but when creating _product_ what is maintained for decades, dependencies are evil and need to be limited. Games are good example. We still expect that game written in 90s can be run natively, on Steam deck. That requires maintenance and that is game developer job keep product working. That maintenance part is what makes difference and inexperienced developers don't understand what dependencies are ok to add.
One practice to handle this is framework where is almost all reusable code, and maintenance is moving code base to newer framework versions next 25+ years as there is changes regularly. That is well working practice when creating service what is maintained.
And when writing software that lasts 50+ years, that requires different angle to write software. I know how that is done.
@ I absolutely agree with you on developers (sadly not only junior) pulling in lots of dependances often for things that are very simple to code directly. That is garbage. A LOT of open source libraries are garbage and are either not designed, are not reliable or have insanely poor performance. Or they feel the need to update frequently and break existing code.
The best reusable code is written in-house specifically to address the needs of that team/group. Over time such code grows to form libraries that offer an array of pre-built and tested capabilities that can be quickly and easily leveraged. Unfortunately there are few software developers who have put the time and effort into getting good at designing and writing reusable code.
Writing software that is expected to last a long time without significant changes is also something many developers never think about. I've seen far too many projects need code changes and be redeployed just to deal with a small change in business environment or schedules. I try to write all software to be configuration driven and in the case of business software provide the users with the tools to perform foreseeable maintenance or correction operations. Results in software that works for long periods without needing changes. I try to teach this to other devs wherever I work but there are quite a lot of devs that are closed minded or just do the minimum.
P.S. I've been coding for about 45 years.
Strange because had i seen this a month ago or so I would have almost no idea what your talking about.
But I recently decided to do a project in Lua as my first language/coding experience. Once i got the code working I began to noticed repeated functions and the like. So i wondered how I could consolidate stuff. Opened up youtube to look up some videos. Boom here is this video hahaha. Perfect timing.
One thing that's immediately clear from the comments is that "code refactoring" can mean somewhat different things to different people…
I like the definition at the start of the "Code Refactoring" page on Wikipedia (which to my reading, lines up pretty much 1:1 with what Tim says in the video):
"In computer programming and software design, code refactoring is the process of restructuring existing source code-changing the factoring-without changing its external behavior. Refactoring is intended to improve the design, structure, and/or implementation of the software (its non-functional attributes), while preserving its functionality. Potential advantages of refactoring may include improved code readability and reduced complexity; these can improve the source code's maintainability and create a simpler, cleaner, or more expressive internal architecture or object model to improve extensibility. Another potential goal for refactoring is improved performance; software engineers face an ongoing challenge to write programs that perform faster or use less memory.
Typically, refactoring applies a series of standardized basic micro-refactorings, each of which is (usually) a tiny change in a computer program's source code that either preserves the behavior of the software, or at least does not modify its conformance to functional requirements. Many development environments provide automated support for performing the mechanical aspects of these basic refactorings. If done well, code refactoring may help software developers discover and fix hidden or dormant bugs or vulnerabilities in the system by simplifying the underlying logic and eliminating unnecessary levels of complexity. If done poorly, it may fail the requirement that external functionality not be changed, and may thus introduce new bugs."
I also particularly like this, from the same page, a little further down:
"Refactoring is usually motivated by noticing a code smell. For example, the method at hand may be very long, or it may be a near duplicate of another nearby method. Once recognized, such problems can be addressed by refactoring the source code, or transforming it into a new form that behaves the same as before but that no longer "smells".
[…] Failure to perform refactoring can result in accumulating technical debt; on the other hand, refactoring is one of the primary means of repaying technical debt."
Would you agree, Tim? Did your code ever seem smelly? 😊
knowing when and how to refactor is a skill that comes with time and experience. I personally wouldnt consider myself great at it but I always try to get low hanging fruits based on simple rules I picked up. Often its enough to have a final look at what you've written in total and then you see how you could combine or simplify a task. If its used a lot, it will be noticeable. Most of the time it doesnt require 200IQ hacks.
I'm refactoring my game Nodrog's Fortress for my enhanced edition release this summer, and your thoughts about where extra cycles might be available really helped. Also, finding spaghetti code and figuring out a better path to execute has been a fun nightmare.
I had a colleague who was obsessive about code refactoring (No, I wasn't entirely sure what the term meant until I watched this video), and he made some really amazing stuff. Of course, that stuff was all he made... I regret that we didn't get to complete our project (simply because the majority of the work needed hadn't been completed), but I did learn a lot from what he was doing (which, of course, was learning in his own right).
Also, side note: I'm fanatical about garbage collection. It's amazing how many programmers haven't heard of it, and don't understand how to cater to it (i.e. avoiding memory leaks).
Coming from a slightly different perspective, as a sysadmin I often need to write bits of code for one thing or another. Often I'll need to do something "quick and dirty" to get the job done, with the intention of refactoring it later for eventual and inevitable re-use. Sometimes that happens, but all too often it doesn't. Something always seems to come up that needs more urgent attention. What I've learned over the years is to *at the very least* put comments on everything, even if it seems silly. Refactoring can come later if there's time, but without those comments you'll never remember why you needed to process something in a very particular way that seems like (or may in fact be) a hack.
i love your stories
My team's programmer has gotten a lot better about avoiding needless refactoring, we've settled on taking note of areas for improvement to do them better next time, and then refactoring only as required.
As a full stack developer I follow the adage 'always be refactoring'. Since the code I work on is constantly being maintained over years, it is a necessity to keep things from getting too confusing and out of hand. This is because the overall system/requirements have changed so much over time.
Anyways, I find it interesting how refactoring would be done differently/less frequently in other fields.
Thanks for these videos.
My question for Tim: Does test-driven development have a place within games development?
Assuming that Tim doesn't pick up the question from here, I'll put it to the comments:
TDD in web development seems to be both looked on immensely favourably whilst also frequently being poorly understood. I see people in web development who criticise TDD and they are invariably citing examples of either problems they do not know how to effectively mock, or else describing solutions that TDD exists to highlight their flaws. In games development, I spoke to one fellow hobbyist who stated that though he used TDD in web development it was inherently impossible in gameDev, and that really only end-to-end testing could be used. At the time I accepted his advice. Since then I recently got a game 90% of the way to MVP only to discover the remaining 10% of bugs would be near impossible to find, and what I in fact had was a prototype. I rebuilt it using TDD and got a great deal of value out of doing it that way. Now I am able to find bugs much, much quicker and every time I come to a problem spot it is invariably because I stepped off the TDD pathway and into the wild.
What thoughts do others have?
I like that question.
Problem with tests is maintaining tests.
And truth is, UI code that handle layouts, it is not worth to write tests because they live so much. Instead it is better idea to optimize code maintainability and have type systems or domain specific languages.
There are parts of game code that are not just logic stuff, making code that controls how character jumps and how it _feels_ . Maintaining those as tests are just overkill. Test rooms works better and manually testing.
But backend and just logic part, it would very dumb to not write tests and ignore proper TDD benefits and that is true in game development too.
Earlier today I was going through my rebuild, and your opening line was, at least superficially, true. Tests built earlier in the rebuild were no longer applicable. The question came as to whether or not to maintain them, and I chose not to at this stage for much the reasons you say.
But I would stress, that the purpose of TDD - and the main misunderstanding about it - is not about having good test coverage, it's about designing from the perspective of tests. In my instance, even though the individual tests weren't no longer providing good coverage, building from them just meant the code was very nicely decoupled and easy to bugfix.
I have never tried TDD in the context of the sort of feel-based content you describe. I admit that I find it hard to envisage, although then again I spend almost all of my gameDev in strictly turn-based systems so I'm already at a disadvantage with that.
Yes, TDD is not about coverage. It is about thinking requirements and writing test for the requirement. It is not unit testing, TDD works on.. "module level".
I can strongly recommend TDD when requirements are known. It is that in reality requirements are not always known and developer need sometimes explore how to make stuff working.
Coverage is important on refactoring and maintaining code, when changes can be made easily and it shows immediately if something is not right.
I don't understand how this isn't the same as just testing if the code works after compiling it, in regards to game design? I mean, I recently wrote a little plugin for my browser where I had an ideal situation it would work in, which I used as a basis for testing if it worked, sure, but... I don't see how that's different from loading up a sandbox level in a game and testing a feature.
Is it just, specifically, coming up with a test first, with the test being the goal? E.g. there's a box in a distance, when a ball is hit from a certain point by a certain object, it should hit the box and the box should be deleted?
It's very important to separate refactoring from rewriting. Refactoring is a mechanical activity that shouldn't introduce any logical changes, only structural ones. It's the kind of thing that can and should be done with the assistance of tools. The moment the changes stop being structural and become logical, then you're in the realm of rewriting, which is where things get more dicey.
That is refactoring, but altering how the code is implemented to make it more flexible, more modular and to allow the addition of new features is also refactoring but isn't a "mechanic activity" as you describe. Some refactoring is simple enough that it can be done by automated tools, but that's not the limit of what refactoring is.
I would argue refactoring that can be done with automated tools is not the kind of refactoring we are talking about here.
The key insight is that most projects have ill defined requirements and actually through the process of creating the project, you discover all of the requirements.
Usually large refactoring is about redesigning a project using hindsight as a guide. Feature requirements you didn't expect but had to be hastily bolted on can be now be supported as a first class citizen.
This requires a robust test suite to be able to guarantee that during your refactoring you do not regress features you know are important but aren't the focus of improvement of the refactor
@@Chex_Mex Excellent points and you are correct, I'd agree that automated refactoring isn't what is being discussed here.
My approach is to try to make the new module have as little influence on the rest of the code as possible and create some bounds in which I can write the worst code imaginable. The first iteration will always be bad no matter what, so it makes sense to try to decouple it so the module won't force extra work later. Refactoring comes along with bugfixes/optimizations/writing code that relies on that module, because it introduces new context and makes it clearer what you need to do with refactoring. The time spent on refactoring this way can be accounted as time spent on fixing the actual bug/writing the new module, so it's also a way to hide the refactoring/tech debt work from the management :) In the end, the tech debt is mostly gone and the next iterations take much less time. If we have time to spend on making the project correct and robust.
The gamedev specific thing is that you have less time and resources to work on these things, so it might not be possible to do proper refactoring in some cases. The code has capacity to have a certain amount of quick dirty fixes before it's a mess, sometimes it makes senes to just throw in a flag and an if and forget about it. The problem is when it becomes a developer culture, I'm currently working on a project when the whole chain of developers starting from Unreal Engine devs (or the C++ commitee...) has failed to come up with sane methods of writing networking-related code (readily available in almost every other language's standard library), trivial changes take literal months. I'm tired, send help.
But I agree with you, good refactoring often requires extra context, otherwise it's mostly a whiteboard masturbation. I sometimes hear critique of Unity ("the developers didn't make a single game, therefore they don't have the experience to make the engine good") or Unreal Engine ("they only made Fortnite and that's why the engine is only good for Fortnite and not as a general-purpose game engine"). This observation might be more fundamental than just refactoring.
In the case of Unity/Unreal, they are somewhat held by backwards compatibility. You just cannot refactor your API if you have customers who would also have to rewrite all their code if you changed yours. This results in Unity throwing all kinds of variant render pipelines while keeping the old one for instance.
Love these kinds of videos, keep them coming, thanks
A fourth reason for code factoring is because major feature is required and the existing code is to unwieldy and/or unclear for anything other then minor bug fixes. And the cost of a entirely new code that produces the same or more correct answers not only costs millions and will take a decade along with the validation required by the customer costs millions but the budget is much less.
I should take your advice on pushing the refactoring further. I am not yet prioritizing game development yet, because I have other hobbies, so usually I do something in the project for 2 weeks, then I leave it be for 2 months😅.
The problem for me is that when I come back to the project, I take a look at the code to familiarize myself with it again and notice that there is something "wrong" in how I'm doing things. But that occurs, because I still learn better coding practices during those 2 months when I'm away.
Maybe someday I'll forgive my earlier self and move on to the next system😅
In my experience the most common reason for refactoring code is to add new features & capabilities to an existing, functional system. In fact its often difficult to get approval to spend time refactoring code just to make it cleaner, more optimized or more reusable without being able to tie that work (and thus those hours) to a new feature or capability. This somewhat depends on the industry the code is for. Most commercial and enterprise systems (sadly) don't really care about performance or optimization anymore. Which is why you'll be interacting with some employee of a company and they'll have to apologize for how long their computer is taking to do something (something that should probably be nearly instant except that most code is poorly written). In game development, however, optimization is much more important and its one of the few niches where performance is still tracked and management cares about it.
One related question I have is about the prevalence (and usefulness) of automated testing, continuous integration and other standard "quality assurance" processes. Is this something people are doing and if so, how is it similar to/different from the practices seen in other domains? Is even worth doing for games? I'm not a professional game developer but I've heard very mixed opinions on it so far.
I imagine having to deal with frequent design changes might make this a nuanced discussion. In my experience, there's many areas in game development that are exceedingly difficult to test, such as rendering code or interactions with platform-specific APIs (e.g., console SDKs). On the other hand, a lot of utility code or libraries which are more general might be easy enough to test that the benefits outweigh the cost?
I've never been on big game production, but currently smaller one, and I'm sure that these kind of processes are widely used, but...... Like I mentioned here in comments, automated tests are not always good idea on UI side like vector based games or UI layouts. Automated builds for every platform are for sure useful. Test rooms and testmaps are used a lot in vector games.
Simultaneously it feels like the gaming industry is very backward in some respects. Like, there are still game titles that don't run natively on Steam deck/Linux, and I found it very odd. Building phase in software is not hard.
One huge difference in my understanding is play testing. Like, put people play game, record it and learn from that "is it fun" or "does players stuck on somewhere too much"
I hope Tim will someday make video where he goes through some practices about this.
In the overall tech industry? Yes, people are doing CI and unit-testing (and TDD/BDD (test driven development / behavior driven development) to a much lesser extent). Though, a lot of companies say they are doing it, but fail to understand how to unit test most effectively. Or just don't have a contradictory organization structure to support it (like having a QA team separate from the dev teams). Do game developers do it? Not so much afaict.
I believe the game development industry has less people doing it and not that many people discussing it at all. One such obstacle is the importance of UI in video games as one person already suggested. UI is difficult to test. You can't unit test UI as easily because you'll just find that you're testing the underlying framework or tool it's built on (which the devs for that tool should've already done), and it's difficult to write integration tests with it, but you can do some high-level snapshot tests to automate it a bit better, but it still could involve manual intervention, so it's not perfect. You should also decouple from the UI, making sure to unit test the APIs/data it needs to use rather than how it gets displayed when unit testing. The goal isn't 100% coverage, but building trust between the different parts of your technologies.
In things like rendering, you might want to test the underlying rendering/graphics engine, to ensure the math is being done right. The trick is remaining decoupled and making your tests decoupled and not dive too deep while providing enough confidence to let you understand what your changes accomplish.
@@gruntaxeman3740 I think some notable games that might get close to having good tests are those with great tooling. Warcraft 3 for example had its amazing editor tools. I doubt that it was bug-free, I'm not sure that any game has been bug-free, but I do believe that it's a step in the right direction of testing just enough. If you focus on testing the tools, you might be able to gain a lot of confidence in the underlying game itself.
It’s nice when coding teachers will give u extra points for being crafty with how u sort your projects. Sometimes it can be like 1 step forward 2 back but I swear going back and re learning stuff only solidifies the ground u walk on. Once you learn about classes and objects it can be really nice u can store so much data n basically make entire organized systems that run on a few lines. Where yah I think having clean code is nice but it’s almost not what you think clean code is just like writing a word like appleSauce vs AppleSauce to save your finger the strain n yah just show u still divided a word for the reader 9/10 it’s like good punctuation n grammar in writing where u put the curly bracket } is another one. However some of it is about how fast you can make things run optimization is name of the game.
This is very familiar to the advice given in "The Mythical Man-Month" by Frederick Brooks almost 50 years ago! My feeling is that even though there has been change in large software design since then the broad stumbling blocks from early computing still happen today? Would you agree with this?
One thing I've been doing recently is refactoring (or factoring) a lot of the code in my projects to try and consolidate all the potentially buggy code in one big quarantine zone. Maybe that'd fall under maintainability but I like to think of it as proactive bug hunting.
Completely off topic: What was your favorite dark ambient music back when you were super into that? There’s soo much of it now, but in the 90s-early 00s it was much more niche. Was checking out a live stream where you said the music for Fallout was dark ambient because you handed the composer a stack of dark ambient CDs; as a composer for media, I’m just curious what albums those were.
Hi Tim.
Excellent video and I agree, but I like to add and clarify something from my own experience:
1. Refactoring for performance.
Just don't do it if there is no reason. Or if refactoring for optimization is done it should be very last phase of development.
2. Refactoring for maintainability.
You can't refactor enough. Refactoring for maintainability improves _almost everything_ . Performance, portability, memory usage, readability... everything is improved. And that matters a lot because most of time, developer is reading and understanding code, not writing it. Code should be treated in way that when there is mess, then clean it. Formal way to describe "refactoring for maintainability" is about minimizing "if" clauses and improving how easy it is to read and understand.
But there is one important thing where premature refactoring for maintainability doesn't work: There are more than single dev, (or tightly working team) and they are repeating stuff. That thing should NOT be refactored because developers usually don't really know exact requirements what they need, and sharing code causes that they need to adapt that later so the shared code easily is conflicting with requirements. When all developers / teams have done all features ready, that repeating stuff from each module can be refactored away when developers truly understand all requirements that shared piece of code must handle.
3. Refactoring for reusability
That late stage refactoring of common repeating code is good candidate for that. Sure the when designing architecture some low level stuff can be actually see early phase that it can be reused.
---
Also refactoring should not almost never break the code. Related to code maintainability and polishing it, it should have also proper tests for every feature before doing on those late stage refactorings. If refactoring cause new bug, then just add test that triggered it so it never pops up so changing code pieces to more efficient or sharing those across modules don't break things. I'm aware that UI or similar this is not always good idea, and maintaining tests may require much more work than maintaining just code.
I have to disagree on some of that. Refactoring for maintainability rarely improves performance, often it degrades performance. Depends on the details and the degree of optimization, but optimized code tends to be harder to read and maintain where code optimized for readability often performs slower than less readable alternatives. Also portability, when that is a factor, often leads to less readable and maintainable code since it may be necessary to use ifdefs/conditionals or similar to have specific code for various platforms. Most software doesn't need to be portable though and in those cases portability should be completely ignored.
I don't understand what you're saying about "repeating stuff", are you talking about code reuse? Code reuse is the #1 best thing a developer / team can do to improve their performance (them accomplish more in a period of time) and improve the quality and reliability of code. Code reuse should be done early when possible, not later. Trying to refactor code to be reusable after the fact is extremely time consuming and error prone and often people say they are going to do this but never do. Spending a bit of time to think about and plan the code before writing it such that it is designed to be reusable and modular is a huge boon to projects. I've personally 5x the output of entire teams on multiple occasions through good code reuse. But writing good reusable code is a skill and it takes time and effort to get good at that skill. Sadly few developers put much effort into getting good at this. However, it is quite possibly the most valuable skill a developer can possess as it will greatly increase their productivity and the quality of what they deliver.
Performance issues are almost always caused if software is doing something stupid. And when refactor code to be easy as possible to read and maintain, it is easier to spot all those parts in code that are doing something stupid. It is not then changing some loop to work more efficiently. Then it is about removing whole code away.
If there is code that can be identified early phase to be reusable when designing architecture and see that is cross cutting whole project, just gather requirements for that, write tests to them and make interface and simplest version that work. Then mark it with comment that this need to be written on proper algorithm later. Optimization is done later.
Code reuse is for sure best thing what team / developer can do to improve performance but I mean situation when there are multiple teams or some reason team is not "tight" on communication. Like team blue is doing feature A and team red is doing feature B, and they can share stuff. Problem is that teams probably don't know exactly all requirements so if they refactor prematurely common code away, they very easily lead up to situation that team red is making tiny change to it so it behavior can cover everything module A. And then the behavior causes bug on module B what team blue is working. Teams may also end up to do some messy hacks to make their features working.
Proper way is that both teams complete their module and they know exactly requirements on the code that can be shared.
Solo dev or closely working team can reuse code better but having other team working on other country, that is not same thing. It is also known that software architecture resembles organizational structure. They are so equal that designing software architecture is also making decision what teams and people do what. That is also reason why it is very important to define interfaces between modules in way that they don't change.
What I mean is that writing code is easy and simple. What is hard is to know what to code. Requirements are not always clearly known and there may be exploration to make it work. Developer just don't get exact specification what to do because the exact specification is the code what developer is writing.
Refactoring working code when features are known and proper tests are written is actually simple. Many architecture level decision are better to do right in beginning. There should be a lot of thought involved.
@gruntaxeman3740 Strongly disagree that the "PROPER way is thay both teams complete their modu,e and they know exactly requirements on code that can be shared". That is completely backwards. Though lots of developers think that way which is why I regularly 5x most developers. Doing code reuse after the fact is extremely wasteful. Writing reusable code is a skill that takes time to get good at. Once mastered it is fairly easy to design code, interfaces, libraries and frameworks that are highly reusable without most of the effort you suggest. The best practice isn't to spend time trying to identify cross-cutting concerns, the best plan is to write most code as reusable. This is the biggest mistake most devs make, they think code needs to surpass some magic threshold to be written as reusable. Well over 50% of all my code is reusable, always.
"That is completely backwards"
It is, but it is faster if all requirements are not clear what is very common. Remember that software developer job is to understand requirements from stakeholder and write it to formal language that computer can understand, and requirements do very often clarify during development. So if there some module written by outsourced team from other country, time is very easily wasted on communication, coordination and later realize that previously written /common module didn't work on both features.
And the important thing: writing code is fast. Debugging, trying to understand some spaghetti or debugging, that takes time. So when requirements are not clear, just let both teams work independently and when features work, common stuff can be later refactored.
You are talking like writing code is slow and that is big effort but it is not. About 30% of time goes to understand requirements. Implementing something what is known is fast.
"Well over 50% of all my code is reusable, always."
I've written currently backend to one game, and when excluding database, language runtime, e2e tests and OS.. 99,12% of code is assembled using prefabricated, reusable components from framework.
So yes, this is using framework approach for maintenance. All game code in backend is very project specific so they are not reusable.
Love to see dev stories that are similar to my non-gaming career! My two cents: I think code refactoring as a reader-targeted optimisation. Then, as with all optimisations, you have to ask if it is worthy - just like Tim described!
Some advice from my schooling that I've kept around over the years: the only way to write code with no bugs is to not write any code at all.
i.e. every line of code written has the potential to have a bug in it.
Great video Tim, thanks for sharing. I always wondered what engineers meant by this.
I have noticed on many projects there tends to be friction between some disciplines.. do you have any tips or advice on how artists and engineers can better work together? Does having tech artists usually alleviate this?
I have a related question: I'm applying to internships right now-for the purposes of projects on a portfolio, is it wise to refactor code, specifically to reduce spaghetti code, even if it doesn't make sense to refactor it in practice for that project? (i.e. spaghetti was added because features were added, but the code works fine). I would maybe assume its helpful because of readability but curious what you'd think.
Wanting to refactor when you can't is like wearing a coarse wool straight jacket on a hot humid day while attending the world lawnmowing championships and you have moderate to severe hay-fever.
This is why I try and follow SOLID principles (especially single responsibility principle) as early as possible. If your code is properly separated and put behind simple interfaces, it's much easier to refactor it without affecting other parts of your code base. Get your interfaces right early, and refactoring the actual implementation later becomes a non-issue.
The biggest problem I've seen with large refactors or re-writes is people running headlong into Second System Syndrome.
Programmers will often think about the new code, without thinking about how they will replace the old code with the new code, or what the customer experiences in the interim.
I've toyed with the idea of not _just_ refactoring code, or making version 2 of a product, but also writing a program that sits above both versions of the product, checks what features are implemented in the new version, and dynamically switches customers from the old version to the new, depending on what features they actually configure.
Unfortunately, it's something I could only do as a hobby myself, since when I explain the difficulty of doing it, people either prefer to just have one product and stall feature deployment for months, or decide that a refactor is altogether too much work and stick with an older product.
Of course, I'm not proposing a refactor for no reason: usually, the reason I'm recommending it in the first place is because the current code base causes developers to take days or weeks on tasks that I know they can do in hours.
Maybe too broad a question but I wonder how your approach to game dev changes if you're doing it specifically to learn and improve your skills (Be it programming or designing) versus creating a product at a company! Love your videos :)
You might enjoy my videos about making a little “toy” game for fun. This is the final video:
Toy Post Mortem
th-cam.com/video/kVV_2Yy_aXY/w-d-xo.html
Its description contains links to the rest.
the problem with "Code Refactoring" is that it doesn't have a clear definition on what it is actually about.
i mean it as in is it:
- abstracting the code?
- making the code DRY?
- restructuring the entire feature/code setup including all classes?
- making the code "Clean"?
- cleaning the just written code up so that the variable and function names adher the agreed syntax and adding comments? (many in this comment section see that as refactoring)
and so on.
what i read way too often about refactoring is that the goal is to abstract and DRY the code to make it more readable and reusable. the problem is that it both is often WAY too early abstraction (and unnecessary) and the attempt to make it DRY leads to very big if else chains or switch cases to handle the different places that aren't supposed to use the same function but do.
how much code are we changing in a Code Refactoring? is it only a function? is it a class? is it a feature? is it the entire file structure and what code is in what file?
is it going through existing code and removing inbetween step variables that don't really need to exist (expect for ease of debugging). does that already count as refactoring?
does refactoring mean you need to write the new code version from scratch with a new approach?
on that node. i haven't found a clear definition of "spaghetie code" either. if cross file calls count as "spaghettie" then 95% of game dev code is inherently spaghettie code. it's near impossible to write the code in a way that doesn't require that you need to go through multiple files if you change a function name of a class object that get's used more often or interacted with in multiple places.
For most, it is fairly clear. I feel you're overthinking the purpose of refactoring. You certainly mention some important parts like DRY and restructuring, but focus too much on large changes.
Many also do it iteratively, so you wouldn't do an entire class in one fell swoop. Here is a bullet point list that describes what is a refactoring and how it differs from other changes you make in code.
Feature Added:
Description: Introducing new functionality.
Impact on Tests: Requires new test cases for the added functionality.
Likelihood of Breaking Changes: High, especially if changes affect existing APIs or dependencies.
Feature Removed:
Description: Eliminating existing functionality.
Impact on Tests: Requires removal or updates of related tests.
Likelihood of Breaking Changes: High, as dependent code or integrations may break.
Optimization:
Description: Improving performance or resource usage.
Impact on Tests: Existing tests should still pass; may require adding performance tests.
Likelihood of Breaking Changes: Low, as long as external behavior is preserved.
Refactoring:
Description: Restructuring code internally without changing behavior.
Impact on Tests: Existing tests should still pass; no new tests needed.
Likelihood of Breaking Changes: Very Low, provided functionality remains unchanged.
@retagainez I understand that there can be a "clear" definition but everytime I hear someone is "refactoring" they are talking about different things and scales. That is what my comment mainly is about. Just "I need to refactor this" doesn't tell anything about the planned actions the dev wants to do because it could mean 10 minutes of work or 3+ weeks of work.
Most of the time just changing a functions internals to be cleaner isn't seen as refactoring but just as cleaning or optimizing the layout.
Too often refactoring is seen as complete rewrite and days to months of work when you talk to the managers. If it only takes a few minutes or an hour then the managers likely will never hear about it.
@@dovos8572 I blame that on poor knowledge of the definition of the word "refactoring" and the history behind it. Even this video misuses it. Cain talks about optimization, not refactoring.
Also, refactoring shouldn't be a task on it's own. It should just be a part of writing tests, writing code to pass those tests, and keeping things tidy once it's done. It should be a part of ordinary coding. If you have some work that will take 3+ weeks of work, that's more of an issue about the code being difficult to change. It COULD be difficult to change because the team doesn't refactor or test their code at all. It's difficult to refactor because it's not designed well (because it doesn't get tested). Note the emphasis on code design, and more importantly test design, designing good tests so that your code may follow.
I mostly do it daily in little steps along programming, but also if there is change necessary and not easy to do as is.
I'm still waiting for the crossover episode with you and Dave Farley from Continuous Delivery
Refactoring code is a tricky subject. Never refactoring leads to both messy code and a tendency to "gold plate" your code - trying to make it perfect the first time out. Constantly refactoring is a waste of time.
Here's my general rules. First, don't worry about getting your first pass of the code perfect. However, follow the boy scout rule. Every time you touch the code - a new feature, a bug, etc - make it a little better. Does the code not work for the new feature? Refactor it. Was the source of the bug a design flaw? Refactor that design flaw out.
I find this to be a good middle ground. It avoids gold plating, it avoids constant refactors, and it avoids your code turning into complete trash. In my experience, as long as you're not shipping in the next couple of months, it speeds you up rather than slows you down.
I know so many people who need to see this video xD
and this pops up right when I'm doing it. Maintainability reasons
Do not underestimate code cleanness. I abandoned my first project (a card game), because over the time I improved a lot, and couldn't stand to deal with the entire code base. Not that I was so bad at it, but my style changed a lot, and I stopped using bad patterns (yes there are bad patterns). My current big project is very well coded. I do refactors solely to improve the logic, especially when it's old code, or when it needs to be reworked.
For me quality code is very important. It's not even just about performance (although that's important to me too), but I'd like to not hate my code and be proud of my work.
Understandable but its important to be careful about this. People can fall into traps where they spend a lot more time being very precise and clean with their code and not end up delivering anything because it took so much time. Balance is needed in such things. I'm a professional software architect and I very much like well design, modular clean code but sometimes less that perfect code is good enough and not worth spending more time on.
Nothing like "We'll refactor post-prototype" and it making it to final build
There are very few cases in which you should refac code in gaming. The product cycle is not too long and refactoring is expensive
When I don't know ahead of time exactly what code I'm going to write, so it involves some degree of improvising, I'm *very much* guilty of the sin of immediately wanting to refactor once I get it to work properly.
On importand consideration in your sorted array insert example that you didnt mention is the average and maximum lengths of the array.
Bubble sort and insertion sort are faster than quicksort for very short arrays. This is tge kind of thing that lots of newer computer science grads might not think about because how much emphasis CS puts on asymptotic time and space. For small n frequently something dead simple with bad asymptotic growth is much, much faster.
I can’t listen to Tim talk about fast insertions with a straight face
Hi Tim!
Do you agree that colleagues are not friends? In my opinion, people can become friends, but only after they no longer work, study, or depend on one another in any way. Where do you think the line is drawn between friends and colleagues? Do you think it’s a good idea to work with friends? If it is, then how do you manage potential conflicts or maintain professionalism in such relationships?
I don't know about American work culture but that's definitely not true in Europe. Most of the friends of my colleagues, are other colleagues. It's natural.
American work culture is horrendous, Europe work culture is much better
I became friends with number of colleagues during my years (I work in IT for more than 25 years). I have a friend whom I started working with very early and multiple times we asked each other to come to work together at our next company multiple times when we left. Besides the usual social engagements like parties and holidays together, etc. I made zero friends when I worked in the US though (and I was there for 10 years). It's a US thing I think, here, in Europe, we are friendlier.
Your question: Do you think it’s a good idea to work with friends?
The answer is yes if you consider your computer a friend. In most cases it (or he) is very reliable. It is there for you 24 hours a day, 7 days a week. Most of the time he doesn't complain and if he does, he has a good reason for it. He has no big demands on you, answers all your questions, does not shout at you and always plays with you if you want to play a game. The only thing it lacks is an endoskeleton.
Hello Tim! What do you think of voice interactable NPCs run by AI ?
Hi Tim. Would more people enjoy shows that repeat the same dialogue again for the rest of your evening because the protagonist constantly died? Speaking on single player open world type rpg campaigns, Do gamers have such lack of awareness they need a game over screen to inform them? Can simply halting progress and being outmatched serve as your tell? Should game designers deliberately make obtuse game mechanics to get players who look up guides to make more posts in a hack attempt to increase engagement in the gaming community?
What do you think about permadeath games?
People try to do too much at once and then end up overthinking. I find it easier to refactor after taking a break from it.
Im a indie dev so a lot of times ill write Spaghetti code just to quickly test different stuff. When I find what I like then Ill clean up the code
Since you're technical talk, I'm curious to hear how unit testing is done in the game development industry.
12:05 "Just having a better idea of how to do it is not a reason to refactor." That one hurt me deeply.
Do you know where the Living One is supposed to have come from? I know a lot must be left to the players imagination but they seem to come from some completely different continent. Sorry if you've been asked this before.
TH-cam should refactor their code so that any of my comments show up
This is the stuff of my damned living-nightmares as a line-of-business software-engineer. So much Spaghettification from how much Scope-Creep my non-programmer bosses keep piling-on to projects that were engineered for a specific and defined set of requirements, which ends up resulting in having to make large-scale refactors of both code and _database schema_ on a regular basis. Bonus Points when something is implemented *to the letter* of the requirements-documentation I was given, and then don't like the thing that does exactly what they claimed they wanted, either during QA, or even better, *after* _they themselves_ approved it for Prod.
the best programmers have to be mind-readers too, smh
I hear from senior engineers a lot that the customer doesn't know what they want
Customer often have vision or concept. Then when asked "how authentication should work or what user roles should have and how UI should look on different places" they don't know anything.
To actually launch business it needs that person who actually interpret that vision and make something that works and give value.
hi tim, it's me, everyone
My foremost game programming mantra: Don't solve problems you don't have.
Eh sometimes but not always. The best most optimal systems are architected from the beginning to be modular and anticipate the later needs of the project. This is effectively what a good, knowledgeable software architect should be doing for a project. But like many things this is a specific skill that requires time and experience to get good at and many developers never even try to get good at this. Possibly why there are so few good software architects.
@@Me__Myself__and__I I don't think those things are *necessarily* at odds. I just think the vast majority of programmers tend to design more flexibility than will ever be called for, and overestimate the cost of changing small rigid systems, and underestimate the cost of changing large overly complex modular systems when their modularity doesn't actually match the way the problem changed.
I will also say that in games, you rarely know what changes will actually be necessary. And the parts you know for certain tend to be part of what most developers call an engine.
Games are also incredibly high performance pieces of code and abstraction tends to have real cost, so you simply cannot abstract everything.
Which all adds up to real benefits of solving problems as small as they actually exist, until they are actually known to be bigger problems. And when you build that small, if you were wrong, throwing out tiny solutions is trivial and didn't waste any time.
@ Don't agree. Most programmers don't build any flexibility or modularity into their code, the problem isn't devs building too much. The rest of that 1st paragraph is a skills issue, that just means those devs don't have enough experience designing good modularity.
I've been building modular systems from my own reusable code for many years now. On every single project its more than paid off. Most systems gain tech debt and cruft over time as they are modified and changed eventually leading to a require re-write. None of my systems have had that problem. Even systems I didn't design but took over and slowly converted into being modular improved and become more stable with less tech debt over time. I literally have decades of experience showing that this approach works very well. But it does require skill and knowledge that takes time to develop - but only if the dev is actually trying to get better at reusability and modularity.
Lastly the game engine thing, absolutely not. Much of what is very useful to build using reusable and modular code is not included in game engines. An engine pretty much just takes care of the presentation layer and the input. Engines don't include game mechanics, characters, skills, items, inventories, NPC AI, etc.
Yeah, (many/most) programmers like to live in the technical bubble, where you don't have to think of production or product realities. Also, probably don't want to think there is anything to refine in their methods, because they are basically Spock.
Most of the programmers don't write game code.
Best practice for business code is to do absolute minimum amount of features that brings value to customer, and just make that minimum amount of code to good. If it brings value, software has future and more features can be added.
And possible this is the most tricky part in game development, that game is often "single release" software, where tons of features should be done and working on release.
That is also the problem on game development because games are often poorly maintained. Related to refactoring and polishing, I'm skewed to direction that while we have now digital distribution and easy way to update, game can be released when all of the features are complete and there are proper test suites that allows easy refactoring.
But In my point of view, that should not development. Game code can be refactored after that, continue polishing and do optimizations, add features like graphics card specific stuff. But when refactorings that require all knowledge from people are done, game can be moved to "maintenance mode" when there is single dev keeping it working small effort, like moving it to newer language versions, newer runtime versions, porting to new OS/platforms, newer engine versions.. Maintenance should not stop for release because all code in the game will rot away, transforming it garbage if no one is taking care.
I believe that game shelf life is somewhat increased so older game releases can generate revenue, but in my understanding most of the sales do happen on first 6-12 months. I think game developers could think ways to keep product on peoples mind longer, like keeping noise on making releases to new platforms, or some cases think game in way that it has sequels based on very same code.
Perfectionism can be its own masochistic hell, trying to make the best thing ever without first profiling to see if it's necessary. Especially with all the critical testing to make sure it works against deadlines with all the time it takes to rewrite. Part of vision is seeing the real priorities. At the very least always back up in case the older version is better lol maybe easier said than done when dysfunctional pride and time is attached.... don't waste time in the first place.
Thank you for the video it was interesting.
4:40 These pause menus where work is supposed to be done only work well if it is a single player game. However, if you want your game to be multiplayer friendly from the start, you usually cannot pause the world calculation. So if a player opens a GUI inventory in multiplayer, for example, the game world cannot be paused. It must continue to run. Could make another video about parallel programming and concurrency?
15:21 🖖👽
If you have a multiplayer game, it's probably better to have separate server and client processes.
@@LDiCesare Of course, but you still have to do calculations.
@ Yes, but the calculations that impact other players can be done on the server or predicted by the other clients.
@@LDiCesare I'm not talking about pure multiplayer games, but rather games that are for single player and multiplayer. For those, you'll want to have as similar a code base as possible, because redundancy only costs work and money and there's a risk that you announce multiplayer as feature for a game, but then only finish the single player part and then have to say that you can't do the multiplayer part because it would be too much work and the code would have to be changed too much.
And in this case, that means that you run the server on the client computer when the player wants to start a single player game. Thus client and server code is running on the same machine. If you want to avoid OS specific task switches, you could run both in the same process, but these are details that I don't want to go into here. The only thing that is relevant here is that the code from both then runs on the same machine.
so this is like overusing the tactic of filesave.
I disagree on the "don't refactor once you're done" stance. I do it almost systematically and often ask my team to have this step as part of their process. You do a first draft that is exploratory, you encounter bugs you would have not thought of before, you see issues with your logic, and you end up with a great knowledge of the problem you are solving. You've also defined a testing process for your code (usually using unit tests but I don't see you mentionning it so I'm guessing they are not used during the projects you've worked with which seems plausible considering when some were done). You can know do something to the quality standards you expect to deliver without anything left from the exploratory phase.
I would also argue that refactoring is a skill that has to be trained. I've seen developers that can't refactor well and some that could refactor a huge part of the codebase efficiently without any issues. The more you refactor, the more you get a grasp on the consequences of refactoring, the more you can refactor or get a feel on the refactorability of a piece of code.
Is someone paying you to refactor the codebase after you've shipped? 😊
(Bear in mind Tim's talking directly about games development, which generally aren't very long-lived products like an operating system, an office suite, etc…)
"Is someone paying you to refactor the codebase after you've shipped?"
Should. Code will rot to garbage if not maintained.
When code is purely in maintenance mode, this is very trivial:
1. Identify change points
2. Find test points
3. Break dependencies
4. Write tests
5. Make changes and refactor
This is the basic routine to move code base to newer OS, newer runtime, newer language version, newer engine version, shovelling of some garbage library and so on.
You were going to be the US VP
Has anyone ever told you that you look like the guy from the danish gaming tv program "Troldspejlet" ?