I've actually read his book and it's quite good. Unfortunately the talk isn't great. Yes, nothing new if you are used to functional programming, immutable data, etc.
There were definitely ways of dealing with immutable data before Clojure implemented Hash Array Mapped Tries in the late 2000s. Functional programming languages had been doing that for years. Rich Hickey didn’t invent HAMTs, that was Phil Bagwell (he said Hickey “discovered” them. Not sure if that’s meaning Hickey used Bagwell’s paper, or not). Structural sharing is different to HAMT, and is bread and butter for functional programming languages, but using the same techniques in non FP languages with mutable data has always ended up being fiddly and error prone if it’s not built in. Sure, you *can* do it, but without the guarantees of immutability, and without a language optimized for handling GC overhead as a result, it’s never great. JSONSchema is great, BUT it’s really about validating data at the edge of a system where it’s receiving input. It’s quite expensive, and there’s NO WAY I’d want to be doing that every time an internal function is called in an app just because there are no guarantees about the shape of any data the app is using.
Its very interesting that his idea of what data oriented means is very different from what other people mean. The game developement/handmade/lowlevel comunity sees data oriented design as "removing all thats not strictly necessary for the functioning of the program" or "getting the clowns out of the car" as formulated by Mike Acton. Of course the result is less code thus easier to understand and by almost coincidence better performance. His idea of data oriented approach is a lot more opiniated and relies mostly on ideaological statements. It mostly boils down to "keep data around and transform it". I find this in direct contradiction with data oriented design which begins with the question: what is it you are trying to do? Load an image process it write some logs and exit. Okay then do exactly that an no more. There is no reason to have classes, factories, singletons, concept of resposibility etc. because fundamentally they are nowhere in that description of needed steps. The steps are take this and do that then take the result and do the next thing. I have seen his idea of data oriented design in the closure comunity before (or maybe its just an echo of his book) and find it very interesting how different comunities can interpret an idea totally differently (and wrong if you ask me).
Fewer edges for each individual node make each a simpler system, assuming the addition of an edge to a node is more than a linear increase in complexity.
@@melo682 yeah, I don't see how that makes it any simpler. You still have the 1:1 mapping from data to functions. You have a stack of papers. You shove them around and make two stacks of papers out of it. Nothing was accomplished.
@@Tekay37the benefits I can think of in terms of code are splitting classes into records and functions forces the code to avoid referring to data via member variables. I broadly agree with the idea presented in the talk, after all we model our data in SQL, which is entirely separate from the application. But I think the talk suffers from insufficient examples
@@melo682 I do think DOD makes code simpler as well. I just don't see that benefit in the given diagrams. There can be much better examples. (e.g. the practical optimizations talk Jason Booth)
He also mentioned that you don't need the second graph. So now it's only one. Even with two graphs it's still simpler because you've broken apart a big mess into two smaller messes.
The presentation was unconvincing and shallow, in my opinion. Nothing concrete was presented. It's all slide-ware, ideological claims (e.g. "this way is better/simpler"), non-sentences (e.g. "data is represented as data"), and a few minutes of random meditation (??). The speaker argues that moving away from classes to describe data and instead of using untyped blobs of JSON + JSON schemas is a better approach. Of course there are no absolute "better" approaches in CS. Only approaches better suited for some problems. For instance, if compatibility is important, you could argue that having a server and a client be flexible about the data they share is important. This is why attributes are all optional in recent versions of Protobuff for instance. But the speaker doesn't share such nuances/trade-offs. They claim that's it's generally simpler/better without talking about any downsides. It's essentially ideology we are served. In our field of science, we expect strong methodology and evidence. Nothing of the sort was shown here.
How is this related in any way to dependency injection? This is more about immuatbility and functional programming I guess. Nothing new as he admits in the conclusion.
So suffice it to say, he is pro- manager classes and/or global functions and static methods. And that means you won't be able to reason about it as well, actually. Maybe you understand the system conceptually but if there are parallel calls going on, i think it'll be more difficult to debug. Singletons and globals are generally what i'd stay away from. Separate models are something i keep at the boundary but try to convert to domain specifications as true class definitions.
I don't agree that global functions or static methods, on their own -- without shared state, are a problem. Why would they be? If anything they are easier to reason about because they have reduced surface area of references that can change at run time. I also don't think this talk suggests using singletons or globals for managing state.
Why is there confusion that this is the same as Data Oriented Design ? Looks like a lot of people seem to assume its the same thing, does the presenter say DOD == DOP ?
Because they both start with "Data oriented". There is also object oriented programming and object oriented design, and they are both the same thing. But what's worse, the speaker/author here is completely discounting the methodologies that came before him. What he calls DOP is actually just Functional Programming as many comments here mentioned. The real "data oriented" paradigm is what Mike Acton started talking about 10 years ago along with many other video game engine developers and system engineers that felt the need give a name to a paradigm that is very different from OOP, functional programming, and procedural programming.
@@vakey I feel like the video ge style DOP is more focused on cache and hardware optimisation. Whereas this is talking about design but without the scary function ideas of monads. Thanks for the heads up on the references. I will be sure to check them out.
Didn't watch the whole thing but I feel like the idea hs is promoting is treating everything as data instead of a model that modeling some concept in real world. I feel like some type of people might like these idea and might be benefical to them but majority like me are only human. However, It is also similiar to "everything is a map" anti-pattern. Put everything in a map, pass the map to others and then receive the output as a map. You also have to constantly check whether a key-value pair is inside the map or not. And whenever you meed to do something like control flow or make a decision, you need to do all checking for all related properties all over agin in every layer. People have seen "everything is a map" in their life know how horrible is. this kimd of philosophy only guarantee you creating a dead-on-arrival system that the whole thing turns into an ununderstandable, unreliable, unreadable, unmaintainable, difficult to chnage, incapable to reason with and error-prone technical debt immediately.
I wonder if the best of both worlds would be advanced type inference on the fields of those maps (say through row polymorphism)... Extend that to tuple types (similarly encoding the type or validator function at each position), and you've got these nice general collections and the code the reuse through generic functions Yehonathan is talking about, all without having to write rigid type definitions but there would be strong type checking preventing you from making the same kind of mistakes that up front type definitions prevent you from making.
I've actually read his book and it's quite good. Unfortunately the talk isn't great. Yes, nothing new if you are used to functional programming, immutable data, etc.
There were definitely ways of dealing with immutable data before Clojure implemented Hash Array Mapped Tries in the late 2000s. Functional programming languages had been doing that for years. Rich Hickey didn’t invent HAMTs, that was Phil Bagwell (he said Hickey “discovered” them. Not sure if that’s meaning Hickey used Bagwell’s paper, or not).
Structural sharing is different to HAMT, and is bread and butter for functional programming languages, but using the same techniques in non FP languages with mutable data has always ended up being fiddly and error prone if it’s not built in. Sure, you *can* do it, but without the guarantees of immutability, and without a language optimized for handling GC overhead as a result, it’s never great.
JSONSchema is great, BUT it’s really about validating data at the edge of a system where it’s receiving input. It’s quite expensive, and there’s NO WAY I’d want to be doing that every time an internal function is called in an app just because there are no guarantees about the shape of any data the app is using.
Its very interesting that his idea of what data oriented means is very different from what other people mean.
The game developement/handmade/lowlevel comunity sees data oriented design as "removing all thats not strictly necessary for the functioning of the program" or "getting the clowns out of the car" as formulated by Mike Acton. Of course the result is less code thus easier to understand and by almost coincidence better performance.
His idea of data oriented approach is a lot more opiniated and relies mostly on ideaological statements. It mostly boils down to "keep data around and transform it".
I find this in direct contradiction with data oriented design which begins with the question: what is it you are trying to do? Load an image process it write some logs and exit. Okay then do exactly that an no more. There is no reason to have classes, factories, singletons, concept of resposibility etc. because fundamentally they are nowhere in that description of needed steps. The steps are take this and do that then take the result and do the next thing.
I have seen his idea of data oriented design in the closure comunity before (or maybe its just an echo of his book) and find it very interesting how different comunities can interpret an idea totally differently (and wrong if you ask me).
Data-Oriented PROGRAMMING ≠ DESIGN
11:00 I don't see how the diagrams are simpler. Instead of one mess I now have two messes.
Fewer edges for each individual node make each a simpler system, assuming the addition of an edge to a node is more than a linear increase in complexity.
@@melo682 yeah, I don't see how that makes it any simpler. You still have the 1:1 mapping from data to functions.
You have a stack of papers. You shove them around and make two stacks of papers out of it. Nothing was accomplished.
@@Tekay37the benefits I can think of in terms of code are splitting classes into records and functions forces the code to avoid referring to data via member variables.
I broadly agree with the idea presented in the talk, after all we model our data in SQL, which is entirely separate from the application. But I think the talk suffers from insufficient examples
@@melo682 I do think DOD makes code simpler as well. I just don't see that benefit in the given diagrams. There can be much better examples. (e.g. the practical optimizations talk Jason Booth)
He also mentioned that you don't need the second graph. So now it's only one. Even with two graphs it's still simpler because you've broken apart a big mess into two smaller messes.
The presentation was unconvincing and shallow, in my opinion.
Nothing concrete was presented. It's all slide-ware, ideological claims (e.g. "this way is better/simpler"), non-sentences (e.g. "data is represented as data"), and a few minutes of random meditation (??).
The speaker argues that moving away from classes to describe data and instead of using untyped blobs of JSON + JSON schemas is a better approach. Of course there are no absolute "better" approaches in CS. Only approaches better suited for some problems. For instance, if compatibility is important, you could argue that having a server and a client be flexible about the data they share is important. This is why attributes are all optional in recent versions of Protobuff for instance. But the speaker doesn't share such nuances/trade-offs. They claim that's it's generally simpler/better without talking about any downsides.
It's essentially ideology we are served. In our field of science, we expect strong methodology and evidence. Nothing of the sort was shown here.
Totally nothing new here - and the performance is terrible: DOOM would run at 1FPS using HAMT
@11:00 I'm not seeing the value added here... why not just call this Procedural and leave it be?
"Static type systems can not express that an integer is included in a range"
Yes, yes they can. Scala 3 can, and so do some proof based languages.
Typescript can easily
@@aybgim3850 how?
But the value of the number is only knows at run time!
Using tagging/branding
@@viebel Never heard of Dependent Typing, or Opaque Types?
what's the difference between this and dependency injection or declarative based programming
?
How is this related in any way to dependency injection? This is more about immuatbility and functional programming I guess. Nothing new as he admits in the conclusion.
This felt pretty pointless
So suffice it to say, he is pro- manager classes and/or global functions and static methods. And that means you won't be able to reason about it as well, actually. Maybe you understand the system conceptually but if there are parallel calls going on, i think it'll be more difficult to debug. Singletons and globals are generally what i'd stay away from. Separate models are something i keep at the boundary but try to convert to domain specifications as true class definitions.
I don't agree that global functions or static methods, on their own -- without shared state, are a problem. Why would they be? If anything they are easier to reason about because they have reduced surface area of references that can change at run time. I also don't think this talk suggests using singletons or globals for managing state.
Why is there confusion that this is the same as Data Oriented Design ? Looks like a lot of people seem to assume its the same thing, does the presenter say DOD == DOP ?
Because they both start with "Data oriented". There is also object oriented programming and object oriented design, and they are both the same thing. But what's worse, the speaker/author here is completely discounting the methodologies that came before him. What he calls DOP is actually just Functional Programming as many comments here mentioned. The real "data oriented" paradigm is what Mike Acton started talking about 10 years ago along with many other video game engine developers and system engineers that felt the need give a name to a paradigm that is very different from OOP, functional programming, and procedural programming.
@@vakey I feel like the video ge style DOP is more focused on cache and hardware optimisation. Whereas this is talking about design but without the scary function ideas of monads. Thanks for the heads up on the references. I will be sure to check them out.
Didn't watch the whole thing but I feel like the idea hs is promoting is treating everything as data instead of a model that modeling some concept in real world.
I feel like some type of people might like these idea and might be benefical to them but majority like me are only human. However, It is also similiar to "everything is a map" anti-pattern. Put everything in a map, pass the map to others and then receive the output as a map. You also have to constantly check whether a key-value pair is inside the map or not. And whenever you meed to do something like control flow or make a decision, you need to do all checking for all related properties all over agin in every layer. People have seen "everything is a map" in their life know how horrible is. this kimd of philosophy only guarantee you creating a dead-on-arrival system that the whole thing turns into an ununderstandable, unreliable, unreadable, unmaintainable, difficult to chnage, incapable to reason with and error-prone technical debt immediately.
I wonder if the best of both worlds would be advanced type inference on the fields of those maps (say through row polymorphism)... Extend that to tuple types (similarly encoding the type or validator function at each position), and you've got these nice general collections and the code the reuse through generic functions Yehonathan is talking about, all without having to write rigid type definitions but there would be strong type checking preventing you from making the same kind of mistakes that up front type definitions prevent you from making.