I didn't understand anything at all but I followed you coding steps and I can say I have a great start on my coding language! I have never done anything like this before I am a Html5 person and I sometimes use python but never something like this when I ran the language in my power shell I was screaming literally. great tutorial!
I really liked the video content! I've been trying to find quality content about creating programming languages in youtube for a few years now, but I couldn't find anything relevant, until a found your channel 🤩🤩🤩
This is very interesting - but I have a slight question: I've noticed that at 31:29 you're adding another character in the while condition. Couldn't you just check the *type* of the token instead? If it's a binary operator then you can check that given that all binary operators are on the same level of prescidence we're handling. I've done this on my project as I'm going along and everything's working just fine.
The reason we handle this differently is because not all binary operators have the same precedence. Addition and Subtraction are lower than Multiplication, Division, and Modulus. However these are all higher precedence than Logical and Relational operators like OR, AND, Equals, Not equals, Less, Greater, and even bitwise or and bitwise and have different precedence. This is why it's handled this way. In this series I have made a good amount of fumbles. Some of the code could be more concise and less verbose. Also the approach I used to parsing is very primitive for the purpose of teaching. So I would always recommend finding multiple sources and reading source code for a language. I wish I could go into depth in everything but then I would be making 20 hours of videos just on the tokenizer lol. Anyway thanks for the comment and j.hooe you enjoy the series. I suggest checking out my more recent series on Pratt parsers and if you want more info lookup "Operator Precedence parsers & Pratt parsers". There are many great resources which go much deeper than myself xD.
@@tylerlaceby Thank you for the thoughtful response. Oh woops you're right, running "12 - 1^2" should output "11" but I get "121." I see, well, that makes sense now. I'll definitely check your new videos out once I'm done with this series. By the way, your series so far has been pretty good actually, I've been following along (using vanilla js and node instead of deno, lol) and while I had to google a few things here and there, I get what it does and I can do basic arithmetic.
the parser has always been the part I procrastinate most on, so I usually use a parser generator cuz it's faster and there's less to procrastinate. Of course, when my language is ready to progress I'll have to write a parser by hand :))
Yes there are a few functions in the global JavaScript scope which we may use. parseFloat and parseInt are amongst the most common. But I don't think there are many others I use.
No plan on doing so. However I would suggest first taking a language like vue and translating it into JavaScript. Pick one feature and start there. If you start with one thing at a time it should become a bit easier. You can also use existing parsers and tools for JavaScript and HTML/ CSS parsers.
I can’t say as I don’t know how the logic was changed. I suggest comparing the differences in your code to mine. Feel free to join the discord server and post the code and your error in there and I can help
I have no clue what you are attempting without using C or C++ haha. No offence but you're heart is into it heavy duty. The whole point of having a Lexer is to make a syntax parse-able for and by the strtok() function in C or C++ to sequence calls to this function to split syntax strings into tokens, which are sequences of contiguous characters separated by any of the characters that are part of delimiters. A lexer takes a syntax and breaks it down by placing spaces between every special character and by grouping and preserving all keywords, user defined variable names, user defined strings. numeric mathematical groupings and does all of it for strtok() to work. Now by naming of all double characters(i.e. ==, !=, ++, --, = etc.) in order for a strtok()Token Stream in the C Languages to be able to parse everything in a more organized way. You must do the following: Doubles are defined so that they become one single token for each pair as example == would become EQEQ or ++ would become INCR for increment operator. Single characterss are easily managed by doing this also such as "=" as EQUAL, "(" as LPAREN, ")" as RPAREN etc. Precedence, States and Logical Orders are not for a Lexer to perform, they are for the Parser and are achieved by using Temporary State Switches(which are temporary variables) that store a type-defined numeric definition for look-a-head's to generate conditionally all the rules for your precedence, states and logical orders in you're new computer language. Node Traversals are only a fancy name and definition for managing Stately Switches and is all they are. So your managing basically optcode-states for the following which all have rules. Targets Definitions Assignments Terms(conditional if's else's) Operators Expressions Example of use: int --operater state-- typedef optstate { MULT=1, DIV=2, ADD=4, SUB=5, etc. } --precedence state-- typedef precstate { LPAREN=1, RPAREN=2, } if(tok[i]=="(" && tok[i+1]) { precstate=LPAREN; } You have the right heart and brains, so why not learn C or C++ and do this the correct way because all your doing is throwing out Organized Despair to those who follow you in blindness causing them to fall off the proverbial cliff of despair. You have the brains so do it son. Thanks and delete this as soon as you read it.
I use c & cpp for my own compiler actually. The point of this series is to teach the basics in a was to understand way. Just as we teach mathematics or any other topic In CS with a much more high level view than reality entails. Thanks for your input and enthusiasm however I will remain doing what I’m doing.
![[#2024MAMA] G-DRAGON - 무제(Untitled, 2014)+POWER+HOME SWEET HOME+뱅뱅뱅+FANTASTIC BABY | Mnet 241123 방송](http://i.ytimg.com/vi/Ox29z5Nf1Uk/mqdefault.jpg)
![[MV] A leap of faith...ความเชื่อใจครั้งสุดท้าย (From "Petrichor the series")](http://i.ytimg.com/vi/U1piZH2CNXA/mqdefault.jpg)
This is the most exciting thing I’ve ever watched.
Same for me! 🙂
Like bro my own coding language!
I finally understand precedence properly thanks to this video after many years of trying it myself
Same :)
THE most important video ever produced on this topic!
I didn't understand anything at all but I followed you coding steps and I can say I have a great start on my coding language! I have never done anything like this before I am a Html5 person and I sometimes use python but never something like this when I ran the language in my power shell I was screaming literally. great tutorial!
This is a top tier computer science channel
You're too kind. Thanks for the kind words.
Really liking the series and the way you explain things, great work 🔥
I really liked the video content! I've been trying to find quality content about creating programming languages in youtube for a few years now, but I couldn't find anything relevant, until a found your channel 🤩🤩🤩
thanks alot, i was stuck on expression parsing for the past 3 weeks
Amazing, I was wondering how its made!
Guess I'm ready for part 4 lol
nice job! exactly what I was looking for
This is very interesting - but I have a slight question: I've noticed that at 31:29 you're adding another character in the while condition. Couldn't you just check the *type* of the token instead? If it's a binary operator then you can check that given that all binary operators are on the same level of prescidence we're handling.
I've done this on my project as I'm going along and everything's working just fine.
The reason we handle this differently is because not all binary operators have the same precedence.
Addition and Subtraction are lower than Multiplication, Division, and Modulus. However these are all higher precedence than Logical and Relational operators like OR, AND, Equals, Not equals, Less, Greater, and even bitwise or and bitwise and have different precedence.
This is why it's handled this way. In this series I have made a good amount of fumbles. Some of the code could be more concise and less verbose. Also the approach I used to parsing is very primitive for the purpose of teaching.
So I would always recommend finding multiple sources and reading source code for a language. I wish I could go into depth in everything but then I would be making 20 hours of videos just on the tokenizer lol.
Anyway thanks for the comment and j.hooe you enjoy the series. I suggest checking out my more recent series on Pratt parsers and if you want more info lookup
"Operator Precedence parsers & Pratt parsers". There are many great resources which go much deeper than myself xD.
@@tylerlaceby Thank you for the thoughtful response.
Oh woops you're right, running "12 - 1^2" should output "11" but I get "121." I see, well, that makes sense now. I'll definitely check your new videos out once I'm done with this series. By the way, your series so far has been pretty good actually, I've been following along (using vanilla js and node instead of deno, lol) and while I had to google a few things here and there, I get what it does and I can do basic arithmetic.
I love this, helps me understand compilers
the parser has always been the part I procrastinate most on, so I usually use a parser generator cuz it's faster and there's less to procrastinate. Of course, when my language is ready to progress I'll have to write a parser by hand :))
Yea I’m the other way. I love the parser and procrastinate a lot with the interpreter xD
However the bytecode generation and vm is really fun even if a bit tricky.
@@tylerlaceby Yeah, working with bytes makes me feel very smart
noticed that parsefloat isnt defined anywhere?
edit:
i later found out that its a js/ts function while i'm using c++ and std::stof is the equivalent
Yes there are a few functions in the global JavaScript scope which we may use. parseFloat and parseInt are amongst the most common. But I don't think there are many others I use.
that was really informative!
Do I need EOF? Couldn't you just check the length?
For this series you could. However there may be reasons to check for EOF when dealing with multiple levels of lookahead
@@tylerlaceby Yeah, nevermind, I wrote a parser and I see how an EOF token is useful. It essentially allows me to always have a token to eat or get.
Will you cover how to make a transpiler? I want to make something like a vue file but transpile it to native web components. Any clue?
No plan on doing so. However I would suggest first taking a language like vue and translating it into JavaScript. Pick one feature and start there.
If you start with one thing at a time it should become a bit easier. You can also use existing parsers and tools for JavaScript and HTML/ CSS parsers.
Just asking is it good practice to use both snake case and camelcase ?? Or stick with one .
Mostly stick to one
Invite link to the discord has expired. Also I suggest you also make a matrix room.
What’s a matrix room. Also I’ll fix the link ASAP. Thanks 🙏
Just thought you might want to know - github link is outdated :) username is wrong (it's okay in later videos though)
When testing my repl, I type in 10 but I keep getting this "Unexpected token found during parsing! { type: 7, value: "EndOfFile" }"
I can’t say as I don’t know how the logic was changed. I suggest comparing the differences in your code to mine.
Feel free to join the discord server and post the code and your error in there and I can help
damn i'm looking for a c++ tutorial. anybody finding one?
This was a dope ass video!
Haha glad you found it helpful
Github link is no longer working : (
Sorry about that.
It’s GitHub.com/tlaceby
vscode theme?
XCode11
@@tylerlaceby r u gonna make more videos
I understand most of this, but every time he says "like so" or "simply" --- DRINK !
Haha
ouch my brain
good video tho
i cant get source code 😅
GitHub.com/tlaceby/emu
Thanks!
I'm now at 21:21 can't wait to get to the interpreter
Deno is not working
?
@@tylerlaceby it says its not defined
I have no clue what you are attempting without using C or C++ haha. No offence but you're heart is into it heavy duty.
The whole point of having a Lexer is to make a syntax parse-able for and by the strtok() function in C or C++ to sequence calls to this function to split syntax strings into tokens, which are sequences of contiguous characters separated by any of the characters that are part of delimiters.
A lexer takes a syntax and breaks it down by placing spaces between every special character and by grouping and preserving all keywords, user defined variable names, user defined strings. numeric mathematical groupings and does all of it for strtok() to work.
Now by naming of all double characters(i.e. ==, !=, ++, --, = etc.) in order for a strtok()Token Stream in the C Languages to be able to parse everything in a more organized way. You must do the following:
Doubles are defined so that they become one single token for each pair as example == would become EQEQ or ++ would become INCR for increment operator. Single characterss are easily managed by doing this also such as "=" as EQUAL, "(" as LPAREN, ")" as RPAREN etc.
Precedence, States and Logical Orders are not for a Lexer to perform, they are for the Parser and are achieved by using Temporary State Switches(which are temporary variables) that store a type-defined numeric definition for look-a-head's to generate conditionally all the rules for your precedence, states and logical orders in you're new computer language.
Node Traversals are only a fancy name and definition for managing Stately Switches and is all they are. So your managing basically optcode-states for the following which all have rules.
Targets
Definitions
Assignments
Terms(conditional if's else's)
Operators
Expressions
Example of use:
int
--operater state--
typedef optstate { MULT=1, DIV=2, ADD=4, SUB=5, etc. }
--precedence state--
typedef precstate { LPAREN=1, RPAREN=2, }
if(tok[i]=="(" && tok[i+1]) {
precstate=LPAREN;
}
You have the right heart and brains, so why not learn C or C++ and do this the correct way because all your doing is throwing out Organized Despair to those who follow you in blindness causing them to fall off the proverbial cliff of despair.
You have the brains so do it son. Thanks and delete this as soon as you read it.
I use c & cpp for my own compiler actually. The point of this series is to teach the basics in a was to understand way.
Just as we teach mathematics or any other topic In CS with a much more high level view than reality entails.
Thanks for your input and enthusiasm however I will remain doing what I’m doing.