@@webgpu Break it up into 15-30 min at a time over a few days and it’s not so bad. It was VERY thorough and amazing to see how deep the rabbit hole goes, so far.
Hey there. I am planning to do this course. I want to get into the gen ai field. I've already done a different training and built some Langchain programs. But I want to go deeper. Are there any prerequisites for this? Do I need to know pytorch or tensorflow or something else
Overall a good course. It's a bit rough around the edges, but if you are persistent or already somewhat knowledgeable you can get through it and come out a bit smarter on the other end. Worth the almost 6 hours.
Anyone can learn hard concepts from this guy, cuz he turns hard concepts into easily understandable ones! Congrats for being a such amazing teacher!! Teaching is all about facilitating the process of learning, but somehow people tend to overcomplicate things so they can sound smarter than majority. This guy does exactly the opposite of it. Thank u and God bless!
@@brunothedev I'd say you should have decent coding experience before starting, and some knowledge of linear algebra and probability theory will also be helpful. Diving right into a bunch of libraries and trying to understand what's going on would be very confusing if you don't know the basics of programming, and while you might be able to complete the course I'd find it hard to believe a beginner would have truly learned much from it.
I don't know how you did it.. but you somehow explained Transformers in a way this absolute python newbie could parse and understand. Great work and I'm glad I spent the time to follow along with this tutorial.
Your patients and ability to explain in basic terms makes learning easy. Thank you, your efforts and willingness to share information is much appreciated.
Just started this tutorial and I'm sure it's going to be a great course. But a REQUEST -- For future tutorials, please use larger fonts / zoomed-in windows & terminals (just a wee-bit larger would help tremendously). After 30 mins of eye-strain, I started to get a headache. Also, a dark theme (as available in jupyter lab and vscode) would also help.
46:00 np.multiply is elementwise multiplication which isn't comparable to dot product multiplication. To compare gpu to cpu you can use torch's @ multiply for both. Since the second two were not loaded to the gpu, they are computed by the cpu.
The code behind these LLMs is not as complicated as it may seem, the issue is the millions of dollars worth of GPU hours you would need to train a big model. But imagine when that gets cheaper, things are going to get very competitive and interesting in the world of LLMs. This course is the next one on my list, I'm half-way through the FCC courses on Pytorch and OpenAPI at the moment and I think I have to finish at least one of them before starting another. Amazing courses, better than paid ones every time.
Why? Neuralink isn't like The Matrix, "now I know kung fu...." That isn't a thing. I mean if you mean access to the video through a mere thought, go for it. But there is no world, even with all of the magic happening right now, where storing the information is the same as the experience of knowing that information.
@@TheRealNickG No, but using no code tools to take ver the world and learning bit by bit, (like I am currently doing) - is much more efficient than starting this today for me. I saved the video, - I am a BJJ brown belt, I know that nothing will replace thousands of hours and gallons of blood* not to mention multiple combat deployments as an Infantry Paratrooper, I am just simply using echelon of tools for time management. Peace be upon and what not - enjoy your negative Kafka esque rhetoric somewhere else lol.
I mean it's really not an argument. Maybe it is "possible" from a technical standpoint but you're still ignoring the warning I'm trying to send. How are you gonna know how to take a punch if you've never experienced it before? Only by the injection of a synthetic experience of someone else. Deep Space Nine had an episode where these aliens made a dude experience a whole prison sentence in an instant. It almost drove him crazy because they pointed out that he might have looked asleep for only a moment, but 20 years still took its toll. You're still under the naive impression that math is easier than martial arts and can't explain why you can't learn calculus by just watching videos on double speed...... It's because there is no such thing as instant learning and there will never be a universe where learning is different than spending legitimate time with a subject in a way that changes you as a person.
if you have an apple silicon mac you can use "mps" (mac gpu) rather than "cuda" or "cpu". this requires a little bit of extra setup but is quite simple.
Just replace his cuda line with the following lines, and your code will work with Windows CUDA or Mac M1 MPS GPUs device = "cuda" if torch.cuda.is_available() else "cpu" device = "mps" if torch.backends.mps.is_available() and torch.backends.mps.is_built() else device
For MAC users(this is what I did): 1. python -m venv VirtualEnv 2. pip install matplotlib numpy pylzma ipykernel jupyter 3. pip3 install torch torchvision torchaudio 4. Create the kernel: python -m ipykernel install --user --name=gpu_kernel --display-name "gpu kernel" xcode-select --install if you get a ERROR: Failed building wheel for cffi while installing libraries Inside the notebook: device = 'mps' if torch.backends.mps.is_available() else 'cpu'
Appreciate the work you put into this, you definetly know what you are talking about. Thanks! But this course also has a couple of methodical downsides which makes it hard to indepently follow: - your "drawings" could be better, I understand the limitations of "drawing with a mouse on the screen", I suggest to prepare those charts in beforehand - please try to establish a clean coding style, don't use shortend var names like i, x, y, q, wei and so on or dummys like "for _ in range(foobar)" - this makes it really hard to reproduce your code, of course, while in the process it's easier and quicker, but if you get back to the code after 1 month vacation: your lost - promise! - also please use common vocab, like x is usuall X, because it's a tensor, while y is lowercase, showing it's a vector or even better: use "features" and "labels" as var names, epoch instead of iters - sometimes your recorded screen is to small, content is being cut off or the content is too small and barely readable - there are some jumps in your video, like: you showed the simple bigram model, but you did not really finish the work, we trained the model and then what? your notebook has two cells printing some gibberish text, without any explanation. - please try to establish a naming convention for your source files, it's hard to follow A really, like really, good course about Pytorch does all those things, it's Daniel Bourkes intro to Pythorch also on "fcc", highly recommended: th-cam.com/video/Z_ikDlimN6A/w-d-xo.html Again: Appreciate the hard work, please don't get me wrong, it's hard to criticise somethiing that is for free, just want to give back honest and constructive feedback =)
Starting this on 04-04-2024, Planning to complete it in 1 week. Date : 04-04-2024 Complete : 36:50 ( Day 1 ) Date : 05-04-2024 Complete : 1:50:00 ( Day 2 ) I was a little occupied for 2 days but I am back baby Date : 08-04-2024 Complete : 2:32:53 (Day 5)
At what point in the tutorial do you ask the LLM a question and it returns the Wizard of Oz text you trained in the model? I can't find it. I just want to see how well it worked. I've listened to 20 minutes at the start and will finish the whole thing, but curious if anyone knows. None of the chapters say something like "Final Testing to Show it Works."
1h 24 min aprox ---> just in case it helps: # Matrix def A = [[1, 2], [3, 4], [5,6]] B = [[7, 8, 9], [10, 11, 12]] # Initializer C = [[0, 0, 0], [0, 0, 0], [0, 0, 0]] # Product for i in range(len(A)): for j in range(len(B[0])): for k in range(len(B)): C[i][j] += A[i][k] * B[k][j] # Print result print("result:") for row in C: print(row) and the best approach: a = torch.tensor([[1,2],[3,4],[5,6]]) b = torch.tensor([[7,8,9],[10,11,12]]) print(a @ b) print(torch.matmul(a,b)) Paint is horrible in the video...
Machines can only interpret numbers. We collect these numbers into sets, called matrices or vectors (Linear Algebra) or arrays (Python), so that we can operate on the entirety of them, in for-loops. Tensors are parallel sets of for-loops. Linear algebra allows us to visualize for-loops. Calculus allows vectors and tensors (ie. our arrays), via a form of linear regression, such as gradient descent, to behave in a way that is meaningful when iterating through our arrays. Linear regression allows us to define functions. The more these numbers represent something meaningful, the more linear regression we have performed on these numbers. The goal is to find a meaningful way to represent information. Try to remember machines can only interpret numbers so we have to construct a representation onto these numbers and or the operations on these numbers and or the mathematical results that we get from the operations on these numbers or sets of numbers. That's language model engineering.
Hats off to this awesome dude... Whether its math or pytorch methods the explanations are spot on. GPT is no longer some super awesome AI... its just a bunch of Pytorch code. Completed the entire video in single sitting, tonnes of information and amazing tools to gain intuition
This course is insanely incredible and the only con that I can put is that the voice is not clearly at all (not because of the mic just because of the way he speaks), anyway I totally recomend it and it's a really incredible contribution to the comunity, thank you so much
People debating over this course being a scratch or not are funny. If you complete it you will learn alot and it will also help you to understand research papers and implement them.
Thank you very much free code camo team for this excellent jewel, the instructor is immensely talented and has a put a lot of efforts , kudos to him. People should remember that there are very few people in this world who teach such a valuable content for free
Been going through this tutorial. Noticed at 1:07:58, the explanation of the softmax function probably can use a better explanation. It's the dimension over which the softmax function is calculated, so, if you have a 2D tensor, dim 0 calculates the softmax across columns and dim 1 is across rows. For classification purposes, you generally want it calculated for each datapoint, and not for one value each across multiple data points. If all you have is a 1 dimensional tensor as input, then it's just one data point and the first dimension, dim=0, is your only choice.
For everyone having troubles with pylzma, try using an older Python version (3.9) for your venv. He says anything from 3.9 - 3.11 works but I only found 3.9 to work. Pylzma has compatibility issues with newer Python versions, so setting up a virtual environment with Python 3.9 should help resolve installation errors and import problems.
Thanks for the amazing tutorial. Btw, if anyone is looking for passing a text and getting its prediction for the bigram.ipynb >> enc = encode("hel") >> decode(m.generate(torch.tensor([enc]), max_new_tokens=5)[0].tolist()) output : 'helll o ' I trained for 50k iterations ( running locally, so not training more)
You can run Pythonic functions in Mojo but the call has to be wrapped with try-except. E.g. def add(a: Int, b: Int) -> Int: return a + b fn main(): try: var c = add(3,4) print(c) except: print("Error")
These days, high-level languages are virtually placed at an even higher level. “Development” today requires installing or using so many “packages” primarily about passing parameters to interface with the code that does the heavy lifting, rendering high-level development to a superficial notion of what software engineering is and how it works behind the curtains.
At approx 40:20, the matrix with the ones on the diagonal and the rest being zeroes is called an Identity matrix - I'm guessing that's why the torch method is called "eye"? Gives you the idea of what the method does but saves keystrokes :)
nice course :) noone noticed the miscalculation around 1:25:12 the dotproduct of (3x7)+(4x10) should be 61 not 47 but the course is amazing just a little smile that everyone can make failures
Was looking for this comment haha. I'm so overwhelmed by learning all this that I wasn't sure if he made a mistake or if I'm wrong until I got a confirmation by you
4:30:00, I started getting so confused. _init_weights got added which didn't exist before with no explanation, and I could not figure out what file you were working from. You went back to using the wizard of oz text, and randomly switched back from the random batch generation method to the old one. It's like a whole 15 minutes of changes were just missing.
Highly appreciate that you guys make things like this for free!
How do I find the anaconda prompt
There goes my weekend. Thank you! This is absolutely amazing material! I’m 5 minutes in and already hooked.
5 minutes ? Great! just watch the remaining 5 hours and 40 minutes ! 😆
@@webgpu Took more than the weekend, but just finished… 🥵
@@JoeD0403 did you finish watching almost all 6 hrs ??? OMG you deserve an award! 🏆🍻 (i'm sure the time spent on it is absolutely worth it 👍)
@@webgpu Break it up into 15-30 min at a time over a few days and it’s not so bad. It was VERY thorough and amazing to see how deep the rabbit hole goes, so far.
Hey there. I am planning to do this course. I want to get into the gen ai field. I've already done a different training and built some Langchain programs. But I want to go deeper. Are there any prerequisites for this? Do I need to know pytorch or tensorflow or something else
Overall a good course. It's a bit rough around the edges, but if you are persistent or already somewhat knowledgeable you can get through it and come out a bit smarter on the other end. Worth the almost 6 hours.
How do I find the anaconda prompt on Mac
"in this course, you're going to learn a LOT of crazy stuff!" I knew this was going to be a good one!
How did it go?
Anyone can learn hard concepts from this guy, cuz he turns hard concepts into easily understandable ones! Congrats for being a such amazing teacher!! Teaching is all about facilitating the process of learning, but somehow people tend to overcomplicate things so they can sound smarter than majority. This guy does exactly the opposite of it. Thank u and God bless!
Just finished this course. One of the best courses out there to understand basic concepts of LLM. Can't believe this goldmine exists for free on YT
@@Kailahu I think he try to look smarter.
@@Kailahu 80/20 rule, look it up
Hey what are the prereq for this course?
@@akshatphadtare2437 Nothing really, just the patience to research and understand the content
@@brunothedev I'd say you should have decent coding experience before starting, and some knowledge of linear algebra and probability theory will also be helpful.
Diving right into a bunch of libraries and trying to understand what's going on would be very confusing if you don't know the basics of programming, and while you might be able to complete the course I'd find it hard to believe a beginner would have truly learned much from it.
I don't know how you did it.. but you somehow explained Transformers in a way this absolute python newbie could parse and understand. Great work and I'm glad I spent the time to follow along with this tutorial.
Thanks!
Your patients and ability to explain in basic terms makes learning easy. Thank you, your efforts and willingness to share information is much appreciated.
You might want to change "patients" to "patience".
How do I find the anaconda prompt on Mac
Yesterday I was searching for this, and today you dropped it! Great job people 🙌
How do I find the anaconda prompt on Mac
Just started this tutorial and I'm sure it's going to be a great course. But a REQUEST -- For future tutorials, please use larger fonts / zoomed-in windows & terminals (just a wee-bit larger would help tremendously). After 30 mins of eye-strain, I started to get a headache. Also, a dark theme (as available in jupyter lab and vscode) would also help.
A dark theme would help to make us feel depressed. No, thank you.
@@santiagomartinez3417 You aren't a real programmer if you use light mode
@@santiagomartinez3417dark theme ruins your eyes. Try working 10 hours with a white screen. Sheesh
Dark theme make your eye force...
How do I find the anaconda prompt on Mac
I have a hack-a--thon tomorrow and I wanted to make a chat bot from some custom material. Bless you guys! This is perfect!
Great content for free.
Wishing that this hits 1 million views soon.
Keep up the great work.
Finally someone explained transformers properly! Great job!
46:00 np.multiply is elementwise multiplication which isn't comparable to dot product multiplication. To compare gpu to cpu you can use torch's @ multiply for both. Since the second two were not loaded to the gpu, they are computed by the cpu.
Yup. That was why the results of CPU were close to GPU. Element-wise are way quicker.
Wait, can you elaborate a bit. Was he suppose to load the numpy's matrix into GPU as well? @@georgehenrique2560
I get a io.h C1083 file error when I try to instal pylzma, It can't find the file and I've doe everything to make it work. Any idea how to fix this?
Hadamard Product, maybe?
Most Interesting Thing Ever!!!!
Cheers to anyone who followed along with the video and completed it!
The code behind these LLMs is not as complicated as it may seem, the issue is the millions of dollars worth of GPU hours you would need to train a big model. But imagine when that gets cheaper, things are going to get very competitive and interesting in the world of LLMs. This course is the next one on my list, I'm half-way through the FCC courses on Pytorch and OpenAPI at the moment and I think I have to finish at least one of them before starting another. Amazing courses, better than paid ones every time.
Share the link of the two courses you mentioned please.
Yes please
Isn’t it possible to use gpu cloud?
@@annansm4293 of course, thats probably what they meant, but that costs alot of money too.
@@annansm4293 that what he's talking about went he said "...GPU hours(cloud gpu)"
Thanks FCC I have been waiting this kind of Course. At last on this❤❤❤❤
How do I find the anaconda prompt
on Mac
Damn, this looks like a jewel. Will definitely look into it. Thank you for sharing this!
How do I find the anaconda prompt on Mac
Thanks FCC, thanks for your effort Elia!! Good job!
How do I find the anaconda prompt on Mac
You guys are Insane. I was implementing this thing from scratch but failed.
Glad this course helped you out!
Putting this in saved for when I get a neurolink chip 🍟 🙌
Why? Neuralink isn't like The Matrix, "now I know kung fu...." That isn't a thing. I mean if you mean access to the video through a mere thought, go for it. But there is no world, even with all of the magic happening right now, where storing the information is the same as the experience of knowing that information.
@@TheRealNickG No, but using no code tools to take ver the world and learning bit by bit, (like I am currently doing) - is much more efficient than starting this today for me. I saved the video, - I am a BJJ brown belt, I know that nothing will replace thousands of hours and gallons of blood* not to mention multiple combat deployments as an Infantry Paratrooper, I am just simply using echelon of tools for time management. Peace be upon and what not - enjoy your negative Kafka esque rhetoric somewhere else lol.
@@TheRealNickGBut with enough knowledge of brain data encoding.. it could be. What would stop the potential to overwrite with ai?
@@TheRealNickGnever say never 😉
I mean it's really not an argument. Maybe it is "possible" from a technical standpoint but you're still ignoring the warning I'm trying to send. How are you gonna know how to take a punch if you've never experienced it before? Only by the injection of a synthetic experience of someone else.
Deep Space Nine had an episode where these aliens made a dude experience a whole prison sentence in an instant. It almost drove him crazy because they pointed out that he might have looked asleep for only a moment, but 20 years still took its toll. You're still under the naive impression that math is easier than martial arts and can't explain why you can't learn calculus by just watching videos on double speed...... It's because there is no such thing as instant learning and there will never be a universe where learning is different than spending legitimate time with a subject in a way that changes you as a person.
Great work as usual❤ can't wait to deep dive in it
How do I find the anaconda prompt on Mac
if you have an apple silicon mac you can use "mps" (mac gpu) rather than "cuda" or "cpu". this requires a little bit of extra setup but is quite simple.
Can you point me to some documentation on how to set this up?
I have an M1 and since it does not support cuda I am a bit lost
@@contadacarta I'm also interested in a mac set up, getting a lot of 'command not found'
Just replace his cuda line with the following lines, and your code will work with Windows CUDA or Mac M1 MPS GPUs
device = "cuda" if torch.cuda.is_available() else "cpu"
device = "mps" if torch.backends.mps.is_available() and torch.backends.mps.is_built() else device
Thank you! But tensorflow still does not fully support mac.
For MAC users(this is what I did):
1. python -m venv VirtualEnv
2. pip install matplotlib numpy pylzma ipykernel jupyter
3. pip3 install torch torchvision torchaudio
4. Create the kernel: python -m ipykernel install --user --name=gpu_kernel --display-name "gpu kernel"
xcode-select --install if you get a ERROR: Failed building wheel for cffi while installing libraries
Inside the notebook:
device = 'mps' if torch.backends.mps.is_available() else 'cpu'
Thank you so much
it's so funny, gpt 4 helped me thru that entire thing lmaooo
Does this work on intel Macbook Pro having Intel's integrated GPU?
I am not able to install pylmza any solution it says wheel error
@@amriteshamrit7128 it seems there is no need to install pylzma. It is native starting since python 3.3
Awesome stuff! Congrats on being such a good teacher!
Appreciate the work you put into this, you definetly know what you are talking about. Thanks!
But this course also has a couple of methodical downsides which makes it hard to indepently follow:
- your "drawings" could be better, I understand the limitations of "drawing with a mouse on the screen", I suggest to prepare those charts in beforehand
- please try to establish a clean coding style, don't use shortend var names like i, x, y, q, wei and so on or dummys like "for _ in range(foobar)" - this makes it really hard to reproduce your code, of course, while in the process it's easier and quicker, but if you get back to the code after 1 month vacation: your lost - promise!
- also please use common vocab, like x is usuall X, because it's a tensor, while y is lowercase, showing it's a vector or even better: use "features" and "labels" as var names, epoch instead of iters
- sometimes your recorded screen is to small, content is being cut off or the content is too small and barely readable
- there are some jumps in your video, like: you showed the simple bigram model, but you did not really finish the work, we trained the model and then what? your notebook has two cells printing some gibberish text, without any explanation.
- please try to establish a naming convention for your source files, it's hard to follow
A really, like really, good course about Pytorch does all those things, it's Daniel Bourkes intro to Pythorch also on "fcc", highly recommended: th-cam.com/video/Z_ikDlimN6A/w-d-xo.html
Again: Appreciate the hard work, please don't get me wrong, it's hard to criticise somethiing that is for free, just want to give back honest and constructive feedback =)
Starting this on 04-04-2024, Planning to complete it in 1 week.
Date : 04-04-2024 Complete : 36:50 ( Day 1 )
Date : 05-04-2024 Complete : 1:50:00 ( Day 2 )
I was a little occupied for 2 days but I am back baby
Date : 08-04-2024 Complete : 2:32:53 (Day 5)
DID U GIVE UP
@@notme_1128 No I did not, I was just preoccupied with a few other things. Now I am back ;)
@@heizou007 does it work and does it make reasonable responses?
@@ajbeery2920 following his work, it shouldn't make a reasonable response
you gave up, didn't you?
damn , you just challenged me . am also studying bsc math but also am into cybersec . thanks mahn for real
This channel is a gift for humanity
At what point in the tutorial do you ask the LLM a question and it returns the Wizard of Oz text you trained in the model? I can't find it. I just want to see how well it worked. I've listened to 20 minutes at the start and will finish the whole thing, but curious if anyone knows. None of the chapters say something like "Final Testing to Show it Works."
An actual developer can take what is in any video tutorial and integrate and test it.
1h 24 min aprox ---> just in case it helps:
# Matrix def
A = [[1, 2],
[3, 4],
[5,6]]
B = [[7, 8, 9],
[10, 11, 12]]
# Initializer
C = [[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]
# Product
for i in range(len(A)):
for j in range(len(B[0])):
for k in range(len(B)):
C[i][j] += A[i][k] * B[k][j]
# Print result
print("result:")
for row in C:
print(row)
and the best approach:
a = torch.tensor([[1,2],[3,4],[5,6]])
b = torch.tensor([[7,8,9],[10,11,12]])
print(a @ b)
print(torch.matmul(a,b))
Paint is horrible in the video...
Machines can only interpret numbers. We collect these numbers into sets, called matrices or vectors (Linear Algebra) or arrays (Python), so that we can operate on the entirety of them, in for-loops. Tensors are parallel sets of for-loops. Linear algebra allows us to visualize for-loops. Calculus allows vectors and tensors (ie. our arrays), via a form of linear regression, such as gradient descent, to behave in a way that is meaningful when iterating through our arrays. Linear regression allows us to define functions. The more these numbers represent something meaningful, the more linear regression we have performed on these numbers. The goal is to find a meaningful way to represent information. Try to remember machines can only interpret numbers so we have to construct a representation onto these numbers and or the operations on these numbers and or the mathematical results that we get from the operations on these numbers or sets of numbers. That's language model engineering.
this made things easier to think process and understand thanks
Perfect timing. I was looking for tutorial for something like this today.
Glad I could get this timing for ya!
Really good tutorial, it's feels like someone is explaining it from yours perspective
this feels like it should be paid its amazing that we can learn things line this explained very well for FREE
Awesome, buddy!! This is great service to AI and tech community and you're sharing for free!!!
Hats off to this awesome dude... Whether its math or pytorch methods the explanations are spot on. GPT is no longer some super awesome AI... its just a bunch of Pytorch code.
Completed the entire video in single sitting, tonnes of information and amazing tools to gain intuition
This huge sharing - thank you 🙏🏻🤙🏻
If you are on a Linux based system, its not `cuda/Scripts/activate`, its `cuda/bin/activate`
you are a legend. Thank you!
This course is insanely incredible and the only con that I can put is that the voice is not clearly at all (not because of the mic just because of the way he speaks), anyway I totally recomend it and it's a really incredible contribution to the comunity, thank you so much
2:32:36 diselooooooooooo
I am so exited when he start explaining the multi-head attention, thank you.
1 hour 30mins done - took 3 hours to complete worth every second
Such maturity for Such a Young Person!!
This is gem for whoever want to know transformer model. Bingo!!👏👏
How do I find the anaconda prompt on Mac
People debating over this course being a scratch or not are funny. If you complete it you will learn alot and it will also help you to understand research papers and implement them.
Thank you very much free code camo team for this excellent jewel, the instructor is immensely talented and has a put a lot of efforts , kudos to him. People should remember that there are very few people in this world who teach such a valuable content for free
Mistake in 1:46' - When you create a class that uses nn.Module you HAVE to define the 'forward' method. It is in the Torch docs for nn.
I finished this course during the weekend and launched my AI company. Google decided to acquire it for 50 million. Thanks!
What’s the company is all about? And what is its name?
I like your style. Well done, Caballero. And, thank you.
Awesome work man. Lots of appreciation. Keep doing good stuff .
Been going through this tutorial. Noticed at 1:07:58, the explanation of the softmax function probably can use a better explanation. It's the dimension over which the softmax function is calculated, so, if you have a 2D tensor, dim 0 calculates the softmax across columns and dim 1 is across rows. For classification purposes, you generally want it calculated for each datapoint, and not for one value each across multiple data points. If all you have is a 1 dimensional tensor as input, then it's just one data point and the first dimension, dim=0, is your only choice.
Favorite result so far:
propmt: "Can you see me?"
generated chars:
CHAPTER 9.
THEY FIGHT THE INVISIBLE BEARS
You have a great skill! Very explanatory!
For everyone having troubles with pylzma, try using an older Python version (3.9) for your venv. He says anything from 3.9 - 3.11 works but I only found 3.9 to work. Pylzma has compatibility issues with newer Python versions, so setting up a virtual environment with Python 3.9 should help resolve installation errors and import problems.
Great job. You have an excellent communication style. Thanks so much.
Simply gold. Really impressed.
How do I find the anaconda prompt on Mac
Amazing video. Absolutely love it.
Adding time as my checkpoint
1. 55:15
2. 2:35:00
Awesome video , well explained. You have a great future in IT research
Elliot brother congrats bud
Free code camp releasing fire 🔥 tutorials these days
I have started learning this today I am dedicated to learn these concepts and apply it in next 15 days! Please follow up If I dont complete it.
Really hooked dude you can be serious if this course is good the you're a god in teaching computer programming
How do I find the anaconda prompt on Mac
justin beiber teaching llms
Thank you for sharing this information with us ❤
How do I find the anaconda prompt on Mac
Thanks for the amazing tutorial. Btw, if anyone is looking for passing a text and getting its prediction for the bigram.ipynb
>> enc = encode("hel")
>> decode(m.generate(torch.tensor([enc]), max_new_tokens=5)[0].tolist())
output : 'helll o '
I trained for 50k iterations ( running locally, so not training more)
Thank you so much sharing this knowledge with us.
Il learnt from this channel , What I could not from college and those 000 costing courses !
You can run Pythonic functions in Mojo but the call has to be wrapped with try-except. E.g.
def add(a: Int, b: Int) -> Int:
return a + b
fn main():
try:
var c = add(3,4)
print(c)
except:
print("Error")
Wow wow this is just an amazing time for me to delve deeper into AI coding
hey, i tried till everything as he said, 0:32:13, but's showing 'cpu' instead of 'cuda', what should i do?
you probably dont have a gpu
40:21 that matrix is called a diagonal matrix , where the diagonal element is not zero and rest of elements are zero
Really great explanation
are you guys gods? 🙇Huge respect as always
Wow. This kid is brilliant 🙏
loved this!!
These days, high-level languages are virtually placed at an even higher level. “Development” today requires installing or using so many “packages” primarily about passing parameters to interface with the code that does the heavy lifting, rendering high-level development to a superficial notion of what software engineering is and how it works behind the curtains.
beautiful - nicer explanination based on Andre Karpathy
At approx 40:20, the matrix with the ones on the diagonal and the rest being zeroes is called an Identity matrix - I'm guessing that's why the torch method is called "eye"? Gives you the idea of what the method does but saves keystrokes :)
How do I find the anaconda prompt on Mac
Lets goooo!
Good coarse! Thanks Elliot!
How do I find the anaconda prompt on Mac
Hows the hardware requirement ? Any minimum CUDA requirement?
Damn this dude looks like 19, same age as me, and is making LLMs. Incredible work.
5:25:26 would have been better if the output was not gibberish
How should we make it to give an output that holds some sort of meaning ?
training it multiple times improves the result to some degree. infact it gives expected text a few times.
Just amazing works.
Merci beaucoup !
How do I find the anaconda prompt on Mac
nice course :) noone noticed the miscalculation around 1:25:12 the dotproduct of (3x7)+(4x10) should be 61 not 47 but the course is amazing just a little smile that everyone can make failures
Was looking for this comment haha. I'm so overwhelmed by learning all this that I wasn't sure if he made a mistake or if I'm wrong until I got a confirmation by you
This course provides a great introduction to building LLM's, but would have loved to also have the code in TensorFlow
Love this guy!
definitely gonna watch this after my final exam
Im doing this now, looking up informational vids thank you
How do I find the anaconda prompt on Mac
Wonderful Tutorial !!!✨
How do I find the anaconda prompt on Mac
Awesome video thank you.
thanks bro your dedication.
nice overview, thanks
Thanks, learnt a lot
How do I find the anaconda prompt on Mac
Booking this for future reference
How do I find the anaconda prompt on Mac
If you are somewhat into deep learning, the video starts in 2:44:00
4:30:00, I started getting so confused. _init_weights got added which didn't exist before with no explanation, and I could not figure out what file you were working from. You went back to using the wizard of oz text, and randomly switched back from the random batch generation method to the old one. It's like a whole 15 minutes of changes were just missing.
when you say "install libraries" i left this place
😂 this is where the hype stops