Copilot helped me a lot with porting code from one language to another. You can leave the original code in a comment above where you're about to write and it manages to translate it really well. It's especially interesting to see that it can translate concepts from one language to another, even concepts that you've special made for your code base.
Wow, that's mind blowing! Didn't know it was this powerful, especially the sentiment analysis example. It's kind of like using programming languages as an "on the go" interface to other transformer models and seeing what they can do. Great video!
Wow, that example with a non-existing library was incredible. I have been using co-pilot for a while now and did not realize it had such amazing predictability.
3 ปีที่แล้ว +7
I've been using copilot for a few months and I really like it. I had a few "wow" moments, but most of the time it just saves time doing simple things, finishing lines (or methods) and even works as a more advanced autocomplete, instead of writing a few letters and getting completions for each of them, it guess the whole package/method you want right away. I got so used to it that when I'm paired with a colleague it feels like there's something off with their IDE, like they are missing an important resource. It's similar to that feeling when you step down from an IDE to use plain text and misses autocomplete, auto import, shortcuts, etc.
I tried writing whole projects with co-pilot and it did not work but writing parts of an app like the popup window with tkinter worked exceptionally well. Also scrapping websites just by saying “scrap (website name)” worked super well. I did not even have to go and find the tag under which the content was located all i needed was to specify if it was title or body and it automatically picked the right one. It is truly amazing to see what it can do.
If you focus heavily on object-orientated coding and break things into modular sections it works scarily well, especially if you tell it what each function/class inputs and outputs.
Yeah, my project which is only two months old, but reaching about 50,000 lines, it was basically no help, as I said in few other comments it's great for small things, but those who are worrying about jobs, if you're just a scripter then yeah you might have more competition now, but I suspect there's enough work out there.
Yeah, been extremely unimpressed with copilot and now admittedly I have mainly attempted to use it with elixir and rust code. With elixir code, it suggests nothing but Ruby with small modifications to fit the elixir syntax. Most of the expressions will not even be valid. I know elixir is not an expressly supported language but you'd think a tool like this could see the context of the file extension. For rust code the issues are much more subtle but still problematic. Since I work in the crypto space, much of what I do is very abstract, copilot has a ton of trouble parsing very abstract code. And in regards to statically typed languages it's almost like copilot makes up its own types like 20-30% of the time. This is especially true for macro derived types. This tool isn't going to take anyone's job any time soon. And it has problems with licensing in the open source community. It's difficult to know if the code it's pulling in is something that it figured out on its own or something that it basically copied from another repo and worse still, it doesn't really respect licences so the code could get you into trouble. I also would never use this tool when writing proprietary code since it's hard to know where your code is being broadcasted. If you use this to write a secure algorithm then you should be extremely careful. I am also worried that this will further dumb down the next generation of coders. It's already the case that developers will learn frameworks and not the underlying concepts so that they can get a job really easily. The problem with this approach though is that these developers won't be able to do anything non-trivial outside of the scope of the framework (there are plenty of JS devs who don't know vanilla JS for example). Co-pilot feels a bit like that; If all you're doing to write code is writing comments, how do you gain a deeper understanding of the code that's being outputted. And I think that's really where us developers are going to be with tools like this emerging. We will always be necessary in some capacity, because even if they create more complex versions of copilot which can build out entire apps with ease, a developer is still going to have to be there to make sure the code is consistent and workable. Also, software is not math or science; there are multiple ways to do something and proving that its complete and consistent is difficult (even with theorem provers its hard to prove a piece of software doesn't have state leaks and potential edge case bugs).
You make good points on laziness and growing ignorance of developers, too many relying on pre-built solutions while having no understanding or ability to innovate on top of it, it's well known that most code written today is very inefficient and the usual excuse is that we'll throw more computing power to counter it, which will get quite interesting when we reach the limits of current silicon miniaturisation which isn't that far off. Also I agree it won't affect programmers, but I can see this affecting a lot of scripters, data entry/science peeps, bot makers and those that don't really venture deeper into programming where you have to contend with complex data structures/patterns, generics, APIs and so on. For example my current project(about two months old) must be reaching about 50,000 lines of code and Code Pilot has a far lower limit than that and I suspect it wouldn't understand most of the context of what I am doing anyway. In short those who aren't simple scripters should be fine, though I do fear the code that is produced might be riddled with bugs and inefficiencies. but beside that it's a cool tool, but will it fall to the wayside like GPT-3 kind of has people were saying the same about it but it's mostly not talked about much anymore.
The argument about laziness is a logical fallacy sifting blame from people to tools. As a start, if you can readily land jobs by learning just the framework and not the underlying concepts, then that is what the business needs, otherwise they would not be constantly hired and retained. Wrong or right, its not a problem with the tools, nor it can be an argument against the existence and further development of those. In my 22years of experience working as a developer and managing others, there have been 0 cases where someone stood out because they wrote everything by hand or memorized syntax. If anything these people would usually make an impression for the wrong reasons - trying to look down and demean colleagues who don't fit their "standards" while being the "work hard not smart" type themselves. Also the "dumbing down" has been going on for way longer, and the same arguments have been said for absolutely any developed tool or framework. It already proven invalid.
I have been using coppella for a couple of months as well honestly it has blown me away as well. I think it cuts both ways, if you're an inexperienced programmer it may be a bad thing to lean on, but an experienced programmer may leverage it properly. Still pretty awesome
It feels almost..... magical. Like It knows what I want to do. It writes all the boilerplate for you and can even write full utility functions, you only need to hint the purpose of function in the name like vector2string() And it even writes custom string literals for you. It know ansi escape sequences for different colours. So even If you write :- std::string operator ""_grn(const char*) It will be automatically generated with correct code to create a string with green ansi color code
There are a few potential problems I see with copilot that weren't addressed here. First and foremost, as you said in the video, it does make mistakes. This is to be expected, but these mistakes might be much harder to catch and debug, especially for inexperienced programmers who might be relying on copilot to write code for them. Second is the legal aspect. Copilot was trained on Github code, and as far as I know, it will sometimes spit out an exact copy of some code it saw somewhere. This may have changed, but as of recently, Microsoft made it clear that it was the programmer's job to check for copyright infringement on code that copilot writes. For small projects, this is a difficult task. For large projects, or anything commercial, it makes copilot unusable. Curious to hear your thoughts about this.
For mistakes in code: I see this as no different than what you might copy and paste from SO or something. Most often the mistake still runs, but isn't what your intention was, so I think the pitfall is the same if you're willing to accept some block of code without really looking. Copilot just lets you do it faster. Copyright: To be honest, I am just not seeing it. The claim from copilot is 0.1% is regurgitated. Given how transformer models actually work, and from my own testing, I am inclined to believe this number and it would seem copyright is a non-issue here.
Copy writing code is a bit fuzzy to me. There is the obvious case of blatant copy paste of large sections or entire programs without permission. But for small snippets, there are only so many possibilities. For example you want a recursive method to return fibonacci for your input. It's essentially a few lines of code, and there are only so many ways of writing it that makes sense in your respective language. Or a sorting algorithm. It's not copying, its implementing the correct solution for the problem. Should that solution be owned by someone and everyone has to write it different? I think it's similar to copy-writing a melody, there are only so many notes you put in so many combinations, like the experiment Damien Riehl did with generating all the melody permutation on the harddrive so they have legal copyright status. Fast-forward to a dystopian world where every permutation of code has been made so everything is copywrite? Idk, seems short-sighted and obtuse. Not concerned tho, I don't see it happening. Just interesting to think about. I suppose we can reframe the question as "what is the threshold of code similarity in which infringement has occurred?".
ja so its good for prototyping. like python is not great for production for many thing will have to convert to c++ or # to get the speed you need and then javascript for web, python used to be clunky? Then hire a coder to convert prototype to production. Also for creating huge systems you need this type of system.
Copilot is extremely cool. Been using it for a while now and it probably doubled my coding speed. Instead of asking stackoverflow how to do X in Y language over and over I just ask copilot.
you helped me kickstart my career 2 years ago, i used to watch your videos, now i landed contract at MNC! Thanks to you, still i see your videos as this one! :)
Just found your channel. Excellent content. It was great to see a review of Copilot being used in different ways by a real developer on time lines with projects!
I have been trying and failing to learn coding as a hobby for too long. Small python scripts and lua games is as good as i got. Yesterday my wife and i made a silly game to play using copilot and comments. We had to change a few little things but we had a "quick draw" game running in just a few minutes. As non-coders, we are impressed at least 😀
That might be it's main use case for people who can not program and simple projects, because for my use case it dies on it's arse, it's starts to add some nasty bugs, misunderstands the context, inefficiencies. Though if you really want to learn you need to stick to one project and keep going until it reaches a decent complexity there's a lot of trap doors in programming which you can only overcome by experiencing them, maybe the above game you did make with co-pilot keep going with it. But in short I think that's fine, it's not meant to be used to make anything remotely complex.
I wonder about a meta version of this that does not generate code, but a list of more refined prompts which in turn are used to generate the actual code. That could be a way to construct more complex software
after 5:00 mark I was testing copilot on my own programming language. it cogent really figure out too much but based on the context of the variable names and such it was able to guess the output of my code. so there's more going on there too
Completely agreed with whatever you said and I wish the technology to be keep getting better I got access to it a week ago and using it for my web development tasks and am literally surprised how copilot kinda speaks my mind whenever I hit a key to write code
Predicting the answer does have an immediate use case and that’s in auto generating proper unit testing, if you know the expected output that avenue is immediately available to automate.
i have had copilot for like 5 days now, the most amazing thing about it is its ability to understand my coding style and also coming up with new features fully implemented exactly the way i would have written the code myself
Damn, such beautiful tool, this'll take us places with more time on hand to draw analytical insights than wondering which syntax to use for data massaging!
I’m a little late to this copilot discussion but had a question for you. The fact that it can predict your output for a made up package leads me to think this language model could be used as a programming language itself. Do you see a future of computing where a language model can just do computing and skip the code middleman?
I wonder if you could get Copilot (or something like Copilot) just for your organisation's repo. Train it up on your own code styles, functions, SCSS, etc.
Would like to mention a use-case for copilot predicting a made up function's output: unit test suggestions for TDD. It's pretty awesome to think about.
Personally I've had a good experience in the week I've had access. On a Django project I've working on it auto completed a set of class views in the exact same style that I already implemented for another model. I was also really good at creating tests. But I have also seen a case where it needed clear instructions in order to implement a functionality, and so it seems to be really sensitive to the context.
Yeah, I've recently got access to it and it's pure magic. I don't know how else to put it, if not for risk of accidentally including copyrighted code I'd use it every single day.
What do you think about Co Pilot lowereing the barrier entrance for Tech Jobs, therefore raising the supply of Software Engineers avalible and si decreasing their salary?
As far as I can tell everyone on social media including these TH-camrs and redditers are downplaying this issue of job loss.. That's because as a TH-camr they cannot afford to be gloomy about programming lol their channel following will reduce.. But the writing is on the wall.. This AI tool will have a negative impact on number of job openings in the near future like maybe 1-2 years
You only have to look to what Sam Altman the CEO of OpenAI himself said regarding co-pilot. He tweeted that there's going to be a job loss due to AI in white collar jobs like programming very soon. Take it from the top man himself.. These TH-camrs are merely placating their subscribers so as to grow their following
@@millenialmusings8451 We'll see. It's not the first time somebody say something similar. I still think Software Engineer career has a bright future ahead. And even Machine Learning Enginner can improve as a career.
Now i know where do i know you from :) Watched your series NNFS and your channel name always sounded familiar... Now i know: You had this Project on Twitch where a NN learns to drive in GTA :P
I've been living under a rock and never bothered to look into copilot, as I hated the kite autocomplete utility that gets shipped with most ides. Six minutes in and I am picking up my dropped jaw from floor. Holy shit this is good. Whom should I kill to get my hands on this ??
Scribes did not think that the printing press would replace them. It would be useful in only specific ways, augmenting the scribes. AI will replace *most* programmers, eventually.
An AI that can replace the architectural aspect of programming would be a non-narrow AI. Copilot or anything like it won't replace jobs and isn't analogous to press and scribes IMO. A non narrow (one that could actually replace programmers) AI would be a paradigm shift in everything we know. I don't think anyone can see past that horizon or reference any historical event for an idea of how it will go. ...but we don't even know for sure that general AI is possible or when we'd achieve it. Til then its just philosophical games.
@@sentdex There is an assumption that you need to do everything a programmer can do in order to replace programmers, that's not true, IMHO. You have to make programmers better and faster in a way that will required less programmers to complete the same tasks. For example if you currently have 10 programmers in a corporation and co-pilot increases productivity by 11%, then one of those programmers is redundant for the current workload. The more productive you can make people, the more likely you can replace *some* of them. And that, IMO, is replacing programmers. Not all of them, but some. This is why I use the printing press analogy. This technology is in its infancy. I remember when people would have scoffed at the idea that AI could have completed the tasks that Copilot is completing. We are just one eureka moment away from all programmers being replaced. To give you some perspective, the game of Go is effectively "solved by AI", that task wasn't supposed to be attainable, but after Chess was solved, it was predicted to be 50 years away. It's only in hindsight that we recognized how a discovery can upset an industry. Cheers
One of the craziest things I've seen Copilot do was completing the pattern of case matching unicode box drawing characters and generating a small grid representation, based on a couple starting examples. It got a some wrong, but it also got a lot right. ChatGPT was worthless at this task.
Regarding the "ai taking our jobs" part. I'd also like to add that some people forget, or just don't know if they haven't worked as a programmer, that writing code is a small fraction of our job. The most difficult things are designing the architecture, debugging, refactoring, applying clients requirements and so on. If AI was that intelligent, we'd be already doomed anyways. This is nothing more than a fancy autocomplete.
Exactly. An AI that replaces all programming would be a general intelligence. IF/when that happens, the "loss of jobs" will be a tiny side effect to a general intelligence in the world, heh.
hi can you make a video on adam optimizer...i'm trying to make a simple neural net using adam optimizer and im stuggling with it ,i don't know how to apply it ....i'm beginner
I'm not sure if you will see this message but here goes: What language/s do you use the most and how long have you been programming? I see a lot of python videos so that's a given, but your thoughts on C,C++,Java, C#, JS etc etc sorry if it's in another video I'm a bit blind at times lol 👍
I almost solely use Python. 2nd most used language by me would be js, but only to pretty basic web dev UI extent. I've tried to learn C++ a few times, but find it a pointless exercise since nothing I do would benefit from it, so it just feels like I'm taking 100x the time to do something I could have already done in Python :D
@@sentdex Thank you for the reply that was quick. Yeah I'm coming from C/Assembly/Verilog etc at Uni and now use C#/C++ Unity/Unreal and prefer C#, I've used Java for android and dabble but not much in JS/Python for web and enjoy that too, python really is taking off last few years as is JS, I think going forward my main use will be C# as it's my favourite followed by JS and Python, I agree with you on the C++ front unless you don't need it I wouldn't bother. I would have been a Java dev if it wasn't for C#, very clean language and hard to fault. I will do some python projects in the future maybe try tensorflow as I see you have a video on it, I will view it later on. I might try a ML.Net project too and compare. I like your style of teaching very laid back and informative. I know you don't do C# but would like to see you do a C# video on ML.Net or Asp.Net or something else like a small Unity Neural Networks video, I might have a go at that too in the near future. Anyway good luck in your future programming and YT videos 👍
Btw copilot can complete code in the (Natural)language that you are coding in, for example i had some variables in Serbian and copilot created other variables on Serbian that extends on previous(i had a var named "levo" which means left and copilot autocompleted new varibles for down,right and up all in Serbian). I am a lil bot scared lmfao
Been using copilot for a while with python and flutter. Literally got no comments for how good it is. Its like I think of something and goddamnit knows what it was
I’m saying this from a very biased perspective as I enjoy more low level and direct programming: To me it seems like it will be used to further dumb down the art of programming, where with lots of abstractions and automations, many programmers don’t understand the inner workings of the behaviour they are implementing would rather see an end result as conveniently as they can.
@@kombangkoedias8062 also it is really not important in which cpu register the address to the string of the Title of your Javascript Website is. That is literally the point of abstraction. Yes. For some things you want to know which register your string is, but not for most things not written for a microcontroller and most certainly not when you're writing a website.
the avg. hourly wage of a SWE in the US is $34. assuming this tool would save you a single hour of time per month, purchasing this tool at $30/month would save the average FT developer money. MSFT and others have certainly noted this. here's to another monthly outgoing 🍻
A friend of mine managed to make the whole school project with copilot, it was written awfully, yet it worked (1st year on uni) it was "pretty" complex as well (basically a turing complete), it's impressive how well it does
The automaters could automate themselves out of a job soon, which sounds like a good argument for UBI. This could allow everyone's hobbies to be vastly improved, and who knows what cool ideas could come from that.
Are you going to be using this plenty going forward, like I'm always going to be doing? I think this **** rules!!! If I wanted to learn any new languages, I can more efficiently.. I heard some concerns that will devalue work or have developers cut or hasten deadlines 95% faster. Seems like plenty of good & bad. But are you going to be using this going forward? I'm enjoying it & making even more elaborate applications these days from it.
If someone, someday would try to fully automate programming.. it would make sense to automate directly to machine code and not the code that were made for humans to understand..
Then humans wouldnt be able to understand it to debug it properly. Also, I think copilot gets most of its context from comments in existing projects. Comments dont exist in machine code, so it wouldnt be able to analyze what specific code blocks are doing. Machine code isnt always better because concepts like reflection and dynamic code manipulation just wouldnt work. Now maybe you wouldnt need that if its all machine code, but it does add tangible power. Also, platform independence adds a lot of value too.
Although the idea of copilot has terrifying implications (from the potential to end jobs to the outstanding potential of GPT-3 itself), I think I can just be happy that at least it requires the programmer to write good comments!
To be fair, the industry has been moving away from comment blocks in code, at-least large amounts because the idea is that your code should be self documenting. Also about ending jobs, the only people who might have to work harder and look over their shoulders are mainly scripters, data entry and similar, because I found it didn't have much understanding of anything even remotely complex, also it doesn't do very well with low level languages it seems more proficient with Python, Ruby and similar languages, I.e very high level ones.
@@devilmanscott I am not sure what industry you are refering to. Programmers who are formed as programmers have gotten better but a lot of people who write code are not "proper" programmers and I can tell they do not give have no idea how to write legible and structure code. These are also the kind of people most likely to use something like Github copilot more naively. So yes, I stand by what I said. On the second point, that might be the case for the current model. Given enough data they improve and succeed. But you are right on who is more at danger with this tool: people who write boilerplate code, the usual scripters, etc. mainly because the one thing these model are still incapable of doing is developing new ideas.
@@MrAntraxico I wouldn't say programmers have gotten better considering efficiency is a big problem, most programs today like Discord or Chrome, both destroy memory and run quite slow. About people using Co-Pilot poorly, those people will be found out pretty quickly, so I don't really worry about them. I can not see Co-Pilot really improving that much more, it doesn't understand concepts or architecture, it will never help you on data structures/patterns, or to choose between rest, graphql or GRPC, how to do sucurity and so on and as I said, it does quite poorly with low-level langauges because there's more pitfalls like threads, generics, pointers and so on. Python fits really well because you really have to be quite bad for Python not to work, it was designed to be more like English than a traditional programming langauge. Also it basically shits the bed with larger projects which was my use-case. Remember GPT-3, that was supposed to kill off coders & programmers, I remember the noise people made when that was released pretty much the same noise as Co-pilot, but the industry just moved pretty much not changed. In short, if you work on Unreal, Chrome, Facebook and other complex projects, you are quite safe, but if the most you do is a some simple graphs and buttons with python or html, well there might be less jobs available in the future.
Yeah definitely, I've used it to learn some subtle new things as well as new libraries. One day, I could see it teaching new languages too, but right now I still need experience in Python to smooth out the edges.
To be honest, I am just not seeing it. The claim from copilot is 0.1% is regurgitated. Given how transformer models actually work, and from my own testing, I am inclined to believe this number and it would seem copyright is a non-issue here.
@@sentdex you have specific algorithms/code snippets that are under a specific license. You cant just simply generated them and everything is fine. Like code under GPL-2 license need to be continue under the same license
Again, copilot isn't regurgitating entire licensed codebases. The license isn't really coming into play here. It certainly would if you used some substantial portion of that code in your project, but, again, this is all philosophical. It's really not happening with copilot. I encourage you to share an example from copilot that you believe is infringing.
Imagine being new to programming and running in to 5:26 , that would be the least productive thing on earth. I've seen a few experienced programmers be impressed with copilot, but i think they forget whats its like to not have a firm grasp of programming in the general sense. Experienced programmers have a lot of guiding principals and thought processes that newbies dont, and i think copilot needs that to thrive.
This addon is really mind blowing. Especially when it autocompletes COMMENTS correctly for me. Feels a little scary that it can predict what I'm about to describe and how.
The silly notion that it would steal our jobs I compare to the fact that you could ask GPT-3 today to write a letter to your mother that you cant come to dinner and are sorry. It will produce the letter but you likely wouldnt send it before you have read what it suggested and approved. Similiarly an AI can produce code that you ask it to but you need someone who can actually read it (hence code themselves) in order to verify if its actually doing what it should. Surely some simple algorithms you can do your own input tests without knowing coding and check its correct, but programming general purpose programs is usually way more complex.
I agree with you about that, but if we refer to the amount of time that a company invests in developers, the time of syntax is not negligible at all and it will reduce the amount of employment, on the other hand there will be many more ideas for development
@@ppcuser100 , I use GitHub copilot every day now and I will have to agree with you. Certain tasks are now done way faster than before. And no doubt proof reading and adjusting an AI output is faster than doing it all from scratch. We will see how things turn out.
its awesome from technology standpoint, but a nightmare from economic standpoint. This will turn programmers into gig workers (if not already) the likes of uber drivers - programmers will complete work in a month what used to take a year and will be hunting for next gig after that - and they'll burn out in about a decade in the industry. I can see UBI staring at right around the corner... good or bad? we'll see.
really good for production, really bad for people who are just learning programming, copilot is so good that newbs will just tab for the code without learning
It's been a long time coming. This isn't something software engineers should be shocked about. They've had lots of preparation time for this. If they're not ready, that's capitalism baby.
Is everyone just going to ignore that weird dark shadow on his right shoulder? WTF is that? Some post processing artefact? Can't....unsee....it.......urngh!
Copilot helped me a lot with porting code from one language to another. You can leave the original code in a comment above where you're about to write and it manages to translate it really well. It's especially interesting to see that it can translate concepts from one language to another, even concepts that you've special made for your code base.
It bounces between writing better code than me and writing my front ends, to introducing nearly imperceptible bugs that suck up a lot of time
@@piyh3962 yes, you always have to audit the code generated. I see anything more than 2 lines of generated code as a template that you'd have to edit.
Yeah same. I was a bit skeptical about copilot until I was unknowingly using it & realized how well it.... Well... Copilots my code lol.
You are still using it?
I've watched a substantial amount of your videos in the last few months and I just want to say thank you.
Glad you like them!
Wow, that's mind blowing! Didn't know it was this powerful, especially the sentiment analysis example. It's kind of like using programming languages as an "on the go" interface to other transformer models and seeing what they can do. Great video!
I've been using copilot for months, blew every expectation out of the water. I would pay to keep this tool if I had to
Something tells me you will :P Just a guess though. This seems to me like quite the expensive thing to run for free.
Hey! Sentdex can you try to replicate copilot using your model and make a extension for vscode.
I considered doing it, but honestly it's about 1000x worse than copilot xD
@@sentdex 😂
@@sentdex Yeah there's no way any single person without access to a HPC would be able to model anything close to copilot
Wow, that example with a non-existing library was incredible. I have been using co-pilot for a while now and did not realize it had such amazing predictability.
I've been using copilot for a few months and I really like it. I had a few "wow" moments, but most of the time it just saves time doing simple things, finishing lines (or methods) and even works as a more advanced autocomplete, instead of writing a few letters and getting completions for each of them, it guess the whole package/method you want right away. I got so used to it that when I'm paired with a colleague it feels like there's something off with their IDE, like they are missing an important resource. It's similar to that feeling when you step down from an IDE to use plain text and misses autocomplete, auto import, shortcuts, etc.
Are you still using it?
I believe great developers who don't like syntax but are very good architects will flourish in this ecosystem.
I tried writing whole projects with co-pilot and it did not work but writing parts of an app like the popup window with tkinter worked exceptionally well. Also scrapping websites just by saying “scrap (website name)” worked super well. I did not even have to go and find the tag under which the content was located all i needed was to specify if it was title or body and it automatically picked the right one. It is truly amazing to see what it can do.
Thats even more insane.
I was a 100% sure that it could not possibly be that advanced.
@@kerb755 so was i
If you focus heavily on object-orientated coding and break things into modular sections it works scarily well, especially if you tell it what each function/class inputs and outputs.
Yeah, my project which is only two months old, but reaching about 50,000 lines, it was basically no help, as I said in few other comments it's great for small things, but those who are worrying about jobs, if you're just a scripter then yeah you might have more competition now, but I suspect there's enough work out there.
@@devilmanscott I agree, scripters and also DevOps will have a bit more competition
Yeah, been extremely unimpressed with copilot and now admittedly I have mainly attempted to use it with elixir and rust code. With elixir code, it suggests nothing but Ruby with small modifications to fit the elixir syntax. Most of the expressions will not even be valid. I know elixir is not an expressly supported language but you'd think a tool like this could see the context of the file extension.
For rust code the issues are much more subtle but still problematic. Since I work in the crypto space, much of what I do is very abstract, copilot has a ton of trouble parsing very abstract code. And in regards to statically typed languages it's almost like copilot makes up its own types like 20-30% of the time. This is especially true for macro derived types.
This tool isn't going to take anyone's job any time soon. And it has problems with licensing in the open source community. It's difficult to know if the code it's pulling in is something that it figured out on its own or something that it basically copied from another repo and worse still, it doesn't really respect licences so the code could get you into trouble.
I also would never use this tool when writing proprietary code since it's hard to know where your code is being broadcasted. If you use this to write a secure algorithm then you should be extremely careful.
I am also worried that this will further dumb down the next generation of coders. It's already the case that developers will learn frameworks and not the underlying concepts so that they can get a job really easily. The problem with this approach though is that these developers won't be able to do anything non-trivial outside of the scope of the framework (there are plenty of JS devs who don't know vanilla JS for example). Co-pilot feels a bit like that; If all you're doing to write code is writing comments, how do you gain a deeper understanding of the code that's being outputted.
And I think that's really where us developers are going to be with tools like this emerging. We will always be necessary in some capacity, because even if they create more complex versions of copilot which can build out entire apps with ease, a developer is still going to have to be there to make sure the code is consistent and workable. Also, software is not math or science; there are multiple ways to do something and proving that its complete and consistent is difficult (even with theorem provers its hard to prove a piece of software doesn't have state leaks and potential edge case bugs).
You make good points on laziness and growing ignorance of developers, too many relying on pre-built solutions while having no understanding or ability to innovate on top of it, it's well known that most code written today is very inefficient and the usual excuse is that we'll throw more computing power to counter it, which will get quite interesting when we reach the limits of current silicon miniaturisation which isn't that far off.
Also I agree it won't affect programmers, but I can see this affecting a lot of scripters, data entry/science peeps, bot makers and those that don't really venture deeper into programming where you have to contend with complex data structures/patterns, generics, APIs and so on.
For example my current project(about two months old) must be reaching about 50,000 lines of code and Code Pilot has a far lower limit than that and I suspect it wouldn't understand most of the context of what I am doing anyway.
In short those who aren't simple scripters should be fine, though I do fear the code that is produced might be riddled with bugs and inefficiencies. but beside that it's a cool tool, but will it fall to the wayside like GPT-3 kind of has people were saying the same about it but it's mostly not talked about much anymore.
The argument about laziness is a logical fallacy sifting blame from people to tools. As a start, if you can readily land jobs by learning just the framework and not the underlying concepts, then that is what the business needs, otherwise they would not be constantly hired and retained. Wrong or right, its not a problem with the tools, nor it can be an argument against the existence and further development of those. In my 22years of experience working as a developer and managing others, there have been 0 cases where someone stood out because they wrote everything by hand or memorized syntax. If anything these people would usually make an impression for the wrong reasons - trying to look down and demean colleagues who don't fit their "standards" while being the "work hard not smart" type themselves. Also the "dumbing down" has been going on for way longer, and the same arguments have been said for absolutely any developed tool or framework. It already proven invalid.
It's not trained full with rust.
I have been using coppella for a couple of months as well honestly it has blown me away as well. I think it cuts both ways, if you're an inexperienced programmer it may be a bad thing to lean on, but an experienced programmer may leverage it properly. Still pretty awesome
It feels almost..... magical.
Like It knows what I want to do.
It writes all the boilerplate for you and can even write full utility functions, you only need to hint the purpose of function in the name like vector2string()
And it even writes custom string literals for you. It know ansi escape sequences for different colours.
So even If you write :-
std::string operator ""_grn(const char*)
It will be automatically generated with correct code to create a string with green ansi color code
There are a few potential problems I see with copilot that weren't addressed here.
First and foremost, as you said in the video, it does make mistakes. This is to be expected, but these mistakes might be much harder to catch and debug, especially for inexperienced programmers who might be relying on copilot to write code for them.
Second is the legal aspect. Copilot was trained on Github code, and as far as I know, it will sometimes spit out an exact copy of some code it saw somewhere. This may have changed, but as of recently, Microsoft made it clear that it was the programmer's job to check for copyright infringement on code that copilot writes. For small projects, this is a difficult task. For large projects, or anything commercial, it makes copilot unusable.
Curious to hear your thoughts about this.
For mistakes in code:
I see this as no different than what you might copy and paste from SO or something. Most often the mistake still runs, but isn't what your intention was, so I think the pitfall is the same if you're willing to accept some block of code without really looking. Copilot just lets you do it faster.
Copyright:
To be honest, I am just not seeing it. The claim from copilot is 0.1% is regurgitated. Given how transformer models actually work, and from my own testing, I am inclined to believe this number and it would seem copyright is a non-issue here.
@@sentdex Hopefully you are right. Please keep doing copilot videos in the future, this video was super informative and I'd love to see more.
# create unit tests here
Copy writing code is a bit fuzzy to me. There is the obvious case of blatant copy paste of large sections or entire programs without permission. But for small snippets, there are only so many possibilities. For example you want a recursive method to return fibonacci for your input. It's essentially a few lines of code, and there are only so many ways of writing it that makes sense in your respective language. Or a sorting algorithm. It's not copying, its implementing the correct solution for the problem. Should that solution be owned by someone and everyone has to write it different? I think it's similar to copy-writing a melody, there are only so many notes you put in so many combinations, like the experiment Damien Riehl did with generating all the melody permutation on the harddrive so they have legal copyright status. Fast-forward to a dystopian world where every permutation of code has been made so everything is copywrite? Idk, seems short-sighted and obtuse. Not concerned tho, I don't see it happening. Just interesting to think about. I suppose we can reframe the question as "what is the threshold of code similarity in which infringement has occurred?".
ja so its good for prototyping. like python is not great for production for many thing will have to convert to c++ or # to get the speed you need and then javascript for web, python used to be clunky? Then hire a coder to convert prototype to production.
Also for creating huge systems you need this type of system.
Copilot is extremely cool. Been using it for a while now and it probably doubled my coding speed. Instead of asking stackoverflow how to do X in Y language over and over I just ask copilot.
Yeah agreed!
Sucks you have to pay for it now tho :/
@@Fallout3131 yea, I made the comment before that happened and stopped using after they went paid unfortunately since its so expensive in my country ☹
you helped me kickstart my career 2 years ago, i used to watch your videos, now i landed contract at MNC! Thanks to you, still i see your videos as this one! :)
Just found your channel. Excellent content. It was great to see a review of Copilot being used in different ways by a real developer on time lines with projects!
I have been trying and failing to learn coding as a hobby for too long. Small python scripts and lua games is as good as i got. Yesterday my wife and i made a silly game to play using copilot and comments. We had to change a few little things but we had a "quick draw" game running in just a few minutes. As non-coders, we are impressed at least 😀
That might be it's main use case for people who can not program and simple projects, because for my use case it dies on it's arse, it's starts to add some nasty bugs, misunderstands the context, inefficiencies.
Though if you really want to learn you need to stick to one project and keep going until it reaches a decent complexity there's a lot of trap doors in programming which you can only overcome by experiencing them, maybe the above game you did make with co-pilot keep going with it.
But in short I think that's fine, it's not meant to be used to make anything remotely complex.
I wonder about a meta version of this that does not generate code, but a list of more refined prompts which in turn are used to generate the actual code. That could be a way to construct more complex software
I've been thinking about this exact thing 😁
after 5:00 mark I was testing copilot on my own programming language. it cogent really figure out too much but based on the context of the variable names and such it was able to guess the output of my code. so there's more going on there too
Completely agreed with whatever you said and I wish the technology to be keep getting better
I got access to it a week ago and using it for my web development tasks and am literally surprised how copilot kinda speaks my mind whenever I hit a key to write code
Predicting the answer does have an immediate use case and that’s in auto generating proper unit testing, if you know the expected output that avenue is immediately available to automate.
I totally agree, i'm using it from almost 3 weeks and is still an amazing feeling when it know exactly where i'm going with my code!
i have had copilot for like 5 days now, the most amazing thing about it is its ability to understand my coding style and also coming up with new features fully implemented exactly the way i would have written the code myself
Are you still using it?
@@openroomxyz yes
Damn, such beautiful tool, this'll take us places with more time on hand to draw analytical insights than wondering which syntax to use for data massaging!
I’m a little late to this copilot discussion but had a question for you. The fact that it can predict your output for a made up package leads me to think this language model could be used as a programming language itself. Do you see a future of computing where a language model can just do computing and skip the code middleman?
I wonder if you could get Copilot (or something like Copilot) just for your organisation's repo. Train it up on your own code styles, functions, SCSS, etc.
More weird experiments with copilot pls! :)
when is the next episode of nnfs coming out? its honestly my favorite series on yt rn and id love if u finished the series :)
What I like about Copilot is that it encourages writing comments and documenting your code, in a way.
Would like to mention a use-case for copilot predicting a made up function's output: unit test suggestions for TDD.
It's pretty awesome to think about.
Personally I've had a good experience in the week I've had access. On a Django project I've working on it auto completed a set of class views in the exact same style that I already implemented for another model. I was also really good at creating tests. But I have also seen a case where it needed clear instructions in order to implement a functionality, and so it seems to be really sensitive to the context.
I follow you and Bitcoin last 6 years. Incredible growth indeed.:D
Yeah, I've recently got access to it and it's pure magic. I don't know how else to put it, if not for risk of accidentally including copyrighted code I'd use it every single day.
Where did they get information about Codex having larger context size than GPT-3? They both have context size equal 4000 tokens as far as I know.
What do you think about Co Pilot lowereing the barrier entrance for Tech Jobs, therefore raising the supply of Software Engineers avalible and si decreasing their salary?
As far as I can tell everyone on social media including these TH-camrs and redditers are downplaying this issue of job loss.. That's because as a TH-camr they cannot afford to be gloomy about programming lol their channel following will reduce.. But the writing is on the wall.. This AI tool will have a negative impact on number of job openings in the near future like maybe 1-2 years
You only have to look to what Sam Altman the CEO of OpenAI himself said regarding co-pilot. He tweeted that there's going to be a job loss due to AI in white collar jobs like programming very soon. Take it from the top man himself.. These TH-camrs are merely placating their subscribers so as to grow their following
@@millenialmusings8451 We'll see. It's not the first time somebody say something similar.
I still think Software Engineer career has a bright future ahead.
And even Machine Learning Enginner can improve as a career.
Thanks for another great video.
Thanks for watching!
Agree, it is like a more advanced intellisense, instead of suggesting parameters, refractoring, … etc, you have a AI that suggests code.
Now i know where do i know you from :) Watched your series NNFS and your channel name always sounded familiar... Now i know: You had this Project on Twitch where a NN learns to drive in GTA :P
I've been living under a rock and never bothered to look into copilot, as I hated the kite autocomplete utility that gets shipped with most ides. Six minutes in and I am picking up my dropped jaw from floor. Holy shit this is good. Whom should I kill to get my hands on this ??
Right now it's in beta and you need to request access: github.com/features/copilot/signup
hol up
4:23 isn't it possible that copilot is just using live code evaluation to tell you the result of your code?
Scribes did not think that the printing press would replace them. It would be useful in only specific ways, augmenting the scribes. AI will replace *most* programmers, eventually.
An AI that can replace the architectural aspect of programming would be a non-narrow AI. Copilot or anything like it won't replace jobs and isn't analogous to press and scribes IMO. A non narrow (one that could actually replace programmers) AI would be a paradigm shift in everything we know. I don't think anyone can see past that horizon or reference any historical event for an idea of how it will go.
...but we don't even know for sure that general AI is possible or when we'd achieve it. Til then its just philosophical games.
@@sentdex There is an assumption that you need to do everything a programmer can do in order to replace programmers, that's not true, IMHO. You have to make programmers better and faster in a way that will required less programmers to complete the same tasks.
For example if you currently have 10 programmers in a corporation and co-pilot increases productivity by 11%, then one of those programmers is redundant for the current workload. The more productive you can make people, the more likely you can replace *some* of them. And that, IMO, is replacing programmers. Not all of them, but some.
This is why I use the printing press analogy. This technology is in its infancy. I remember when people would have scoffed at the idea that AI could have completed the tasks that Copilot is completing. We are just one eureka moment away from all programmers being replaced.
To give you some perspective, the game of Go is effectively "solved by AI", that task wasn't supposed to be attainable, but after Chess was solved, it was predicted to be 50 years away. It's only in hindsight that we recognized how a discovery can upset an industry.
Cheers
One of the craziest things I've seen Copilot do was completing the pattern of case matching unicode box drawing characters and generating a small grid representation, based on a couple starting examples. It got a some wrong, but it also got a lot right. ChatGPT was worthless at this task.
So does it have an emacs mode? And does it do any language that's not python?
When will co-pilot be available for the general public?
Just a quick question...are you using a windows manager with linux?
No, using Ubuntu as my OS
Regarding the "ai taking our jobs" part. I'd also like to add that some people forget, or just don't know if they haven't worked as a programmer, that writing code is a small fraction of our job. The most difficult things are designing the architecture, debugging, refactoring, applying clients requirements and so on. If AI was that intelligent, we'd be already doomed anyways. This is nothing more than a fancy autocomplete.
Exactly. An AI that replaces all programming would be a general intelligence. IF/when that happens, the "loss of jobs" will be a tiny side effect to a general intelligence in the world, heh.
hi can you make a video on adam optimizer...i'm trying to make a simple neural net using adam optimizer and im stuggling with it ,i don't know how to apply it ....i'm beginner
Very interesting. Thanks. The programming methods may be changed soon. Time will show.
I got access last week and I'm obsessed with using it on all PRs
Are we (your community) your co-pilot?
OR am I the community's copilot?
Charles is the Plane
If AI kills programming, then there will be a new job position. The AI wrangler!
I don't know about you, but that sounds pretty badass.
I'm not sure if you will see this message but here goes: What language/s do you use the most and how long have you been programming? I see a lot of python videos so that's a given, but your thoughts on C,C++,Java, C#, JS etc etc sorry if it's in another video I'm a bit blind at times lol 👍
I almost solely use Python. 2nd most used language by me would be js, but only to pretty basic web dev UI extent. I've tried to learn C++ a few times, but find it a pointless exercise since nothing I do would benefit from it, so it just feels like I'm taking 100x the time to do something I could have already done in Python :D
@@sentdex Thank you for the reply that was quick. Yeah I'm coming from C/Assembly/Verilog etc at Uni and now use C#/C++ Unity/Unreal and prefer C#, I've used Java for android and dabble but not much in JS/Python for web and enjoy that too, python really is taking off last few years as is JS, I think going forward my main use will be C# as it's my favourite followed by JS and Python, I agree with you on the C++ front unless you don't need it I wouldn't bother. I would have been a Java dev if it wasn't for C#, very clean language and hard to fault. I will do some python projects in the future maybe try tensorflow as I see you have a video on it, I will view it later on. I might try a ML.Net project too and compare. I like your style of teaching very laid back and informative. I know you don't do C# but would like to see you do a C# video on ML.Net or Asp.Net or something else like a small Unity Neural Networks video, I might have a go at that too in the near future. Anyway good luck in your future programming and YT videos 👍
Maybe our grand kids will finally live in a non-boring cyberpunk world, unlike us. Those lucky bastards.
Btw copilot can complete code in the (Natural)language that you are coding in, for example i had some variables in Serbian and copilot created other variables on Serbian that extends on previous(i had a var named "levo" which means left and copilot autocompleted new varibles for down,right and up all in Serbian). I am a lil bot scared lmfao
Totally agree with you
this is really cool actually nice
Next: make a drone pilot for your drones.
Been using copilot for a while with python and flutter. Literally got no comments for how good it is. Its like I think of something and goddamnit knows what it was
can it be integrated with pycharm?
Super good vid dex! So helpful may family loves watching you! Keep it up! I would love to talk to you about opencv someday! Maybe intern 🤥
why i still need to wait in the waitlist?
I’m saying this from a very biased perspective as I enjoy more low level and direct programming: To me it seems like it will be used to further dumb down the art of programming, where with lots of abstractions and automations, many programmers don’t understand the inner workings of the behaviour they are implementing would rather see an end result as conveniently as they can.
well to be fair, it's all business. And business wants results. So, copilot offers exactly that.
@@kombangkoedias8062 also it is really not important in which cpu register the address to the string of the Title of your Javascript Website is. That is literally the point of abstraction.
Yes. For some things you want to know which register your string is, but not for most things not written for a microcontroller and most certainly not when you're writing a website.
the avg. hourly wage of a SWE in the US is $34. assuming this tool would save you a single hour of time per month, purchasing this tool at $30/month would save the average FT developer money. MSFT and others have certainly noted this. here's to another monthly outgoing 🍻
learning new things is getting easier with co-pilot.
how using copilot?
A friend of mine managed to make the whole school project with copilot, it was written awfully, yet it worked (1st year on uni) it was "pretty" complex as well (basically a turing complete), it's impressive how well it does
with copilot you can finally code like in the movies
The automaters could automate themselves out of a job soon, which sounds like a good argument for UBI. This could allow everyone's hobbies to be vastly improved, and who knows what cool ideas could come from that.
it seems like a really powerful translator to convert our wacky english into machine code.
Are you going to be using this plenty going forward, like I'm always going to be doing? I think this **** rules!!! If I wanted to learn any new languages, I can more efficiently.. I heard some concerns that will devalue work or have developers cut or hasten deadlines 95% faster. Seems like plenty of good & bad. But are you going to be using this going forward? I'm enjoying it & making even more elaborate applications these days from it.
Damn... It's answering interview questions.
If someone, someday would try to fully automate programming.. it would make sense to automate directly to machine code and not the code that were made for humans to understand..
Then humans wouldnt be able to understand it to debug it properly. Also, I think copilot gets most of its context from comments in existing projects. Comments dont exist in machine code, so it wouldnt be able to analyze what specific code blocks are doing. Machine code isnt always better because concepts like reflection and dynamic code manipulation just wouldnt work. Now maybe you wouldnt need that if its all machine code, but it does add tangible power. Also, platform independence adds a lot of value too.
Although the idea of copilot has terrifying implications (from the potential to end jobs to the outstanding potential of GPT-3 itself), I think I can just be happy that at least it requires the programmer to write good comments!
To be fair, the industry has been moving away from comment blocks in code, at-least large amounts because the idea is that your code should be self documenting.
Also about ending jobs, the only people who might have to work harder and look over their shoulders are mainly scripters, data entry and similar, because I found it didn't have much understanding of anything even remotely complex, also it doesn't do very well with low level languages it seems more proficient with Python, Ruby and similar languages, I.e very high level ones.
@@devilmanscott I am not sure what industry you are refering to. Programmers who are formed as programmers have gotten better but a lot of people who write code are not "proper" programmers and I can tell they do not give have no idea how to write legible and structure code. These are also the kind of people most likely to use something like Github copilot more naively. So yes, I stand by what I said.
On the second point, that might be the case for the current model. Given enough data they improve and succeed. But you are right on who is more at danger with this tool: people who write boilerplate code, the usual scripters, etc. mainly because the one thing these model are still incapable of doing is developing new ideas.
@@MrAntraxico I wouldn't say programmers have gotten better considering efficiency is a big problem, most programs today like Discord or Chrome, both destroy memory and run quite slow.
About people using Co-Pilot poorly, those people will be found out pretty quickly, so I don't really worry about them.
I can not see Co-Pilot really improving that much more, it doesn't understand concepts or architecture, it will never help you on data structures/patterns, or to choose between rest, graphql or GRPC, how to do sucurity and so on and as I said, it does quite poorly with low-level langauges because there's more pitfalls like threads, generics, pointers and so on.
Python fits really well because you really have to be quite bad for Python not to work, it was designed to be more like English than a traditional programming langauge.
Also it basically shits the bed with larger projects which was my use-case.
Remember GPT-3, that was supposed to kill off coders & programmers, I remember the noise people made when that was released pretty much the same noise as Co-pilot, but the industry just moved pretty much not changed.
In short, if you work on Unreal, Chrome, Facebook and other complex projects, you are quite safe, but if the most you do is a some simple graphs and buttons with python or html, well there might be less jobs available in the future.
I've just started with computer graphics, and there is a lot of boilerplate..
If you use this and your manager has no idea what Copilot is, better not finish tasks too quickly if they're into inventing shit to make you busy ;)
I do like the regex function copilot did
Writing this comment before watching the video but it seems this tool can be a good educational tool as well as a time-saver
Yeah definitely, I've used it to learn some subtle new things as well as new libraries. One day, I could see it teaching new languages too, but right now I still need experience in Python to smooth out the edges.
copy right concerns as well
To be honest, I am just not seeing it. The claim from copilot is 0.1% is regurgitated. Given how transformer models actually work, and from my own testing, I am inclined to believe this number and it would seem copyright is a non-issue here.
@@sentdex you have specific algorithms/code snippets that are under a specific license. You cant just simply generated them and everything is fine. Like code under GPL-2 license need to be continue under the same license
@@sonOfLiberty100 Not sure claiming copyright on snippets of code would hold up in any court.
@@notead If the code is unique and is under a specific license than yes. Thats why I wrote, an algorithm under a license
Again, copilot isn't regurgitating entire licensed codebases. The license isn't really coming into play here.
It certainly would if you used some substantial portion of that code in your project, but, again, this is all philosophical. It's really not happening with copilot. I encourage you to share an example from copilot that you believe is infringing.
Imagine being new to programming and running in to 5:26 , that would be the least productive thing on earth.
I've seen a few experienced programmers be impressed with copilot, but i think they forget whats its like to not have a firm grasp of programming in the general sense. Experienced programmers have a lot of guiding principals and thought processes that newbies dont, and i think copilot needs that to thrive.
I love copilot ❤
This addon is really mind blowing. Especially when it autocompletes COMMENTS correctly for me. Feels a little scary that it can predict what I'm about to describe and how.
The silly notion that it would steal our jobs I compare to the fact that you could ask GPT-3 today to write a letter to your mother that you cant come to dinner and are sorry. It will produce the letter but you likely wouldnt send it before you have read what it suggested and approved. Similiarly an AI can produce code that you ask it to but you need someone who can actually read it (hence code themselves) in order to verify if its actually doing what it should. Surely some simple algorithms you can do your own input tests without knowing coding and check its correct, but programming general purpose programs is usually way more complex.
I agree with you about that, but if we refer to the amount of time that a company invests in developers, the time of syntax is not negligible at all and it will reduce the amount of employment, on the other hand there will be many more ideas for development
@@ppcuser100 , I use GitHub copilot every day now and I will have to agree with you. Certain tasks are now done way faster than before. And no doubt proof reading and adjusting an AI output is faster than doing it all from scratch. We will see how things turn out.
its awesome from technology standpoint, but a nightmare from economic standpoint.
This will turn programmers into gig workers (if not already) the likes of uber drivers - programmers will complete work in a month what used to take a year and will be hunting for next gig after that - and they'll burn out in about a decade in the industry. I can see UBI staring at right around the corner... good or bad? we'll see.
will you use it?
I have used it since getting access. Still enjoying the heck out of it so far.
At least it has been a coding producitivty boost for me ;)
really good for production, really bad for people who are just learning programming, copilot is so good that newbs will just tab for the code without learning
Its crazy to think it can know the output without running the code.
Are you still using it?
Yep, every day.
@@sentdex I am considering trying it out, if it's an usefull and better than usual autocomplete interesting to see how it works on a bigger projects.
Sometimes, the code will be suggested in an infinite loop, so, it would just suggest the same code again and again.
yeah I have found that if you have a lot of variables, eventually it just gets stuck suggesting variables with lengthening names..etc.
3:42 Comment (regex) completer
# Create next facebook
output : Meta
I wonder how sentiment analyzes passive aggressive statements like "She is as sharp as a rock"
Now I need to practice writing comments
It's been a long time coming. This isn't something software engineers should be shocked about. They've had lots of preparation time for this. If they're not ready, that's capitalism baby.
example_sentence = "Should I use copilot?"
#output should be: 'positive'
Is everyone just going to ignore that weird dark shadow on his right shoulder? WTF is that? Some post processing artefact? Can't....unsee....it.......urngh!
It's the shadow of the microphone
Nice
they took er jobs!
just sittin there and reading comments as they appear, ay harrison? nice analysis btw
i think that if you're a beginner in software developmnent, junior to below it, you should'nt use it...
Hello. Good. Have a nice day!
I use copilot to do my world history vocab.