This is so bad it's dangerous (Copilot CLI review)
ฝัง
- เผยแพร่เมื่อ 9 เม.ย. 2024
- The Github CLI is questionable quality wise. The Copilot extension though...woof. I did not expect to hate this so much. Wow. Ugh. Never again.
I am trying Warp now...will report back.
SOURCES
/ 1768026786456096978
github.com/github/gh-copilot
• NOW AVAILABLE: GitHub ...
Check out my Twitch, Twitter, Discord more at t3.gg
S/O Ph4se0n3 for the awesome edit 🙏 - วิทยาศาสตร์และเทคโนโลยี
![ไม่ต้องมีที่ที่ให้ฉันอยู่ แต่ขอแค่มีฉันอยู่ก็พอ (Spaceless) - getsunova [OFFICIAL MV]](http://i.ytimg.com/vi/4N8idbxZ-7M/mqdefault.jpg)
Theo so enraged by this tool he forgot to ask it why people aren't subscribed yet.
That was a good one! lol 🤣
Considering how awful AI seems to be with Powershell, the idea of Co-Pilot in the CLI seems like a terrible idea.
gotta love theo complaining about the gh cli then immediately fighting with ffmpeg
ofcourse ffmpeg is complicated because it does complicated things, and the gh cli is complicated out of spite, so minor diffrences
As a counter-point, if I'm unfamiliar with ffmpeg, my options are to Google search the answer or use the AI Microsoft wants me to pay for. Thus, showing how well it works with something I'd be likely to asking for help with is a good, "ultimate" use case. (Yes, I agree start with something simpler to see if it can do that, but at the end of the day, if I'm paying money for a service that claims to help me, I want to know it can help me on the questions I'm most likely to have - such as a very complicated video tool!)
@@trumpetbob15 oh i wasnt saying theo made a bad choice using ffmpeg as an example, just pointing out the diffrences
ffmpeg is arguably the most complex and advanced media management tool, it literally supports every input, intermediate, and output video, audio, and subtitle format on the planet. Using every type of hardware on every type of architecture on the planet. Oh and if you spend a week reading through the wiki you can pretty much learn it.
The Github CLI is just a command line so you don't have to waste time working with the also inconvenient github website.
If it can only do the very simple stuff, then what's the point? Why do we accept paying for a product that is half way there?
This reminds me of an anecdote I read years ago about the perils of DWIM - Do What I Mean.
Some person some years ago developed a command line shell that could fix some of his common mistakes and boost his productivity with aliases and predefined actions. It worked well, until another person wanted to try it on his own machine.
This other person had a good habit of preserving old files by appending a "$" sign to the end of the filenames, and purging them at the end of the day using the equivalent of rm -r **/*\$. When he tried to use the DWIM shell, he was told to just "delete *$" and it would remove the files matching the pattern recursively. And so it did. Until one day, when he didn't make any backup files, but ran "delete *$" out of habit. The shell didn't find any files matching the pattern, so it assumed that it was a typo, "fixed" the pattern by removing the "$" character, and ran the command again, now as "delete *".
should have installed the "undo" command
Let's get that installed on nuclear control terminals.
I haven't finished reading yet. Just wanted to put in on the record that having "a good habit" and "purging them at the end of the day using the equivalent of rm -r **/*\$" on the same line, that's reeeaaaaly sus.
done now. 🌟 😀
"gh copilot explain" is the rich man's man
Would have been as fast to google the question
That's an idea for a video. Have two people try to do something complex in ffmpeg. One uses AI, the other uses StackOverflow. See who smashes their computer first.
@@alancarroll7648 Trick question. The computer is smashed by the time the contestants realize they have to use ffmpeg
We said we wanted a faster horse. Instead we got CLI googling, but worse.
I can't agree with you complaining about showing your IP. It's a command line... there's no such thing as 'click to see'. It's also not really something you should be streaming without already knowing the commands anyway.
So, -h doesn’t do this already?
Well you have to know the command in this case, it's more akin to apropos
No, -h is actually useful
we already have `man`, `info`, `tldr`, `cht.sh`, the `-h` flag, and the `help` builtin
Manpages are so 2023, Today we use Google and Stackoverflow.
man
6:19 I wonder how many ppl, other than streamers, would be bothered by that
Definitely an edge case that doesn't justify that outburst.
If you're sharing your screen you're dead essentially
also with this stuff i feel like hiding it would make a lot of users just ignore it? idk
What about demos for conference talks, anyone doing a public webinar, etc…. There are TONS of people with development skills or responsibility who are not 100% developers. Technical marketing, sales engineers, etc.
@@AdamPippert would your ip matter at a conference talk or any of these events?
A video about a bad CLI that segues into another difficult CLI.
Glad to see I'm not missing out on anything by not spending money on Copilot
you're missing out on a great opportunity to give Microsoft your data while using a horrible product!
AI for commands is terrible. I wrote an earlier version of Github Copilot CLI (I called it dunno) which provided similar functionality in the CLI. It ended wiping my drive with an rsync command.
😂
I've done that without AI help before 😂
Is it the tool or the user?
Maybe read the command before execution?
@@Leeway4434 It wasn't obvious. It's not an rf -rf or anything like that. It was a seemingly normal command with undocumented side effects. (What it does is copy files WITHOUT deleting anything. However, it actually overwrites folders for a brief moment and somehow crashed the system before the folder was restored, making it so that they weren't restored)
"ghcs" when i have multiple haskells
"ghce" glasglow haskell but energy efficient
Only thing in brain when i hear that
I mean you did pick a non-trivial audio source, the 7.1.4 channel layout is going to blow up almost any target container and codec without a bunch of hand holding. It's just the nature of cinema codecs that they are a pain to deal with no matter what, besides even professionals can't really agree on the right downmixing approach so AI would never do it correctly.
I was about to say… 12 channels
I never used ffmpeg, but how the hell does it make any difference how many channels the file has? 7.1.4 is a perfectly standard type of audio that I have in many of my files.
@@JanVerny ooh it matters a lot 7.1.4 is not standard at all, normally that only shows up with specific codecs, most of which are proprietary and ffmpeg always has a bit of trouble encoding those. The only real standards are stereo and mono.
@EraYaN First of all, yes it's standard, but ok let's say I grant you that somehow for ffmpeg it's very hard to handle audio with more than 2 channels (rofl).
What I don't get is why would that matter in such a simple case as one one Theo has? How is it fundamentally different to join 2ch tracks together over joining 800ch tracks together? I don't see it
My brain@3:04
Theo: "I'm gonna throw them straight into hardmode and see how they do with FFMPEG"
Copilot to Bill Gates: "Daddy! The man in the mustache is being mean to me! 😭😭😭"
The level of rage feels like I’m watching an old Game Grumps clip. Honestly here for it
This video was everything but what I expected it to be. But I learned new things about ffmpeg, so I'm not complaining lol
I think one of the most wearing aspects of AI text-to-speech is that they never pause leaving you taking a long stream of words without a break so any complex thought or idea that a person might pause over for you to properly absorb is just trampled over so you're now processing one thing while trying to also take in what comes after which in itself can be very wearing and is contrary to the point of giving any kind of instruction very much like the way this sentence is overly long with no punctuation of breaks of any kind
I don't believe this is an AI voice. From what I can tell, it sounds like they processed the voice recording through Adobe Podcast, which ruins your recordings and makes you sound robotic. OpenAI did it too with their GPT-4 launch teaser last year, for whatever reason. It bugs me a lot, especially with the fact that it makes your lip movements feel disconnected from your voice.
Edit: looks like you can't delimit code by grave either 19:38
Really liked the video. Watching you actually go through, troubleshoot, and talk through it is both entertaining and helpful.
Edit 2: Maybe they'll add in triple grave to interpret code blocks with line breaks?
Theo: Build me a rocket.
AI: 🚀
Theo: This sucks!
I would think a good test for github copilot would be git and github based questions.
Here docs/here strings should help with the long string arguments, right? A bit at least
yep, theo seems to not know much about what a shell vs terminal is, let alone knowing the syntax of the bash shell, he attenpted to delimit an argument to a command with backticks `` thinking it would be passed literally
Totally off topic: To concatenate 2 MP4 videos of the same resolution and all other parameters (i.e. recorded both with my phone) I use: MP4Box -add 1.mp4 -cat 2.mp4 output.mp4
I guess ffmpeg -c copy does the same?
Out of curiosity, how is an IP address sensitive information? I thought it could only be used to find rough (city-scale) geographic location...
Anyone who's intrigued by this is probably someone very unfamiliar with the CLI - and maybe that's the point, but that also makes it dangerous. Better to use stuff like man or --help (or just google) and actually learn what the commands are doing and what the flags and options are. This tool doesn't feel far off from copying and pasting shell commands you find online - and to anyone reading this, never do that unless you understand what the command is doing. But also just never do it.
Spot on.
Do you have a video explaining how to make my terminal as fancy as yours? Like the bottom side?
That's tmux with custom config
It’s sad to see LLMs being used so in effectively. I have dyslexia and ASD, ADHD that clearly have caused cognitive problems. I didn’t pursue my dream of being a programmer 25 years ago, but it’s now within the realm of possibility. Its been about two years since I started using GPT. It has allowed me to recover some of the damage that has been done, strengthening my ability to learn and comprehend. I have such a poor “context” window it frightens me. I “error/hallucinate ” far more than my transformed self. My dad suffers from Dementia and is held in a home because he cannot look after himself. My goal is to build my own personal system so I can prolong my independence, preserve my cognitive abilities, and hopefully protect others from such a fate as well. The outcome is uncertain, but success would allow me to offer a valuable gift to anyone who could benefit from such a system.
Loving the classic Theo baseball look
State-of-the-art, multi-millon dollar A.I. vs '
Who would win?
4:40 yes!! AI has really often been a solution to fix bad ux in general.
Like code generation for exemple, it's mostly good when you're using a tool you don't quite know yet, because it's better than most docs.
And ux is hard to do properly (and nobody learnt that in school, let's face it).
And yes proper docs are part of ux, they're one of many solution to answer the question: how can new user get going with the tool i'm building.
Good error messages would be another way to answer the same question. And typically in graphic interfaces, having just a few visible named menus / buttons is another good way to hint at existing features, but docs do help the same ux goal
Honestly having access to an LLM in the terminal can be nice, instead of having to open a tab to Gemini. But, err, maybe one with less form-like questions.
@11:29 - I'd guess the code block is labeled Arduino because the language is auto-detected by the syntax highlighting package, and there wasn't much to go off of with that block so it just got it wrong.
Not saying it is very well designed but you can get around using quotes if you put the command after a double dash (gh copilot explain -- command -flags)
I imagine that still wouldn't be safe for something like that fork bomb which would be evaluated by the shell before the shell starts the gh app.
@@MSheepdog That is true. My suggestion only bypasses the inconvenience of having to add quotes, maybe escaping them inside the pasted command etc. but as always you'll still have to make sure what you paste is actually safe (which to some extend might defeat the point of a tool that explains to you what the command does)
the moment I saw the icon on my desktop i went into gpedit and turned it off
Literally the first thing I asked it to do (create a .txt listing all the ".ts" files inside a folder) it entered in an infinite loop and started to occupy all my storage, it went down 6Gb before I shut down everything.
It suggested a very long command that I just trusted would work for some reason, my fault ig
the funny thing is that normal copilot in chat has better cli answers than the copilot cli
Why dont they just create a tmux extension?
I felt like her presentation was really off putting, hard to describe why but it sounded like she was just listing things as quick as possible.
I think another thing was the way it was edited with her sentences each being like a cut with little pause.
How frustrated can you be, chill man
Thanks for advertising copilot, buying it now
is this also a thing on win10
I wanna see how Copilot explains the XZ commands.
Now you have to do warp terminal, I guess.
I would honestly hate working with you haha
wild
It's kinda hilarious that youre absolutely raging at gh cli asking questions becasue somehow getting information from you sequentially is bad ux but using ffpg with 0 mention of how absolutely horrendous it is, top quality commentary right there
Yeah I couldn’t watch this one past the UX raging - it sounds like Theo has some real strong and, to me, more than a little odd opinions on CLI UX, no shame for having strong opinions but hard disagree.
The part where he says “why do I have to have all these CLI options? What happens if I don’t?” is followed by the CLI helpfully guiding him through the important decisions. It’s met with “why was that so many steps?!?” 🙃
Don't worry, I also use my downloads folder for everything
As I have half a dozen gpt4 tabs open anyway, I'll just use that.
Plugging the terminal into remote is a step too far for me. Really not a fan of tools like warp. Also generally terminal operations should be fairly simple. And where they're not you typically open your bashscript in an IDE with an lsp etc.
It got me into a mess with a bunch of git file deletes and also got me out of the mess when i felt like crying. So.... let's call it even. 50/100
Infecting. That's a good way of describing it.
Human speech has a lot of room for error because there's so much redundant information. Approximation from an LLM in that context doesn't hurt that much. Source code is far less forgiving, but usually you have a compiler in between generation and execution to catch the most obvious mistakes. Command line instructions is almost the worst thing I could think of getting an LLM to do. There's no room for error, and the side effects are immediate (accidental rm -rf / anyone?).
The only application I think could be worse is using an LLM as a compiler. Just going straight from prompt to binary blob. No way to inspect what it did, no checks between.
opens google and searches the command:
people say AI hallucinates but its actually just Bipolar
What is this cool powerline kind of status bar in your terminal ?
She does sound AI generated, and assuming that she isn't the reason it sounds like it is that she has cut the sound from multiple recordings. The very short pauses and change in voice intonation makes it sound artificial.
Its not even just LLMs, human language is fundamentally ambiguous/approximate, there's a reason we developed different languages for unambiguous instructions (in maths and programming), this is why I find the whole program your computer by having a conversation with it fundamentally stupid and a recipe for disaster. It good for semantic search, but that's about it I think.
Yeah exactly.
We already have a way of telling our computers what to do. With very explicit commands and code. No room for for the computer to "learn wrong" or "hallucinate".
Plus didn't ChatGPT have that poisoning attack earlier that was causing it to sputter a bunch of memes and nonsense? Wouldn't this CLI LLM have that same vulnerability? Except instead of getting something funny, you get something that could brick your install.
Seems like 'man' with extra steps
As a newbie, it’s godsend for me
Ive always said this about Warp: logging into your shell with an email address feels wrong
I'm not going to install copilot so I can't test this, but it's possible you could fix the input with a double dash. The double dash is a convention (not rule, not automatic) for programs to stop accepting flags for arguments to the right. Example:
echo test > -file
cat -file
# cat: invalid option -- 'f'
# Try 'cat --help' for more information.
cat -- -file
# test
21:07 it will not be executed
I don't use AI or Copilot in windows or Terminal. So I don't care what they are doing with copilot in windows or in terminal.
I have a feeling that copilot stuff is GPT-3 at best
i wish Microsoft would actually put the real gpt4 in their copilot prodcuts, theyre still using some old Microsoft finetune of gpt4 or something
personally been using shellgpt with gpt4 and chatgpt, never got issues
I'm so glad to see I'm not the only lazy "downloads" folder user 😀
I tried to play blindfold chess against GPT-3 and it made all kinds of illegal moves. I played a game against copilot and it won, legitimately (to be fair, it was probably searching on Wikipedia for known continuations, while GPT-3 doesn't have access to the Internet, try asking it what happened in Israel on Simchat Torah and it will give a brief paragraph summarizing the meaning behind the holiday, without mentioning the bloodshed of last year's massacre).
Honestly, the open beta with installing an npm package was way easier than whatever the hell this is.
Microsoft embraced other os a while back and with this tool they know developers use macOS so it makes sense for the demo.
Good product but not great yet, I think their would be more improvement in the future
Did you know I cut a whole documentary in ffmpeg?
Proud of you. But that also sounds miserable. There's a reason why gui/wysiwyg editors are standard.
I prefer my own custom zsh completions…
Just add descriptions to the them if you need explanations…
AI sucks at anything slightly more complex than "hello world"
4:38 - If that is an AI generated voice and it said 'axe' instead of 'ask' .... hmmmmmm
6:10 - The Google Login Dropdowns piss me off. I used to stream on a random platform, and I would load a site, only to have it doxx my IRL name in the corner... I hate that.
I don't think it's a TTS or AI voice, from what I can tell, they processed the original recording through Adobe Podcast, which will completely ruin your recordings and makes you sound robotic.
I truly fear for the poor soul that asks copilot to generate any sort of `rm`-related command
why would you unlist a video just because people disagreed or misunderstood it
Why does she say gHKOPILot
No one is going to talk about the fact that they are using MacOS?
edit: talking about the github cli video
I have fucked up enough command line things that id nevvvver use this, even if people saying it’s perfect
This looks like a project forced by a SDE2 to get the promo to SDE3 lol
Open Interpreter is actually a practical project for controlling your computer.
Microsoft presents : man pages, 200x slower and with random mistakes !
Need commands to create a new context and to clear the current one.
Copilot in the terminal sounds even more idiotic than copilot embedded in the operating system. lol
copilot, write a hello world react app better than what theo can do.
BSSSZZZTTTT !
If you just gonna trim a video with ffmpeg copy the codec instead of re-encoding it which take time and losing quality.
I just don't understand why you might use gh copilot instead of open interpreter ¯\_(ツ)_/¯
Warp already has ai in the terminal and its free
Next time ask the AI to explain arcane Perl scripts
The copilot cli is aparrently unsuable
can i cite this when people panic about ai, because its clear ai can barley make a correct command (granted ffmpeg is a pain in the ass)
trust but verify!
I think the ncurses magicians would have made this good UX actually.
why did you show the microsoft copilot logo when you're talking about the github copilot?
is this better than the warp terminal AI? i was always very sceptic of their advertising of it
or you can use Warp...
sorry this cli is trash, haha thks for the fork bomb didn't know that ill be more carefull now when looking for help
Cannot take seriously people using macOs and homebrew 🤷🏼♂️🤣
This is the same language model. It won't magically understand more CLI tools than before. So if it provided mediocre results before, it will be the same now. Just will be faster because you can call it from the terminal. Ultra mehhh