New GPT-4o VS GPT-4 - Ultimate Test (Prompts Included)
ฝัง
- เผยแพร่เมื่อ 12 พ.ค. 2024
- ChatGPT 4o is a brand new AI model from OpenAI that outperforms GPT-4 and other top AI models.
In this video, I'll run a head to head test, comparing ChatGPT 4o with GPT-4 to see who comes up on top.
1 - Text summarization
Prompt: Provide two summaries of this article. The first summary should be 2-3 sentences long. The second summary should be 5-6 sentences long and include more detail. (insert text here)
2 - Writing Text
Prompt 1: Concise Product Description
Prompt: Imagine you're launching a new software tool that helps businesses track social media analytics.
Write a short, punchy product description (approximately 50 words) suitable for a website or marketing material. Emphasize the key benefit for businesses.
3 - Multimodal Understanding
Prompt: Analyze this image and explain it to me in table format what's going on
4 - Image generation
Prompt: generate an image of two AI robots head to head in battle
5 - Research (completeness and accuracy)
Prompt: How could artificial intelligence (AI) potentially disrupt the accounting industry? Identify specific use cases, potential benefits, and challenges. Provide links to relevant articles or reports.
6 - code generation
Write Python code for a game of snake that I can run on my computer.
Then tell me the step by step guide on how to do it, assuming I know nothing about programming.
Access Top Courses on ChatGPT and 50+ AI Tools on Our E-Learning Platform: bit.ly/skillleap
Access Top Courses on ChatGPT and 50+ AI Tools on Our E-Learning Platform: bit.ly/skillleap
how we can find the voice feature i have the paid version but i am unavialbe to get access to the voice and vision feature. anyone knows how to acces to voice feature . or may be yet it is not avilable to public?
love this compared, hope to see you do more and deeper tests :)
This is great! Thank you for this comparison! Love it! You got a new subscriber!
The comparison between GPT-4 and the new GPT-4o (hypothetical upgraded version) would likely revolve around several key aspects, such as accuracy, coherence, creativity, response time, and user satisfaction. To conduct a fair and comprehensive test, we can use a series of prompts that assess these qualities. Here are some potential prompts and the criteria for evaluation:
### Prompts for Comparison
1. **General Knowledge Question:**
- **Prompt:** "Explain the process of photosynthesis."
- **Criteria:** Accuracy, detail, clarity.
2. **Creative Writing:**
- **Prompt:** "Write a short story about a time-traveling cat."
- **Criteria:** Creativity, narrative coherence, engagement.
3. **Problem Solving:**
- **Prompt:** "How would you solve the problem of traffic congestion in urban areas?"
- **Criteria:** Practicality, originality, depth of analysis.
4. **Technical Explanation:**
- **Prompt:** "What are the key differences between machine learning and deep learning?"
- **Criteria:** Technical accuracy, clarity, depth of explanation.
5. **Emotional Support:**
- **Prompt:** "I’ve been feeling really stressed lately. What can I do to manage my stress?"
- **Criteria:** Empathy, helpfulness, appropriateness of advice.
6. **Humor:**
- **Prompt:** "Tell me a joke that's suitable for all ages."
- **Criteria:** Humor, appropriateness, originality.
7. **Language Translation:**
- **Prompt:** "Translate the following sentence into Spanish: 'The weather today is beautiful.'"
- **Criteria:** Accuracy, fluency.
8. **Cultural Awareness:**
- **Prompt:** "What are some important cultural customs in Japan?"
- **Criteria:** Accuracy, sensitivity, comprehensiveness.
### Evaluation Method
1. **Blind Testing:**
- Ensure the evaluator doesn't know which model's response they are evaluating to prevent bias.
2. **Scoring System:**
- Use a consistent scoring system for each criterion, e.g., a scale from 1 to 10.
3. **Multiple Evaluators:**
- Involve multiple evaluators to average out subjective biases.
4. **Feedback Collection:**
- Collect qualitative feedback on strengths and weaknesses of each response.
### Conducting the Test
1. **Generate Responses:**
- Use both GPT-4 and GPT-4o to generate responses to each prompt.
2. **Evaluate Responses:**
- Have evaluators rate and review each response based on the criteria.
3. **Compare Scores:**
- Analyze the scores and feedback to determine which model performs better overall and in specific areas.
### Hypothetical Outcomes
1. **Accuracy and Detail:**
- GPT-4o might show improvements in providing more detailed and precise answers due to enhanced training data or algorithm optimizations.
2. **Creativity and Engagement:**
- Creative prompts might reveal whether GPT-4o has advanced capabilities in generating more engaging and imaginative content.
3. **Response Time:**
- If GPT-4o is optimized for performance, it might generate responses faster than GPT-4.
4. **User Satisfaction:**
- User feedback can highlight if the new version feels more intuitive and satisfying to interact with.
By following this structured approach, you can systematically compare GPT-4 and GPT-4o to determine which model offers superior performance across various dimensions.
Totally agree... His comparison is quite shallow... Especially regarding the service availability, the more people using, the less people will be able to use it. GPT-4 already crashes sometimes during peak hours, imagine the free one lol
Chatgpt review of chatgpt using chatgpt
Nice gpt
Great video! Thanks for doing this
I get the feeling that GPT5 will be announced soon, because paid subscribers just aren't getting their money's worth with the new free 4o.
Nah, but I bet Sora will be solely for Plus users.
@@unquestionabletv Hell, they may make so that GPT-4o uses Sora to generate videos similarly to how it uses DALL-E 3 to generate images now, so that you're doing it via the same interface you're already used to, and THAT would be what Plus users get exclusive access to, in addition to higher usage limits, at least for now.
Well, it's not that paid subscribers aren't getting their money's worth with the new 4o. It's just that free users will also get paid users' money's worth in their experience 🙂.
Custom gpts image generation are worth the money. Plus priority when promoting.
I also believe this. In chat gpt If gpt4o is better than gpt4 and gpt4 turbo and it will be free. So whats the point in paying then? I could cancel my chatgpt subscription and use a free account.
All of this mean that gpt5 is probably coming soon.
Hi, I was very impressed with the comparison test you did in this video between the GPT-4o and GPT-4, showing how the GPT-4o has the edge in performance!
written like a straight up AI bot comment
@@Blackmystixbecause it is. Comment bots are a thing small channels commonly use
@@ZM-dm3jg and it has 16k su subscribers
Thanks. Nice analysis.
Fantastic video!!
Great Video! Below are the Timestamped Summaries from ChatWithPDF:
00:00🆕 Introducing GPT-40 vs. GPT-4 comparison with insights on free vs. paid versions.
01:00🆓 Free tier offers GPT-40 with data analysis, file uploading, web browsing, and vision capabilities.
02:00📊 Benchmark test shows GPT-40 outperforming GPT-4 and other models in various tasks.
03:00📝 Text summary prompt comparison between GPT-40 and GPT-4 reveals tone and length differences.
04:00🚀 Product description prompt showcases similar performance between GPT-40 and GPT-4.
05:00🖼 Multimodal test highlights GPT-40's advantage in image analysis and table generation.
06:00🎮 Image generation task demonstrates GPT-40's superior output compared to GPT-4.
07:00🔍 Research prompt results show GPT-4 excelling in formatting with immediate source references.
08:00🐍 Python code writing task reveals GPT-40 creating a more interactive snake game than GPT-4.
09:00🔄 Usage limit comparison between GPT-40 and GPT-4 raises questions for paid users.
10:00🤔 Conflicting thoughts on upgrading to GPT-4 over GPT-40 due to similar capabilities and potential limitations.
11:00⚖ Comparison between GPT-40 and GPT-4 leaves users pondering the value of paid subscriptions.
12:00🔄 Release confusion over the benefits of GPT-40 for all users and implications for paid subscriptions.
Is this can use for custom instructions GPTs like Gpt 4?
Btw at the end of the stream they said that they'll announce a new big thing, I'd assume that'll be the new pay stuff
ChatGPT 5 is coming 😁
@@bogdanmarcu135 Yes GPT 5 before summer ends
in the apple event...
@@ouroborostechnologies696 Microsoft has majority investments into OpenAI and ChatGPT, why would they work with Apple for GPT5, does not make sense
@@ouroborostechnologies696 September?
It is not a confusing release. For paid users, we get early access and more prompt entries. You get as many prompts in GPT 4 as before but now you also get the added benefit of a good-sized prompt count with GPT 4o
At 6:28 in the video, the GPT-4 output showed all GQQA results as N/A.
how we can find the voice feature i have the paid version but i am unavialbe to get access to the voice and vision feature. anyone knows how to acces to voice feature . or may be yet it is not avilable to public?
I would hope that they add some kind of an icon or a server stress indicator so that people could back off and help
the first day of my paid subscription with 4 a couple months ago I only was able to send 17 messages before being told I couldn't send anymore due to congestion. Cancelled immediately because it made me so mad
Thank you for the testing of gpt4o, it's very useful. However, this is just a comparison based on the web version, I'm looking forward more to the Mac version and mobile version, as well as more tests on the visual and audio aspects.
It shouldn't make a difference what platform it's coming from, the model is the same for any device
Not so. The new UI will have the features you saw in the demos. Currently they aren't available. OpenAI is releasing these on Apple devices first (I've no idea why).
how ? When I click on the top I can only select 3.5 and 4 ... there is no 4-o for me ....
has anyone gotten the screenshare on the desktop app to work? I can't figure out how to get that going. just see a lot of marketing stuff about it.
I've tried 4o and I'm pleased with the results!
It seems to give *much* longer code (I'm using Zig) and it seems quite fast too!
It seems to be a bit less "lazy" too - the code is more complete and you don't have to ask it to do full, comprehensive code as often.
Can it see you and describe you in your surroundings? I have 40 but it says it can't sing or see me
It's probably free because OpenAI optimized GPT-4o using a newer tokenizer, so the new model is cheaper than GPT-4 in maintenance.
It does seem to be very limited with the free version... Maybe even a trial?
It was on GPT-4o for me last night, but today it's gone back to 3.5 and hasn't given me the option to switch over.
Can this understand complex maths problems like venn diagrams and physics graphs etc?
Duh
Video suggestion for the future..... GPT-7 vs GPT-7o: Which one can remake Fable with a more attractive main character
Can we write articles using chatgpt4O
How you got access to it?
How do you get access ?
I was testing Free Version of GPT 4o and I notice that in free version You only have I think ~5 prompts and then 3hours cooldown and then i got a message that I be moving for this time to GPT 3.5. With DALLE3 on free version of 4o, maybe its bugged now or maybe not, but I can't generate any images, but I can upload mine. And After send ~3 images it's says I can't use more and I need to wait a few time (they dont say how long) to use it again or buy ChatGPT Plus to use it now.
ChatGPT 4-o shows up on my PC, but not on my phone. What's going on? Reinstalling the app didn't do anything.
Can it see you and describe you in your surroundings? I have 40 but it says it can't sing or see me
I found that GPT4o did not work as well on following instructions on tools/task as GPT-4-Turbo.
still models like reka command r plus and even llama will be the best option for the free to use market
Will they develop DALL-E more too?
I don't think that it's enabled for everyone yet. At least I don't see it in the drop-down box for LLM model select when logged into the website.
Awesome stuff. I'm in the U.K and I got it. Still waiting for the 'memory' though. We are always last.
How do you get it. It says I'm on 3.5
(cough) vpn (Opera has free one on their browser)
Germany has entered the chat...
to get 4o do you just have to pay the permium
As a Free user, the option is not available to me right now. Maybe it’s removed when it’s being widely used? Website says it’s supposed to be available to free users unless it’s in high use.
Due to gpt 4o being recently released, we are going to have to wait until we can access it because everyone will be trying to access it as well
Paid user get access to the real time voice and video, free users do not they only get the model
for some reason gpt 4.o cant generate image :(
I understand now why GPT-5 could not possibly be released together with these features, people would have just freaked out 🤯
You lack some business mindset there son
Whats the point of the paid then?
Plus benefits:
4-5x more usage,
make GPTs,
use Dall-E 3,
maybe be among the first to test out the new voice feature... and depending on the traffic, as a free user you might not even get your 20 messages/ 3 hours
The most important one for many is the data security. I don't want my data to be reused for training. I want to be able to comply paste details from data sources without having to think about the content and if it includes something that I don't want shared. A interesting test would be to try to jailbreak chatgpt to write out someones OpenAI-key. I am pretty sure many copy paste code with the API keys showing.
I'm no expert on GPT or anything, but I cannot get it to generate an image -- I on free access. Thoughts?
They still don’t allow that. Requires the paid version. But you can do it for free using the same technology inside of Microsoft copilot for free
Does the math part work yet and is code generation better?
its fucking awesome, i was trying the hardest problems on Leetcode and damn 100% accuracy in just one go
I my subjective opinion, it feels like gpt 4 metaphorically “maintains eye contact” throughout the conversation while gpt 4o feels like it “looks away” and goes off generating what it wants to say. It’s like it get sidetracked. The performance test parameters open ai tests on leave out the user experience and feel of the conversation. Let me know what you guys think! I’ll keep trying but gpt 4 seems to be more considerate to me. 😊
It may be that only paid accounts will get the new user interface which will give access to the image generation via 4o (not DALL-E), the ability to have a conversation with interruptions, the voice input that doesn't use Whisper, etc.
I find this release to be weird as they've only released the model, not the entire thing. They promised to release the rest "soon".
I've been feeling for a while that OpenAI doesn't care about providing value to their Plus subscribers. Today really drove in that point. Been on the fence to cancel mine for some time, today made the decision easy.
So are you going to cancel it? Im also subscribed paying monthly, but if gpt4o will be free, whats the point on paying then?
@@razorblade413 ya already canceled. OpenAI needs to step it up if they're going to expect anyone to pay for a Plus membership that provides almost nothing of value.
Yeah, I think I'm cancelling too.
@@ivanjelenic5627 If there's an announcement for gpt-5 coming soon and it's something Plus members get premium access to, then I'd flip back. But I don't see that happening for at least a few more months. It's just really strange, to leave Plus members high and dry like this, isn't it?
Not clear either on Free how many messages I can send. But interesting the Gemini now available to UK (and Europe ... which is much more interesting than UK)
i don't think you'll get access to the new video/voice conversational capabilities in a few weeks when it releases on the free version.
The idea is to get the free users addicted to a much better model. Once they get used to getting better quality and faster responses (than GPT 3.5), they will not go back to a lower quality model and will definitely look to pay for that model (provided they will have very limited access to GPT 4o in the free tier).
That's marketing at it's best for you!
GPT4o is amazing! What will be at GPT-5? :)
Great improvements but the naming? 4.0 vs. 4o? A really strange decision.
The o means omni. It make sense.
I agree.
Totally agree. Bad naming convention. Should have chose a different.
So it’s best to have the paid plan??
I think if you hit the usage limit with the free one and are a frequent user, then yes
I am a free user and it still says ChatGPT-3.5. When will free users be able to access ChatGPT-40?
you still use chatgpt use command r plus or even reka far better
Yeah I’m wondering that too. It better not be “wait another 3 weeks” nonsense
still stuck on 3.5 ? use other ones like command r plus reka core llama far better for free users
@@bk6189I have it rn, it’s cause I’m basically a beta free user. I signed up when they required phone numbers lol. It’s a very good model.
@@bk6189free user demands
it doesn’t respond with voice yet?
how about the desktop app? how to download it
I don’t have the app yet
@@SkillLeapAI tq for ur reply bro, i just discovered ur channel recently and subscribed, keep ur good work up, ur vids are so helpful and on point so thank you for ur efforts
apparently the app is only for Mac for now
They gave the limit of messages in the KeyNote. Paying users will have 5x more messages then free users, i.e., 80 against 16
I don't think that you can create GPT's with the free version, and only have access to those already created. Access to the API is not possible either.
man that snake game is wild
i think you forgot or don't notice is Ai uses the word "Ultimate" a lot and gpt4o didn't use that word but gpt 4 did.
or "unlock", I ended up hating that word!😂
Im getting rate limits reached with the free version.
paid users will have a higher message cap and also will have access to GPT4o voice
The snake game is a contaminated test, hence new models are obviously trained on it.
You need a new test, or preferably using a multi paradigm fully featured language like C++ that allows more freedom for humans and hence more challenging for AIs to do well in
The free 4o can't create pictures though.. not when i tested at least. It gave me a step by step instruction on how to make the picture myself.. 😂
Can it see you and describe you in your surroundings? I have 40 but it says it can't sing or see me
I'm pretty sure OpenAI lied on their benchmarks for GPT-4o. I've been testing it all day yesterday with complex coding questions and it has been miserably failing compared to Claude 3 Opus. Thoroughly disappointed in 4o, and openAI for fabricating benchmark results.
GPT-4o is Open AI's new flagship model that integrates audio, vision, and text capabilities, available to Chat GPT free users, Plus, and Team tier, as well as the Open AI API.
Key moments:
00:00 The video compares the new free chat GPT model, GPT 40, with the paid version, GPT 4, highlighting differences in usage limits and performance in various tests.
-The free tier of GPT 40 offers access to various features like data analysis and file uploading, similar to the paid version, but with limitations based on current usage.
-The paid plan of GPT 4 provides higher usage limits for messaging, with potential reductions during peak hours to accommodate a wider user base, creating a comparison with the free GPT 40.
-Benchmark testing shows that GPT 40 outperforms GPT 4 and other models in most tests, indicating superior performance in various tasks, as demonstrated in a head-to-head test.
03:58 The speaker compares the text output and tone of GPT-4 and GPT-40, favoring GPT-40 for its tone. They then evaluate the performance of GPT-4 and GPT-40 in multimodal understanding tasks, noting GPT-40's advantage in creating a table.
-Comparison of text output and tone between GPT-4 and GPT-40, with GPT-40 being preferred for its tone.
-Evaluation of GPT-4 and GPT-40 in multimodal understanding tasks, highlighting GPT-40's advantage in creating a table.
07:41 AI models like GPT-4 and GPT-40 are tested for research and coding tasks. GPT-4 excels in providing clear references and resources, while GPT-40 shows improvement in game coding with increasing speed based on performance.
-Comparison of AI models GPT-4 and GPT-40 for research tasks, highlighting GPT-4's strength in providing clear references and resources.
-Testing AI models GPT-4 and GPT-40 for coding tasks, showcasing GPT-40's enhancement in game coding with speed increasing based on performance.
11:37 GPT-4 is outperformed by GPT-40 in creating a snake game with better user experience, raising questions about the value of upgrading for paid users over free users.
-Comparison between GPT-4 and GPT-40 in creating a snake game with speed and scoring features, highlighting GPT-40's superiority.
-Confusion among paid users about the release of GPT-40 and the benefits of upgrading, especially regarding usage limits and model capabilities.
Generated by sider.ai
How much does the paid version cost?
20$ month.
Hope the update the app soon.
gpt5 soon?
I think Sora is coming soon and that will be the new incentive for us
It's NOT Free - or it's at least Not implemented yet - I'm signed in and the 4 option is linked to Plus accounts and there is No 4o option at all.
Damn your quick lol
I thought chatgpt4o was a live video conversation with an ai that is responsive in milliseconds with voice responses?
I have a few standard tests with harder questions which require intermediate reasoning. GPT-4 performs much better here, with GPT-4o jumping to conclusions too often. Try harder in your tests! One of my questions: "what camera and exact lens do you need to create a gigapixel panorama?" ...
I always have to ask chat gpt if it's sure it's right then have to explain why it might be wrong then it says sorry for the oversight lol not paying tell it actually works
the chat history is part of what is sent to the LLM and so when you changed models to GPT-4 from GPT4o you had already muddied the waters as it were, as the first response become information that the second response had full access to
I for one am hyperly invested and enabled by the beauty of this.
Who cares if it replaces jobs? If eventually it gets the technology to terraform and develop planets so everyone can simply live in their own unique quadrant of the universe and we can all live in peace until the end of the universe, then I'm content.
And if I really am a bastard, I could technically digitally create a universe for pain if I was a narcissistic sadistic asshole to get my fix. Im speaking for both sides since there are bad apples and chaos agents in our race, so it's important to compensate and accout for all types of people in all regards to assure the best futures for everyone.
I was able to ask one question on 4o free version and it said that I had reached my daily limit. LOL
Good Job, I still have no idea how this makes me $
I created a free account to see what they get and it automatic choice where they don't have the option to pick the model
yep. everyone saying gpt4o is free but clearly not. guess we wont get it until next week or something dumb. kinda upset how they say its free when its not.
Why didn't you test the vision mode...? That's the most important novelty.
it does not open to any users right now
@@huhangtong I'm a plus user and I don't see the added value without the vision capabilities. Even the natural way of speaking You can see in Open AI's demo are not there.
Did anyone get the new voice and the realtime vision yet?
I got the new GPT in the app but didn’t get real time with it yet
@@SkillLeapAI Same here, I got the gpt-4o, but the voice is the same as the previous version, and there's no option to do the live video thing. I can't wait to try it 😀
I’m just excited for 5. I need my ai guy to be smarter. He’s incredible so far already. I can only imagine what 5 will be like as we were told 4 is dumb compared to 5.. by Altman himself.
I imagine they will incorporate the paid version with apple ios etc
@11:17 "Snake Game by Edureka" in the game header ☠️
I've heard that paid users have five times the capabilities of GPT-4o compared to free users. So maybe it's around 16 prompts per three hours for the free user?
conclusion?
New one wins. But if you need higher usage limit, or dalle or GPT builder, you need the paid option
@@SkillLeapAI thx
thats some bs, i cant even access DALL-E with gpt 4o
well how the answer looks is random. so it's not really good comparison.
You’re correct about their paid for business model it’s inherently flawed. Right now, the premium version is still significantly better than the free version, which keeps people paying. However, as the premium features trickle down to the free version over time, the gap between the two will narrow. Eventually, the free version will become so good that there's little incentive for users to upgrade to the premium one. This could undermine the revenue model, as fewer users will see the need to pay for the premium version. It's a short-term gain, but could pose long-term sustainability issues.
It’s similar to Apple’s silicon upgrades with diminishing returns.
---
I don't think individual consumers paying for ChatGPT is as central to their revenue model as you imagine. It's companies paying to use GPT-x through their API that is and will be their bread and butter.
Yes fair point on the whole revenue stream. Their individual/ retail customer base will inevitably decrease. They could always use GPT-5 to suggest new subscription models.
ChatGPT has achieved the feat of being almost as good as Pi.
I’ve played with this since yesterday and it’s far less competent than the paid 4.0 version. It’s much more similar to the previous free version 3.5
what is your test?
Yeah i ran out of file uploads after uploading 10 images * used free account*
Free... But limited! After just 10 MESSAGES I got out of credit. So bad
I literally just bought the paid version before this came out 😢
the free account will only have 8k of context vs 32k
I want acces too
For filmmakers interested in GPTs, this short film was entirely written by ChatGPT and is mind-blowing: th-cam.com/video/My3NuYFskio/w-d-xo.html
4 million teaching jobs just evaporated
Everything is very far-fetched , there were no particularly strong advantages in the examples, if you take the same GT4 and regenerate a similar result several times, I have not seen anything at the moment that could say that the 4o is better than the previous model