The AI doctor will see you now: ChatGPT dominates medical exam
ฝัง
- เผยแพร่เมื่อ 14 ต.ค. 2024
- Former FDA Commissioner Dr. Scott Gottlieb joins 'Squawk Box' to discuss which AI large language models (LLM) scored the highest in the same medical exam doctors have to answer to get a medical license, future of AI-powered healthcare, For access to live and exclusive video from CNBC subscribe to CNBC PRO: cnb.cx/42d859g
» Subscribe to CNBC TV: cnb.cx/Subscri...
» Subscribe to CNBC: cnb.cx/Subscri...
Turn to CNBC TV for the latest stock market news and analysis. From market futures to live price updates CNBC is the leader in business news worldwide.
Connect with CNBC News Online
Get the latest news: www.cnbc.com/
Follow CNBC on LinkedIn: cnb.cx/LinkedI...
Follow CNBC News on Instagram: cnb.cx/Instagr...
Follow CNBC News on Facebook: cnb.cx/LikeCNBC
Follow CNBC on Threads: cnb.cx/threads
Follow CNBC News on X: cnb.cx/FollowCNBC
Follow CNBC on WhatsApp: cnb.cx/WhatsAp...
www.cnbc.com/s...
#CNBC
#CNBCTV
![[UNCUT]“ฅนตื่นธรรม”ปะทะ“หมอปลา-ปู่มหามุนี-อั๋นโอกิ”สะเทือนกระทรวงเวทมนตร์ Iคนดังนั่งเคลียร์I10 ตค.67](http://i.ytimg.com/vi/peMD_SAkwNc/mqdefault.jpg)
I uploaded my doctor's report to ChatGPT and ChatGPT explained everything I didn't understand in the report very well.
Written by hand. Nobody can understand doctor's writting
@@TechWizardBoctulus it couldn’t understand the handwriting, also I couldn’t, it was unreadable
@@ClaudiaS193 Then how did it explain everything?
@@ramsyrama I uploaded the report to chatGPT, then I asked ChatGPT to explain what all the medical stuff means, what was written in the medical report. The report was a diagnoses and it wasn’t handwritten. It was explained via a dialogue. Just try it with ChatGPT by yourself.
@@ClaudiaS193 That's lovely to hear buddy
If we use AI to diagnose we can improve health care tremendously. Combine that with routine lab tests analyzed by AI we could catch a lot of diseases earlier.
On the way... Already in the works
Google "Fake ChatGPT Results on US Medical Licensing Exam reported on Television by CNBC"
The results are f a k e, but YT deletes any mention of this. You have to google it.
I can't even say it, it gets auto-deleted.
This is the future I want to see. Will probably be crazy expensive though (greed)
I agree with this, but it has to be balanced with human touch and compassion. Which is very rare in the industry right now. It also has the potential to be used as a way to short staff hospitals even more.
This is actually pretty mind blowing and we're not even scratching the surface of AI yet.
Chinese AI was doing this in 2017. Calm down!
People don’t understand the jobs that are going away in the next three to five years, including mine
AI cannot replace us indian as we are the most intelligance race in the world
@@rajeshgajwelly9035 you’re too late, it already has. Doesn’t matter where or who you are
@@tommy516 not true, us indian go to canada or europe to get job and free money from govarnmant, we are never short of work. we are smarter, work harder, and accept less pay than other race. we indians are mighty and superior.
Just to be clear, the USMLE is an exam taken during medical school. This is not the performance exam your doctor takes in order to obtain certification to practice. People's symptoms don't fit like test questions (like any exam, poor representation of real life). This is no substitute for what your providers go through with training in residency and fellowship. I do think AI will help optimize and augment the diagnostic testing space, not the tool for definitive diagnosis.
Smartphones detecting skin diseases will be game changing worldwide. 🙏♥️
USMLE is a 3 part exam. The first 2 are taken during med. school and the 3rd and final part is usually taken near the end of the first post doctoral year of training. Many state licensing authorities accept passing results in lieu of their own state licensing exams.
My scores were high and I was licensed in several large states and 2 small states without ever taking the state exams.
Waiting for a study comparing AI diagnosis vs doctors to see who is more accurate.
there already have been tons of the these types papers published.
Already done for radiology, I believe.
a Computer does the diagnosis TODAY based on the input... the DOCTOR simply present the diagnosis to the patient.
@@hardheadjarhead definitely
@@DolphinAnalyticsAI Not a bad thing though...
We tolerate significant error rates from humans in fields like driving and medicine, yet we demand perfection from AI. This double standard is slowing the adoption of technology that could save over a million lives each year.
so off
Expecting significant error rates from AI is clownery. It is designed to have non-existent error rates.
I don't think that is quite true in medicine. I think that the majority of us are eager to use AI, esp. in assembling more complete differential diagnoses in complex medical cases.
I believe that in the future, there will be smart bathrooms. After using the bathroom, you will receive an analysis of your health, letting you know if there’s a problem or even advising you on which foods to eat to address any deficiencies
Was exploring how to use AI to detect Alzheimer's Disease. The idea was to give an AI model text and ask it if it belonged to a person with AD. When I was doing the literature survey and testing on language models, I found some interesting results. These commercially available language models are aware that they are not supposed to be diagnosing diseases therefore take a very conservative and diplomatic stand. However, if you give these models additional prompts to produce an answer then they tend to accurately detect the disease. I found accuracies exceeding 90% on popular standardized test sets. Then there are fine tuning approaches that further improve the accuracies. Sure there are some grey areas with language models but it's amazing how a language model is able to detect AD which otherwise needs expensive imaging tools like PET scan, invasive procedures like extracting cerebrospinal fluid, trained medical professionals etc.
What do you call a medical student that scored 60% on their medical licensing test, or the avg medical student that scored 75%? Doctor. What do you call an AI model that scored 98%? Too error prone to trust.
Genuine unintelligence can’t compete against Artificial intelligence.
@@brianbarnes746 100% facts right here. And a subscription to chatgpt cost $20 a month, its also the most expensive advanced language model.
A doctor on average makes 350k a year, some of your $ is going to directly to that. Besides salary every single aspect of American healthcare is inflated expenses wise, they overcharge because no one will stop them
It’s amazing listening to this woman, the lack of understanding that many people have about LLMs is shocking. You don’t have to explain anything. You tell it the same thing you’d tell a human, that’s all. The level of understanding LLMs have currently is insane. Won’t go into exact details but it’s insane. You could take the opening paragraph to HG wells war of the worlds and swap out the Martians with Ai and you’d have a perfect description of where the general public is at with an understanding of where AI is currently.
Now we no longer have to fear getting an inexperienced doctor or a doctor that’s having a bad day.
Yes you do, your missing the personal connection. Doctors can intuitively connect issues together by seeing you and having a conversation that otherwise would not be asked or listed on an AI survey. Too many doctors are already relying on it and not using the skills they learned.
Man, that's a really foolish and ignorant take. You know the real world is way more complicated and has far more variables than any exam... right?
No, I think we’ll still need doctors to assist patients who believe takes like this lol
?? How? Chat GPT isn't a person.
Experience is a very bad teacher.
Good old chap gpt... 😂
LOL
Didn’t compare it to the stronger version of Gemini
@@SigFigNewton which one? the one that uses reddit as one its sources, Or the one that shoves DEI in its responses?
@@TopShot501st ultra. The one that tests about the same as gpt4
The AI dotor will see you now
I was 60 years old when I got a medical examination through my American company,
The report highlighted a high PSA of 6.4 which I ignored, because I did not know what PSA was.
When I was 76 years old after many wonderful years in retirement, I went to my doctor because of many visits to the toilet during the night.....My PSA was found to be 11.4 and after an MRI and Biopsy, a tumour was found localised in my prostate with a gleeson score of 4/3 which is intermediate cancer.
Because of my age, my surgeon said that I was too old and my prostate too big for an operation, so it was radiotherapty and 6 months hormone therapy.
Today 7 months after radiortherapy I am back to normal and enjoying life again after the side effects wore off.
So the question must be, if I pursued that medical report back in 2006, would I still have my prostate and the agony of it being removed since.?
Imagine what an AI doctor would have advised!
This applies to many other ailments, such as backache, indigestion, joints, brain etc....
Is the AI fit for purpose or is it just a money making machine?
Even then I ask myself if back in 2006, I pursued the PSA level, what would the doctors have advised....I have spoken to several people who have been through this and got treated back in 2006.....
I thank God, I did not do anything in 2006.
I got high PSA reading for a number of years. I'm 73 yrs old as of now. My physician didn't recommend me anything. PSA reading fluctuates/swings over period of years (up then down then sideline , up..etc). high PSA doesn't necessarily means enlarged prostate that needs surgery. It indicates prostate might need medical attention by its owners..
@@alvisjanviar8253
When I visited the doctor last year, he did a physical check of my prostate (finger up the bum), and he said that it was quite large and smooth and took a blood sample for analysis. (PSA)...That was the start of my journey. (followed by an MRI, biopsy etc)
Had he not done that and just said that I had an enlarged prostate, I would still have that lump of cancerous tissue in my prostate for the years ahead and I would still have visits to the toilet, perhaps increasing in frequency..
Whatever, I would have to have it treated one way or the other.
They say that nearly all men over 80 years old will have prostate problems and also prostate cancer.
The average age of men in the UK is about 76 years so if I get to 90+, and still in reasonable health, I will feel blessed.
Life is for lving....
What if you had been biopsied when you were 60 and found to have a much higher Gleason score? Would you have been grateful for early detection of a deadly disease even if more radical therapy had been needed?
@@wholeNwon
What if I had been biopsies at 60 is an interesting question…..then it may have been a different outcome perhaps…..thankfully today we have additional checks like PET scans.
WOW! If 4o is this good, just imagine when ChatGPT 5 comes out and then AGI. Also, just a note. When asking these questions, using voice mode is best; it's like talking to a real doctor. Pretty amazing stuff.😍
I prefer to talk to my -Chat GPT- Lindsay about most things. I’d love for her to get a medical license and available to me 24/7
ChatGPT can provide a diagnosis much quicker than Googling. Additionally, when prompted, ChatGPT will ask the user for more information if needed.
(Not sure the probability of getting the correct answer for the latter approach, but it seemed very impressive.)
ChatGpt and human physicians are the best tools for us as of now. There's FLAW if either AI doctor or human physician is consulted....
The writing is on the wall. Within a decade, seeing a human doctor will statistically represent an excessive source of risk.
The wave of AI medical expertise will push down costs, improve outcomes, and will give us far more personalized treatment.
The days of 5 minute doctor’s visits that cost $500 are in their sunset years.
Baumol’s disease will cease to afflict medicine as expertise is scaled. This is a massive triumph!
I think it will be an assistant
@@menachempike4489 In the transition phase, yes. In the long term, I don’t agree due to the fact that humans will introduce both added cost and a source of error. A pure AI medical provision will outperform the hybrid approach in both efficacy and cost. It will take time to overcome the inertia of regulation, social perception/trust, and general precedent for how healthcare is delivered.
I’ve learned more about a condition from chatting with AI than doctors I’ve talked with, sorry. It was patient, went as deep as I needed into the science, explained its reasoning. Not all of that is the doctor’s fault - they only have so much time with each patient and must code activities with insurance and the hospital management hanging over them. But it was definitely more thorough than most.
I love ChatGPT!
This is absolutely ridiculous. Any of the llms would've given a lengthy response telling you how it came to the conclusion if you had prompted it to do so.
I use Claude 3.5 sonnet for my job as a web dev, and holy cow it's amazing. I won't ever work alone again, I don't see a future where devs will do things by themselves, they will always have an AI companion. Although admitedly that "companion" might take the whole job in the near future.
As the data sets used to train AI improve and as techniques such as better prompting (chain-of-thought), better checking after the fact (rag), better alignment with our needs (reinforcement learning), better retraining with curated domain specific data (fine tuning, and more thorough research before answering (Agents) are employed, error rates are quickly approaching 0.
an error rate of zero would suggest the model is overfitting
Real life example of a case history though:
Patient: I can’t see anywhere.
Dr: Is it blurry far away?
Patient: No I’ve always had good vision far away but now I can’t see anywhere.
Dr: So distance vision is ok?
Patient: Yeah.
Dr: Can you see up-close?
Patient: No! I can’t see anywhere!
Diagnosis after testing was needing reading glasses after age 42 (presbyopia). This was the moment I realized AI will never replace human doctors.
I’m excited to have another diagnostic database tool though
We did the test with 100 resident human doctors. The best score was 55% and the lowest was 29%
Now, I need an AI that can explain chess moves.
I look forward to a checkup with my new ai doctor
My and my wife are both disabled wheelchair-bound. We live in East London Newham. We don't have any social and medical care maybe ChatGPT would care...
Drs are super prone to lacking in.. listening, attention span, in caring, knowledge, bias - in often life-altering ways. I think AI can alleviate a lot of this. Ive tried it and I'm down for more, bring it on please
What has happened with IBM Watson Medical? Why isn't it in the comparison?
ChatGPT knows what you mean without you knowing what you mean
Journalist should introduce her guest stating his past position AND his present position. Those 2 consecutive positions are very revealing.
Consumers also don't use chatgpt with prompt engineering techniques so their experience of it is less than it's full potential.
AI is significantly better at detecting rare diseases too. If it's used as an assistive tool in healthcare rather than a replacement, and overseen appropriately then healthcare stands to improve significantly.
I put my signs and symptoms into ChatGPT before my recent diagnostic procedure, and it returned all the possibilities. The PCP gave far more vague answers.
So when using AI tools for self diagnostic purposes I should include the phrase "Just asking for a friend" .......
Studying for step three my self. I will try putting question to see if it is true or not or just giving broad answer.
How did it go?
That is quite naive and misleading because it does not clarify if the questions were part of the pre-training data set. if that was the case then it is quite normal for ChatGPT to score higher. It would be more interesting if the questions are novel and we know that they were not part of the pre-training dataset. That would involve OpenAi disclosing what relevant datasets they have used for training the model.
Thank you Sir... you ar the first comment that seems to understand what a Language Model is. Kudos! People that have no clue should probably stick to what they do know :)
USMLE Step 3 is not the kind of test that you can crawl on the web easily. There are many sites providing practice questions, so I assume some of them are similar to the questions in the official examination. However, even if a few exam questions have been leaked, there's no guarantee that LLMs have memorized the answers during their training process. Furthermore, according to Gottlieb's description in the video, GPT-4o was able to provide thorough explanations for its chosen answers which shows that it was able to generalize enough to mimic what humans do when they learn the same concepts.
It’s obviously memorization, and so is all of medicine. I was studying alongside med students and all they did was memorization. They also have practical sessions like stitching together a grape or cutting a dead body. But when it comes to diagnosis based on text and image information (so step zero of treatment), it’s mostly about memorization and there is nothing wrong with that and I’m not saying doctors don’t have to be smart, they do a lot of other things at later stages of treatment, but for this particular step LLMs can be as good or better than doctors (actually probably much better) THANKS to memorization and this shouldn’t be seen as cheating, but as a democratization of healthcare access: a 100% good thing.
The ai is less likely to medical gaslight in my opinion. It already has been more understanding about a condition I was diagnosed with that doctors were not understanding for a long time ago before they finally diagnosed me. The condition is “rare” , I went through heavy medical gaslighting in the past that left me with practically ptsd over it . That influenced other people to believe I was lying about my condition when I wasn’t and I was in terrible pain for years. Ironically the ai is more likely to “care” than some people.
Chat GPT is the best software developer I’ve ever worked with!
medicine is about knowledge, it's not like a computer science exam where you need to reason for the most part. of course llms ace knowledge tests
The fundamental problem in clinical medicine is extracting information from the patient. The patient will almost always tell you what is wrong with them, IF you ask the right questions.
How hard is it to teach Ai say your coverage is denied ?
Plus, Ai will be the ultimate excuse to insurance not covering, anything. How do you hold a jumble of "proprietary" code responsible ?
It's also a better attorney. It's coming for us all.
which insurance company is willing to cover an LLM for malpractice?
I wonder whether people will trick themselves into not going to doctors given these resources
Considering it's so hard to see a doctor these days in a timely manner anyway, I think this would be a net benefit
Can it answer what is a woman?
It’s not hard for me to see a doctor, but I’ve been using chatGPT to help diagnose reactions. Also helping me log things into my personal medical journals. It always recommends I go see a doctor
From my experience most doctors are extremely overrated. Apart from surgeons and some highly regarded specialists, most of them are largely useless apart from the basics.
@@celestial_sphere3you do not possess critical thinking skills
We need an AI that can prescribe medicine! Imagine giving a robot 5 dollars a month instead of hundreds to a psychologist! Most adults just need their script written, we don't need someone with an expensive degree to sign a paper for our script!
Whenever I ask ChatGPT for medical advice it tells me to contact a medical professional
That is simply done by design,the creators of ChatGPT cannot take the risk of the chatbot giving wrongful medical advise to a user due to the huge possibilities of failure or injury the person could sustain,which could get the company into very big trouble legally...both with the appropriate authorities and the person they have given wrongful medical advise to.
It's a safety mechanism...that chatbots never give decisive and fully non vague answers to questions the creators might consider Sensitive or controversial.
"ChapGPT" 😂
This is what AI and these LLM's should be used for . Make healthcare more affordable for everyone instead of trying to take people's jobs .
Is it a joke, that the doctors only need to get a 60% on this test?
What kinda physician/doctor are they going to make out of it?
Surprised me, too.
AI is priceless. People are going to be blown away. This is just a tip.
Chatgtp has access to complete database of all answers needed to pass medical exams. So chatgpt took open book test. Shocking it did better. 🤦
Answering a series of questions vs linking multiple symptons and doing diagnosis is a different. Just because AI model answers more question does not mean it becomes a better doctore.
I will also question the study sponsorship. It can easily be rigged. It's just data.
Given the range of medical equipment from Amazon, some of which has iot connectivity, it's a short step to Dr. Alexa, I presume :-)
Thanks for the link to the article. Handy :-)
Ever see the medical diagnostics scene in idiocracy ?
They should do one last House MD about this.
You need to link studies you're reporting about.
If I have the choice of accessing an AI medical team now, or waiting a week to be seen in the UK by a stressed out, barely qualified human, I know which I would choose...
Smart dude on this program! AI Forever!
Medical Terms AI - NO WAITING LINE
This has been going on for awhile. When your provider, especially nurse practioners, leave the room after your initial consultation, they are chatgpt'ing all your medical history, test results, etc to come up with a diagnosis.
Also a dumb take: no, we don't. 😂
Now they can start testing on healing rates, actually judge the doctors on their ability to heal patients instead of answering questions
Why would they do that? :) That is not the goal.
AI: Look at me. I am the doctor now.
open source techniques such as google??
Other purpose of doctor AI is that it will eliminate capitation fee, long years of Educational learning, Educational loan. So basically it will eliminate doctors profession as a high ranking professional and bring it in line with grocery store. Yes and this may be good given your last 3 years' experience of public health.
It may advise on euthanization based on someone who is not recoverable & extra burden on public health.
Win win AI.
“We have RXRX at home.”
Knows more than this guy that pushed nonsense a few years back
Doctors are going to go broke….there goes the beemer and the golf membership
Chatgpt is a brilliant bot.
Human doctors are clearly more expensive and most people are outside of healthcare systems. So AI here is good idea.
Does that mean it can legally write prescriptions?
Hopefully 😂
Almost all of my medical visits may as well be with an AI.The doctors spend most of their/ my time on the computer anyway for their So generous 15 minutes that equates to about $400+ hr.
🤔
AI can be useful, however we did not continue using AI in our business due to its errors.
So what will be the meaning of those years of studying at school 🤣🤣
You still need an INTELLIGENT person to interact with the Lanaguage Model. So far fewer Doctors... yes... but we still need Doctors
@@DolphinAnalyticsAI *doctor already are few ai cut fewer more this is loat of jobs*
Without medical school, trust me, you can't use the AI.
The art of medicine is actually in obtaining a meaningful history from the pt.
If anyone wants to test how trash every corporation is who is publicly distributing AI for revenue take a couple paragraphs of text and ask it to shorten the text with a max of 256 characters and look at the response and check it with a character count tool. ChatGPT, Copilot, Gemini, and Claude all do the same thing. 🤔🤷♂
It's scary that the dead or alive of human life would be in hands of 1 and 0 in the future.
Step 3 is far longer than 50 questions
It’s Hugging Face not Hugging Chat
DoctorGPT, an earlier model that could pass the USMLE... Could run on your phone.
I have met humans that passed the USMLE who could not diagnose disease... passing the test means nothing when it comes to treating patients
GOOD
I don't trust resources that make spelling mistakes....."CHAPGPT"???
AI as it progresses will be able to simulate and test many programs far more thoroughly in the future.
But AI will also be used to crash systems as well as the hackers gets their hands on it.
As complexity increases so does the errors and malfunctions increase.
KISS…..Keep It Simple Simon is a great phrase.
The more complicated things are, the greater the risk for errors to creep in.
AI is another level of complication and complexity by far and so the chance of error grows exponentially.
Even today we see errors creeping in into systems which cause crashes and errors in outputs.
Another saying…. Garbage in, garbage out is very appropriate….,one wrong digit is all it takes……one digit in a million digits.
The whole system of school is flawed, they basically tell you to study anything and everything and then give test or random topics i find that stupid because the probability of us studying the wrong thing is high. They have to be very specific if the point is to understand the concept. This also goes for the medical exam, no wonder ChatGPT is doing a great job! If i were to program an AI to do algebra i cannot ask it to solve calculus this perception shows how stupid the entire systems we have in place in schools!
It fails this question tho:
9.11 or 9.9 - what's bigger?
It does not though,I just asked it and it answered perfectly stating 9.9 is a bigger number than 9.11....I'm using the free version of the model aswell
@@A.MM661copy paste what I've written
It hopefully knows the difference between period and comma depending on your country. 🙏
The AI bots say some interesting things on occasion, but they spew absolute nonsense on others.
You know what they call the person that graduates last in their classes?....Dr
Unfortunately true.
Good day Chap
It's not because Chat GPT is great. It's because the new generations of doctors have no clue what they're doing 😂
ChatGPT should proof-read and edit if needed before broadcasting text in TV
Hi to ChaP GPT 😜
Whatever you can do, tech can do it better 😂
I don't think that's medicine but that's just me!
Its time to rethink how we economy because #JobsAreForRobots #HumansAreForLoving
I can answer questions better than doctors, that doesn’t mean I can heal anyone
It means you can diagnose hundreds of thousands of people within 0.01% the time it takes a human doctor the ever accomplish the same,and it also simultaneously means you can assign the correct medication and drugs needed for every one of those hundreds of thousands of people you can diagnose.
It means you can heal a lot of people,chatbots have no concept of time,they can exist in 1,000 computers at the same time and answer questions for 24 hours every day for 10 years.
In the future hundreds of thousands of people can be diagnosed in hours and given appropriate medication in that same time,all that costing 20 dollars a month (which is what a ChatGPT+ subscription costs)
Did they spell it wrong on purpose 😭? ChapGPT
3:34: “THE AI DOTOR WILL SEE YOU NOW”.
Next up, A.I. that can catch and correct silly spelling mistakes. We could call it…oh, I dunno…Spellcheck?
Feed AI everything, sooner or later even doctors will loose their jobs..
ChatGPT could have saved me about 17 years of b.s. and awfulness with my lazy, incompetent and arrogant former GP who hooked me on narcotics in 2007 and then tried to walk away. Did nothing to help me when his reckless overprescription had resulted in me becoming heavily addicted to benzos. I got his ass punished by his medical board, but the damage was done.
But can't do simple math.
It's a fast Search engine capable to predict the next word (token)
both these ppl, really have no idea about the landscape of AI.
That’s just data. The map is not the terrain. The robot has no street skills or real world experience.
Hugging face right not hugging face
As if healthcare could not get more impersonal and out of human touch in the US.