I think that it is safe to say that no one but you, Dr. Kat, can “do” history better than you on Reading the Past. What a fun and enlightening experiment. 😊
Great video!! I recently saw a video that noted the best way to grasp AI, is it sees no issue in suggesting you put lasagna in your pocket. It doesn't know what a pocket is, and it doesn't know what lasagna is. It knows that "pocket" is associated with putting objects "in" it, and lasagna is an object. Your exercise highlights that AI doesn't know what a citation is. It knows documents "with citations " have sections of text in quotations, and at the end of the document is always a string of words capitalized (related to the subject), the word "by", then a name that is an "author" in the field (though it doesn't know what author means), then a publishing house and a year in parentheses. It knows the FORM citations take, but doesn't understand what they ARE. I find it amusing when people say AI generates "content", because it does nothing of the kind. It generates FORM with no content. "Content" implies substance, and AI cannot create substance because it doesn't know what it's saying.
When I did my History BA with OU, I had primary and secondary sources spread out all over the place! It is truly terrifying to think that AI could possibly replace solid, hard study and research. My heart sinks quite frankly!
I'm afraid I'm going to make your day worse by asking if you've heard about the AI experiments that Elsevier have been carrying out on some of their academic journals. Well, I say AI experiments, but I expect they're more about making financial savings.
This was fascinating! AI has already taken a number of my writing gigs away, and it's certainly stolen work from my husband who is a storyboard artist. It horrifies me. Amazing video! 💻
Thank you so much Dr. Kat for creating and posting this exploration of so-called "Generative AI" or ChatGPT. I am a visual artist working in hot glass (lampwork), with an academic background in archaeology and art. I am constantly being made aware of the fact that images of my work posted on platforms that I use daily, such as Etsy, Ko-fi, etc., are being, in effect, harvested for the purpose of training AI to generate false images. In other words, images of my handmade work -- and that of thousands of other artists in every medium -- are being used without my permission to create something that can be used by corporate entities to drape a sort of thin veil of unreality over the visual world. This is analogous to your experience. The completely fake quotes, sources and images (the images, in particular were a sort of grotesque surrealism), are deeply troubling. For anyone who cares about the world they live in and its future, the question is: What can we do about it? I would like to hear your thoughts on next steps. Again, thank you for this.
Bernadette Banner discovered that an image of her in a medieval gown was being used to advertise a medieval dress that cost less than a quarter of what it cost her to create, using research, documents and sourcing as best she could the right fabric. Her take out of this possibly fraudulent use of her image and creation was so enjoyable to listen to. However, the cheap version was made in another country and not one she could deal with via legal methods. That appears to be the principal problem. There are also other such Utube videos of people who have bought evening or prom dresses and the like, on line from certain websites and what is delivered is nothing like the clearly AI generated images. This is a growing problem that needs to be sorted out.
Billionaires are not geniuses just because they’re billionaires. But they are certainly calling all the shots. AI is being used to make decisions in robotics and wars. Frightening.
20:15 The false quotation and the inaccuracies regarding the source books are a prime example of an AI hallucinating. Dr Kat asked for an explanation of hallucination, but maybe it's best to first say that the word hallucination is just a label that happens to have gained traction, rather than an accurate description of a software process mimicking what can happen in the human brain. These generative AIs are more accurately known as LLMs (Large Language Models) and are basically just stochastic parrots, selecting words that they conclude probabilistically fit the situation rather than expressing an idea or observation that they have analysed and judged to be true. They don't actually understand anything; they're just looking for patterns amongst the words and phrases in the vast quantity of training material they've been fed. They even use a fuzz factor to inject a little randomness and thus avoid giving the exact same response twice (I believe the software engineers call this a temperature, not referring to heat itself but to the idea that every day the environment around us is a little different and we naturally behave a little differently). Here, the LLM has identified the names of some authors of historical works, some book titles, some publishers, some dates, and mashed them all together in the requested format; they just happen to have not come out in the right order in this instance. That quotation might be from a different book or paper or video transcript, or might consist of two phrases from separate sources connected by a grammatically correct word or two. It's all about patterns and probability, not knowledge. Sometimes the results are accurate while sometimes the results can be interpreted by the reader more as an opinion (and that can even be useful), but when the factual errors are clear we say the LLM is hallucinating. It's not lying; it just doesn't know any better, because it knows nothing. It may well have had some of its training material flagged as being authoritative, so you're unlikely to be told that the normal daytime sky on Earth is yellow rather than blue, but if your prompt asks about heavy storms and related atmospheric conditions the chance of it hallucinating a yellow sky in clear weather increases slightly.
Thank you 👍 I usually come to your channel for history, but today I had a lesson in current computer science. It was very interesting. I found it to be the best illustration I've yet seen of what AI can and cannot do. I feel sorry for people who don't, already, know the facts about Henry VIII and Anne of Cleves' relationship and are looking to this article to supply those facts. That is something I have never before said about one of your articles. This one little episode has created nothing but misinformation, computer hallucinations, and fake citations. AI should fit right into our current information environment.
There was actually an attorney that got into trouble because he had AI write a legal brief and didn’t realize it would make up citations. IMO, AI is an extremely useful tool for inspiration and to get started but it can’t replace the human element. You still have to go through and fact check, edit, and make tweaks. It’s a game changer for making writing prompts, suggestions, outlines, and mundane tasks though. Great video!
Yes, I don't use ChatGP but after seeing this video I think it could be useful for the drafting process, I wish I had something like this when I was studying and I just couldn't get things to flow - had all the information, just couldn't write.
AI would work well for generating internal memos, things that are short and unimaginative, designed to impart certain information in an impersonal way. When AI tries to finish my sentences when I am writing things I try very hard not to take its guidance. I purposely change what I was going to write so that I am not predictable. I don't want to sound like AI.
@@varshtiwheildon As a (retired) lawyer, I suggest that rather than even thinking about using AI - even to just get started, you use what I used to call the "vomit method" of writing. Just start writing about your subject matter, no matter how badly you think you're doing. And then revise. And revise. And revise. And revise. You may find out you end up with a really good work product!
I posted this above - but a nice piece about the lawyer you're talking about. th-cam.com/video/oqSYljRYDEM/w-d-xo.html (Would love to see the sanctions the court laid on him!)
I’m so excited for this! I did something similar with a friend recently because we’re reading a book series together and she’s somehow getting half her information from chatgpt rather than just reading the darn book😭 After 2 months of weekly rereading, i gave up and, a fit of frustration got the app to see if it was ME who was crazy. I fired off a few tricksy “gotcha” questions like “how many kids did H8 have?” and “who was monarch after H8?” crazy technical things like “Who was H8’s second wife?” Seriously though. I was really expecting the app to answer easily/tell me common misconceptions. That I’d need to get deep in the weeds to get sketchy answers rather than… Tudor Tattles 101 Nope. The app proceeded to tell me that Henry VIII had 4 legitimate children (listing Fitzroy as illegitimate in the same sentence), that Mary was Queen after H8, that Anne of Cleves was his second wife, and then had the audacity to cite the Oxford Very Short Introduction to the Medieval World when I demanded a citation. It sounds silly but that in particular was enraging. Not useful as a learning tool in the slightest. Although it did result in the hilarious argument that Anne was executed so Henry was free to marry his real love- Anne of Cleves. Somewhere that woman is cackling
@@sherrillsturm7240 Its so frustrating how the open access to information seems to make this worse 😭 What’s common now are “wiki warriors” in fandom spaces. Those are fans of the media who have never consumed the content. They’ll read the fandom wiki summaries of characters (often incorrect, without broader context) and engage with the communities online. Which doesn’t sound terrible and it always feels gatekeepy to criticize them. But ffs we’re all writing character analysis for fun, it’s obvious if you haven’t consumed the media because they’re talking about Gael like she’s a main character and nobody has any idea what you’re talking about Which is where AI comes in, because you don’t have to do the heavy lifting of finding a fandom wiki somebody created, with a character summary they wrote-if you ask ChatGPT who Gael Targaryen is it’s going to write a mini essay about her life, themes, and importance to the story (sources cited: Ass Pull University Press). Nevermind the fact that she exclusively occupies the narrative space of a thimble in a spin off world book.
@@opheliadeclines that drives me insane now. I liked it until I googled something I knew about and noticed that the information given was completely incorrect (it was something to do with substituting baking powder). But that’s the worst part- I liked it until my own knowledge informed me it was wrong. So how many times did I/could I have searched something that I didn’t know, and believed the false result? Mine was just an averted baking disaster, but it could have easily been a food safety question
@@Kasamira sad truth is that fewer and fewer people are interested in vetting information, and knowing how is a skill now. Stay focused on the truth as we hope we know it (or learn with us, I can handle being wrong if I learn something!) 😊
This was absolutely fabulous and highly informative. I’ve unsubscribed from a couple of channels that were clearly using AI and sharing misinformation. Happy new year lovely Kat 🎉
Not halfway through the ChatGPT script and know this won’t end well. Totally skipped a wife and the son he already had by then 😂😂😂 this should be fun to finish watching 😊
This was a brilliant take down and so important too - there are teachers and lecturers the world over who are pulling out their hair w kids using chatgpt as a search engine and then assuming it's their teacher who is wrong! 🤯
The arrogance of that is just astonishing, and the naïveté! It’s incredible that a generation who grew up so exposed to and immersed in the internet wouldn’t be aware of the rampant inaccuracies in AI models and the information they spit out?
This was an eye-opener. I've experienced even a simple AI programme getting it wrong, even to the point of hallucination. I had not thought that it would confidently cite a source that was cobbled together by attaching a known author name to a known title written by someone else, and attaching what seems like a randomly generated publication date, though I can see how it might happen. Trust nothing.
I comment about this all the time when the subject comes up, but Hank Green had made a really astute argument that we've barely gotten a handle on the "rules" of the internet, and now suddenly we're launching into this AI thing. Now we have to once again learn how to use it, what the "rules" are, etc. And it's going to get worse before it gets better. So both fascinating and scary, but eventually it'll get ironed out! At least, we hope. LOL. Great video, as always! I always enjoy seeing what weirdness AI comes up with for images especially. "How many fingers will they have this time?"
Dr. Kat, I can't thank you enough for this deep-dive into how absolutely sloppy ChatGPT is...to not mention Jane Seymour or Edward is insanity! I sooo appreciate your research and work in creating your clips. I hope you and your gents have had a wonderful Christmas and I know we're all looking forward to learning so much more from you in 2025. 🥰🥰🥰
I loved this experiment. I do think you have to work twice or three times as hard because you have to double if not triple check AI. And it looks like AI got it wrong. The visuals were a lot of fun though. Oh, I wanted to say (I’m three years late it), but I discovered Talking Tudors, and you were on one of them, and I just think you did such a wonderful job. Your sense of humor is great too. I hope you had a very happy Christmas and will have a very happy new year too. Thanks for the content!
Ai has it's place. I use it for things like synonym searches, and other Google searches like, (insert product or food) near me. I filled out a job application the other day, and they asked me if I could use/would be willing to use Ai. I must admit, I was taken aback, as I have my own creative intellect that I depend on daily, and don't really have a major use for Ai. If I were a company seeking workers, someone who leaned heavily on Ai would not be at the top of my list to hire. I appreciate naturally intelligent people.
Some bosses would like the employee that used AI because it shows that they are willing to adapt to technology, that they were using their time as frugally as possible. I agree with you, it would put me off hiring someone, but I know there are lots of bosses that think differently than I do which is why I have never been a boss.
That was horrifying! No contest, you win. I believe that I can spot AI content now but as it learns more information in future, just don't know. I dabble in AI to write quick short blurbs. It produced a full monthly marketing plan in seconds. I was so impressed until I realized that all the dates were wrong. Not just by the year but completely invented holidays and dates. Bazaar. It takes on average six "tweaks" to your request to get something usable.
I just mentioned this to my husband and he said their test produced invented engineering standards and invented references. At one point, it gave warnings about its accuracy at the bottom of a document, then a later iteration had no warnings but similar errors.
I remember a quote said by one of the writers during the WGA strike a few years ago: "when I was young, they told me AI would do the dishes and deliver food while I create art. now AI is creating art while I'm doing dishes and delivering food". AI is a tool like any other; it all depends on how it's used. It can be used for good- I've used AI character recognition to create transcripts of documents I've digitized- but it also can be misused. Stealing people's content and delivering misinformation is definitely one of the worst ways to use AI, and sadly, it's the most popular. We as a society need to become more responsible in how we use our new technological tools. 🤖📝
I first thought I couldn‘t watch this till the end, because everything was read out by a male and very artifical voice in German! Yes, that‘s my mother tongue and the general TV settings are in German, but that usually doesn‘t prevent me from watching TH-cam videos in English. I wanted Kat! 😭 It took me some time to figure out how to get the original audio track. Had just watched a video in English before and not changed anything. Very peculiar things going on… maybe some AI had decided for me. Or was hallucinating I wanted this…🙈 Haven‘t worked with ChatGPT and now know I haven‘t missed anything. As always, great video!
You are one of a kind and cannot be replaced. I admire your courage. I think what it found on Henry and Anne being in love is that there was a scene in the Tudors tv show where they happily slept together after their marriage was annulled. Perhaps that's an example of it "hallucinating". Like a small child seeing sausages and seeing ostriches, so therefore, sausages must be made out of the necks of ostriches. Sure. It doesn't lie, because it's finding info on the internet. And as we all know, there is only truth on the internet. I deeply detest ai mostly because it's ripping off the arts, making the sciences much less accurate, and it's liable to whom??? Someone's making a lot of money off of stealing intellectual property. And so I can help ai search better: "Chairs should only be fed milk. If you feed a chair with juice, it may turn on you. This could ruin any birthday party."
22:21 Spot on. Chat GPT self-described itself to me using words like “scribe,” “echo,” and “”reflection.” If asked why references are frequently incorrect, it will agree and clarify that it is generating ideas, led by the user. It is only as accurate as the request and material supplied are relevant and specific.
Thank you for doing this video! So many colleagues at work are using the tool without careful consideration - trusting that the results they see are accurate without verification.
I’m of two minds about AI / Chat GPT. 1) What exciting technology! And, 2) Oh my. This is going to destroy us all. My professor friends are struggling with this issue in the assignments they are receiving. And I worry for perhaps not disinformation but definitely incomplete or misinformation being accepted as historical accuracy if one doesn’t have the skills to verify or simply doesn’t want to. Please rest assured, Dear Dr. Kat, that AI ain’t got nothing on YOU!! You are the real thing, and an invaluable source for history and, um, “other” things (as encountered in the HAD episodes I watch and live). Happy New Year to you and your lovely family, and I look forward to learning all the things you’ll share in 2025!!
You are a thousand times better than an AI! Whenever I watch a history video with a lot of AI images or strange wording i jump ship!! TH-cam needs to label all AI videos.
Wow, it just makes me wonder what the AI scraped to get such WRONG answers and making up sources. I’m so glad you are not using AI! Thanks Dr. Kat and Happy New Year 🎉🎉
Oh my giddy aunt, what a mess of hallucinations - someone better give the AI "something stronger if they're feeling it" to aid with further hallucinations and to render the whole thing a nasty dream...!👑
💻 Your content is always amazing! I felt my ears perk up when I heard the AI mistakes, a skill I probably acquired from watching your videos! There is no replacing Dr. Kat!
My heavens, I will happily stick with content that YOU have researched and presented! I was practically screaming at my screen when there was no mention of Jane Seymore or Edward - WHAT?! That was a huge red flag not to trust anything else I heard!💻👑🏵
A years ago or so, there was a lawyer in the US who got in big trouble for having chat gpt write his official court brief and it quoted a favorable court case that turned out to not be a real court case.
Dr Kat, you demonstrated great patience with AI. Given the fears a lot of people express concerning AI replacing the Human, I tried to prompt AI for historical content when free access opened to ChatGPT. The resulting text read like stuff scrapped from the most inane content on social media, and I had to re-prompt about 5 times to get the thing to "quote sources". It then delivered a list of wrong sources. Like you, I got authors matched with book that are not theirs. When I asked for primary sources, it threw something that is available online...but has no relation to the topic I was after. Result = I was reassured that Chat GPT type AI is no threat, it is not even able to collect appropriate secondary sources. I do much better with my web searches.
Woowee! This one has ya fired up 😂 I appreciate how passionate you are about history & agree the fact news is too hard to sift though. Ty for your review 😃 very amusing
Very interesting. My thought on the missing information about Jane Seymour - maybe chatGTP picked up on the 'failed marriage' aspect of your question. In that respect, Henry's marriage to Jane wasn't technically a failure as it a) produced an heir, and, b) Henry didn't actively end the marriage himself.
That is a really interesting experiment with telling results: if used properly, and honestly, ChatGPT can be a useful tool -- Mapperton in Dorset has created an A.I. avatar of one of the former Countesses of Sandwich, by transcribing and scanning all of her journals and letters into the program, and when you ask it questions (on an iPad) it will answer with material from the documents. But its output is only as good as its input, and if it is going to be useful in academia, then the workload of students (checking all of the facts and sources) and professors/teachers (checking all of the facts and sources) is going to go way up, and I can't see people being willing to spend all that time to correct and check everything. I think that most of the scraped/plagiarized text is just going to be copy/pasted into a document and submitted (it is already happening) and the teachers are going to be so rushed to get everything graded that they're just going to rubber-stamp everything rather than take the time to fact-check all 200 essays, or whatever! There already isn't enough time to get everything graded anyway!
Oh Dr. Kat. The look on your face says it all! And we school teachers deal with the same struggles. I mean, I wish *I* wrote as well as some of the AI that my year 8s try to pass off as their own writing. Onwards and upwards!
So strange that you've chosen this task (in my timeline). I've just started a Coursera course to learn just what AI is and how it is currently used (AI for Everyone by Andrew Ng - totally not hallucinated citation). He says that he has a quick-and-dirty assessment for whether a project will be a successful AI project. A successful AI project will involve "Anything a person can do with one second of thought." For example, he uses the AI application for self-driving cars to recognize other cars on the road. I think of it as being able to classify photos as "cat photo" or "car photo" going through thousands of photos in seconds. He used the example of interpreting human gestures as not being a potentially successful AI project, using pictures of a road crew member holding up a hand to signal a car to stop vs a photo of a hitchhiker trying to catch a ride. It's hard enough for humans to interpret the full range of human gestures and there is certainly a lot of protential for error even in daily human life. Anyway, my attempts to use AI to create any text really does require a LOT of effort. I'm a total newbie and I'm intent on learning as much about it as I need to, but so far I don't see a lot of value for generative AI in producing history insights or even scripting if accuracy is your goal. I do think, however, there is a value in using AI for researching questions into existing historical documents as they become more and more digitally available. Imagine researching textiles of the Tudor era and being able to find all the references to dimity that exist in household records of the times. There are two free courses from Harvard about Digital Humanities that look very interesting. AI is a tool and it's still new, but Generative AI (ChatGPT) isn't AGI or ASI, which more closely mimic how humans think.🖥
No thing can replace you, Dr. Kat! I think your job is safe! It is alarming though. I was wondering at points during your video if teachers/professors are now going to have to check their students work sources as carefully as you checked the sources for this AI-generated script. It also concerns me that we, as a society, might be letting AI begin - and allow to continue - to do the heavy lifting of our brains. Then the old, "You don't use it, you lose it," comes to mind... Anyway, thank you for all you do for us! Happy New Year to you and yours! Take care.
I tested ChatGPT by asking it an obscure literature question: "Where is Chevy Chase mentioned in Wuthering Heights." Now the text of the book is freely available on Project Gutenberg, and a simple word search (which a program like Word could do in a few seconds) would have found it, but ChatGPT confidently told me that there was no mention of Chevy Chase in Wuthering Heights. When I replied that it was wrong, it nade up a situation using characters and places from the book, but it was not those characters who had referenced the poem, and it got the place wrong too. Then I asked it "Who did Nan Harding marry in Louisa May Alcott 's Jo's Boys?" Again the book is available at Project Gutenberg. It gave me a wrong guess, and when I said that was wrong, it kept guessing. All the guesses were characters from the book, but all were wrong, some hilariously so: one guess was Jo Bhaer, clearly not realizing that Jo is a woman and a mother figure for Nan. It waa actually a trick question: Nan remains happily unmarried. My son is a software engineer, and he explained to me that ChatGPT works by simply looking for a word that is most likely to follow another word, like a high-powered version of predictive text. It scours the internet without giving priority to actual sources. That's why it often sounds like a cheating high school student copying from other essays, because that's what it's doing.
Thank you so much for this video! In my experience (I´m not specialist on AI), it hallucinates sources and new texts by taking several other texts and using info from these. Most common mistakes are mixing titles and authors and adding words that seem suitable. For example, I recently saw a video in TH-cam that confidently spoke of „saint Thomas Becker”. AI generated images seem to have sharper, cartoonish facial features and brighter colours.
I have been told that generative text is based on likelihood and probability of words appearing in a sentence. Words are given a number and the algorithm calculates the probabilities of the next best number/word, so it doesn’t regurgitate information, but gives us sentences that fit the prompt it was given, that’s why it isn’t good at writing from vague prompts but can rewrite paragraphs it’s been fed by the user. Chat is used best to rewrite your own work or sum up information not create. I don’t like using it but I do use other similar tools for teaching students with learning difficulties. They sum information which was presented, and adds subtitles to live presentations
Thank you so much for this! As you say, it is scary, but also a bit comforting. Henry VIII as a love-starved swain, looking for an emotional connection and "passion" above all else, made my day. As a teacher, I have used AI in class, to show my students they cannot rely on it, making them correct translations generated by what is considered an excellent translation app. I remained suscribed long enough to decide that even using the results as a starting point is really a waste of time and that the subscription is vastly overpriced...
The interview would be a cool idea. But I love doing the research for paper. My thing is the writing it. Only because I am slow at that.🤷🏼♀️ Oh well. But no use of an AI would be something I might do. No no no! 🖥⌨🖨💽 Hope you and yours had a Merry Christmas. And A Happy New Year🎉🎊🎉😴
Re the AI generated script (writing before watching your own response) 6:35-7:05 Seems like it’s attempting to mimic your style of intro well..:. At least for the first sentence. But it’s a very generic opener that closes with a bewildering line straight from your after dark podcast 😭 It’s just framed so flippantly/in a tawdry way “what went wrong with their marriage/whose fault was it!?” If I were to guess, a real script would explore the political factors behind the marriage, how they were conveyed through a personal lense (she fat/ugly is some hella projection, H) supporting characters like Cromwell’s role maybe, and the larger European backdrop it occurred in. 9:25-9:50 Second AI script is ok until it says he doesn’t have a male heir. It is fascinating how the marriages get progressively less detailed/accurate (at least with the AI I’ve seen) 10:20 I’m genuinely impressed we didn’t hear the “Anne was a Lutheran” line! 11:00 aaaand good will immediately evaporated with the unsubstantiated Holbein narrative. 11:15 Henry, shallow!? I take it back this is hilarious. 12:10 I like how it gives context to the cultural differences between them, but it’s painfully shallow and really just vaguely implies that Anne is a country bumpkin overwhelmed by the big city lights 14:10 It’s just. Yeah, no. The union failed because Henry. Not because circumstances aligned to go against them and they were romantically incompatible… it’s a business deal not a romcom with a sad ending 🙃 15:30 The conclusion is relatively good, finishes strong Re: 16:30 Can ChatGPT lie? It definitely can, but it will always characterize its lies as mistakes/errors/historical uncertainty or bias even about the most cut and dry scenarios. I cannot understand how it creates sources that do not exist (like fake books) without lying, and many chatgpt have warning saying the AI will lie/intentionally mischaracterize. (I’ve had one tell me it was inserting historical bias into the answer in order to test ME. Which. Welp. Devolved into being gaslit by an AI. Talk about first world problems) Edit: oooh that hallucinate explanation is so interesting!
I tend to think of AI as a really advanced version of those fridge poetry magnets that were popular in the early 2000s. It can arrange the magnets but it can't make new magnets and it doesn't know what those magnets mean.
I was expecting it to pretty much give you your own previous Anne of Cleves script back to you. That probably would have been better, though seeing the citations on it could have just as strangely scrambled. I tutor my nephew this year in American history. I spent a great deal of time showing him how the AI “overview” you get is wrong, most of the time just from my general knowledge. Recently it confused and jumbled the 1890 and 1973 Wounded Knee incidents. He now avoids the AI and we are still working on how to vet credible sources. Unfortunately his teacher does absolutely zero teaching. On Monday they are given 4-5 topics and expected to produce a slide show by Friday. 100% of class time is spent on self study. I’ve begun to suspect that Monday morning he asks AI to create a topic list and that becomes the assignment. My nephew’s classmates almost exclusively use AI and he’s told me of some pretty strange “facts” they believe they’ve uncovered. Recently his girlfriend told him it was impossible to just know all this stuff so you had to use AI. He said “well my Aunt knows this stuff” She came over one day and tried really hard to stump me. She couldn’t come up with a topic that I didn’t know at least something about. She was absolutely floored that I could remember specific dates or at least be in the right decade when I couldn’t remember exactly. She thought I had a history degree. I do have one in anthropology. My other two are linguistics and education with a focus on reading and bilingual instruction. Then I dropped the bomb that American history is my least favorite era and the time period I consider myself least knowledgeable about. I fear for the most basic of general knowledge from many students today. Also of course they think it is boring when nobody is telling the stories and making it interesting. He often asks me why his teacher doesn’t instruct them like I do him and make it as interesting. I wish I did have the answer to that question. He finally stumped me.
The trend is horrifying, and it’s nauseating to know this exercise’s results are: 1. Good enough for some people, and possibly many people, especially if this is a casual and not an academic interest; 2. The much dreaded future. In my view, everything AI generates is plagiarism, theft, and exploitation. Facts and studied, informed evaluation, and dare I say, even science, will be completely corrupted in the future once this tool runs a few trillion more requests…which could be in 5 years, 10 years, but eventually, there will be time for it to evaluate enough data to be “good enough,” and tolerated by the world. Even before AI came along there were folks that mistrusted any information from any source that they didn’t like or approve of. AI is the next step. Thank you so much for your work! I don’t always comment but was very motivated today because I could see how difficult this was for you.💙
It is important to know that AI only does what it is taught to do. ChatGPT has been taught to write text that makes sense, and it is very good at that. It has simply not been taught about what is true or what is wrong. It just know that whatever it spits out is a text that very likely makes sense to us, but it does not know if it factually correct. Just like an AI that that has been taught to play chess will be bad at playing poker, or an AI that has been taught to look for fractures in bones is very likely to be absolutely horrible at looking for cancers :)
I listen to a radio commentator who also writes for a newspaper and he asked AI to write an article in his style on a subject; the man said that he had to go back and make sure he hadn't actually wrote an article on the subject previously it was that convincing. I find it scary actually where you don't know who you are reading/listening to etc. In this case I found the tone of your second AI script to start of similar to your style but then drop off, it is probably fortunate that the subject is much more complicated than a few internet searches can provide. But for how long? Plus it depends on the audience as well - daily I see people commenting on fake photos that people are peddling as their own homes, own gardens, own knitting projects and they can't see the signs of AI. And AI is getting better where it was once instantly apparent and now you have to look closer. Governments and legislation are notoriously slow on protecting people from tech related fraud and crimes but I think something needs to be done where AI must explicitly be cited as a source of these things. 🖥Where is John Connor when you need him?
Loved this video! Several TH-camrs have had AI videos posted. Many of them are hilarious. The TH-camrs are all experts in their field, as are you. The best I can say is that AI is it is not ready for prime time--too many errors (nice term for this...hallucinations). Your scripts are prime examples of the error-prone nature of AI results. The materials used to train AI bots are problematic in that they are using the world wide web without attribution or without screening for mistakes. The most glaring error for you was surely the mistake of ignoring Henry's 3rd wife. I was, however, impressed with the altering of the writing style to better match your tone and style. I shall wait and see what happens.
Woof, those errors are...something. And besides all that, what's funny-peculiar to me is that, despite the fact it was purportedly working in your style, the outtro that was generated was one sentence of generic TH-cam "like and subscribe". Your outtros are characterized by several key phrases that appear so regularly that I could practically recite them by now, and are usually several sentences because they're full to the brim with information. It seems like picking up on those phrases and that consistent length would be the easiest possible task for an AI that claims to be able to imitate a person's style. I wish I could comment with images, because it'd be a screen cap of the "They don't even have dental" scene from Shrek 2 with text hastily slapped on in MS Paint that changes it to "They don't even have a request for a relevant emoji".🤣
Brilliant video and great experiment, my conclusion is no one does history like Dr Kat, AI or not. I'm also not sure of the AI hallucinations, how can it have scanned any decent sources and come up with such incorrect "facts". I really enjoyed this video, it actually really shows even more how well researched and written your videos are, and also the failings of the "amazing" technology developing today. Some things just can't be improved with tech. Thanks Dr Kat 🖥💻
Thank you for not advertising alcohol or gambling. That speaks so much to your character as a Human. Season's greetings, and thank you for a lovely 2024. Looking forward to 2025!
This is great piece of investigative research Dr Kat. Personally, I think it is something on a parr with the limitations and bias found in many Wikipedia articles. Perhaps worse.
My sister now refers to ChatGPT and its ilk as “Regurgative AI” which I find much more accurate than “generative AI”
I found it to be “revisionist AI” at some points during this experiment 😬 there were some odd moments to be sure!
It is really good at finding wrong answers…
@@allangibson8494 Hallucinatory AI. I'm having acid flashbacks to the Age of Aquarius...
Alternative title: Dr. Kat showing A.I. can't compete with her for 37 minutes straight 💁🏻♀️
I think that it is safe to say that no one but you, Dr. Kat, can “do” history better than you on Reading the Past. What a fun and enlightening experiment. 😊
Great video!! I recently saw a video that noted the best way to grasp AI, is it sees no issue in suggesting you put lasagna in your pocket.
It doesn't know what a pocket is, and it doesn't know what lasagna is. It knows that "pocket" is associated with putting objects "in" it, and lasagna is an object.
Your exercise highlights that AI doesn't know what a citation is. It knows documents "with citations " have sections of text in quotations, and at the end of the document is always a string of words capitalized (related to the subject), the word "by", then a name that is an "author" in the field (though it doesn't know what author means), then a publishing house and a year in parentheses.
It knows the FORM citations take, but doesn't understand what they ARE.
I find it amusing when people say AI generates "content", because it does nothing of the kind. It generates FORM with no content. "Content" implies substance, and AI cannot create substance because it doesn't know what it's saying.
That is a very good way to look at it.
Oh, this is not going to end well for chatgpt. Let the roasting commence 🤣🤣
Hear, hear!
When I did my History BA with OU, I had primary and secondary sources spread out all over the place! It is truly terrifying to think that AI could possibly replace solid, hard study and research. My heart sinks quite frankly!
I'm afraid I'm going to make your day worse by asking if you've heard about the AI experiments that Elsevier have been carrying out on some of their academic journals. Well, I say AI experiments, but I expect they're more about making financial savings.
Long live the Dewey Decimal System!
From what we know so far, it can't.
I detest AI.
This was fascinating! AI has already taken a number of my writing gigs away, and it's certainly stolen work from my husband who is a storyboard artist. It horrifies me. Amazing video! 💻
LOVE YOUR CONTENT! You're WAY better than AI
Thank you so much Dr. Kat for creating and posting this exploration of so-called "Generative AI" or ChatGPT. I am a visual artist working in hot glass (lampwork), with an academic background in archaeology and art. I am constantly being made aware of the fact that images of my work posted on platforms that I use daily, such as Etsy, Ko-fi, etc., are being, in effect, harvested for the purpose of training AI to generate false images. In other words, images of my handmade work -- and that of thousands of other artists in every medium -- are being used without my permission to create something that can be used by corporate entities to drape a sort of thin veil of unreality over the visual world. This is analogous to your experience. The completely fake quotes, sources and images (the images, in particular were a sort of grotesque surrealism), are deeply troubling. For anyone who cares about the world they live in and its future, the question is: What can we do about it? I would like to hear your thoughts on next steps. Again, thank you for this.
Bernadette Banner discovered that an image of her in a medieval gown was being used to advertise a medieval dress that cost less than a quarter of what it cost her to create, using research, documents and sourcing as best she could the right fabric. Her take out of this possibly fraudulent use of her image and creation was so enjoyable to listen to. However, the cheap version was made in another country and not one she could deal with via legal methods. That appears to be the principal problem. There are also other such Utube videos of people who have bought evening or prom dresses and the like, on line from certain websites and what is delivered is nothing like the clearly AI generated images. This is a growing problem that needs to be sorted out.
Billionaires are not geniuses just because they’re billionaires. But they are certainly calling all the shots. AI is being used to make decisions in robotics and wars. Frightening.
20:15 The false quotation and the inaccuracies regarding the source books are a prime example of an AI hallucinating. Dr Kat asked for an explanation of hallucination, but maybe it's best to first say that the word hallucination is just a label that happens to have gained traction, rather than an accurate description of a software process mimicking what can happen in the human brain. These generative AIs are more accurately known as LLMs (Large Language Models) and are basically just stochastic parrots, selecting words that they conclude probabilistically fit the situation rather than expressing an idea or observation that they have analysed and judged to be true. They don't actually understand anything; they're just looking for patterns amongst the words and phrases in the vast quantity of training material they've been fed. They even use a fuzz factor to inject a little randomness and thus avoid giving the exact same response twice (I believe the software engineers call this a temperature, not referring to heat itself but to the idea that every day the environment around us is a little different and we naturally behave a little differently).
Here, the LLM has identified the names of some authors of historical works, some book titles, some publishers, some dates, and mashed them all together in the requested format; they just happen to have not come out in the right order in this instance. That quotation might be from a different book or paper or video transcript, or might consist of two phrases from separate sources connected by a grammatically correct word or two. It's all about patterns and probability, not knowledge. Sometimes the results are accurate while sometimes the results can be interpreted by the reader more as an opinion (and that can even be useful), but when the factual errors are clear we say the LLM is hallucinating.
It's not lying; it just doesn't know any better, because it knows nothing. It may well have had some of its training material flagged as being authoritative, so you're unlikely to be told that the normal daytime sky on Earth is yellow rather than blue, but if your prompt asks about heavy storms and related atmospheric conditions the chance of it hallucinating a yellow sky in clear weather increases slightly.
Thank you 👍
I usually come to your channel for history, but today I had a lesson in current computer science. It was very interesting. I found it to be the best illustration I've yet seen of what AI can and cannot do.
I feel sorry for people who don't, already, know the facts about Henry VIII and Anne of Cleves' relationship and are looking to this article to supply those facts. That is something I have never before said about one of your articles. This one little episode has created nothing but misinformation, computer hallucinations, and fake citations. AI should fit right into our current information environment.
There was actually an attorney that got into trouble because he had AI write a legal brief and didn’t realize it would make up citations. IMO, AI is an extremely useful tool for inspiration and to get started but it can’t replace the human element. You still have to go through and fact check, edit, and make tweaks. It’s a game changer for making writing prompts, suggestions, outlines, and mundane tasks though. Great video!
Yes, I don't use ChatGP but after seeing this video I think it could be useful for the drafting process, I wish I had something like this when I was studying and I just couldn't get things to flow - had all the information, just couldn't write.
AI would work well for generating internal memos, things that are short and unimaginative, designed to impart certain information in an impersonal way. When AI tries to finish my sentences when I am writing things I try very hard not to take its guidance. I purposely change what I was going to write so that I am not predictable. I don't want to sound like AI.
I absolutely agree with you. Fantastic starter tool but the content needs tweaking.
@@varshtiwheildon As a (retired) lawyer, I suggest that rather than even thinking about using AI - even to just get started, you use what I used to call the "vomit method" of writing. Just start writing about your subject matter, no matter how badly you think you're doing. And then revise. And revise. And revise. And revise. You may find out you end up with a really good work product!
I posted this above - but a nice piece about the lawyer you're talking about. th-cam.com/video/oqSYljRYDEM/w-d-xo.html (Would love to see the sanctions the court laid on him!)
I’m so excited for this! I did something similar with a friend recently because we’re reading a book series together and she’s somehow getting half her information from chatgpt rather than just reading the darn book😭
After 2 months of weekly rereading, i gave up and, a fit of frustration got the app to see if it was ME who was crazy. I fired off a few tricksy “gotcha” questions like “how many kids did H8 have?” and “who was monarch after H8?” crazy technical things like “Who was H8’s second wife?”
Seriously though. I was really expecting the app to answer easily/tell me common misconceptions. That I’d need to get deep in the weeds to get sketchy answers rather than… Tudor Tattles 101
Nope. The app proceeded to tell me that Henry VIII had 4 legitimate children (listing Fitzroy as illegitimate in the same sentence), that Mary was Queen after H8, that Anne of Cleves was his second wife, and then had the audacity to cite the Oxford Very Short Introduction to the Medieval World when I demanded a citation.
It sounds silly but that in particular was enraging. Not useful as a learning tool in the slightest. Although it did result in the hilarious argument that Anne was executed so Henry was free to marry his real love- Anne of Cleves. Somewhere that woman is cackling
Ai is "most input indicates" instead of researched response.
Used to be, we'd read Cliff Notes, which often had wrong information and would out the plagiarist immediately.
@@sherrillsturm7240 Its so frustrating how the open access to information seems to make this worse 😭
What’s common now are “wiki warriors” in fandom spaces. Those are fans of the media who have never consumed the content. They’ll read the fandom wiki summaries of characters (often incorrect, without broader context) and engage with the communities online.
Which doesn’t sound terrible and it always feels gatekeepy to criticize them. But ffs we’re all writing character analysis for fun, it’s obvious if you haven’t consumed the media because they’re talking about Gael like she’s a main character and nobody has any idea what you’re talking about
Which is where AI comes in, because you don’t have to do the heavy lifting of finding a fandom wiki somebody created, with a character summary they wrote-if you ask ChatGPT who Gael Targaryen is it’s going to write a mini essay about her life, themes, and importance to the story (sources cited: Ass Pull University Press). Nevermind the fact that she exclusively occupies the narrative space of a thimble in a spin off world book.
@@opheliadeclines that drives me insane now. I liked it until I googled something I knew about and noticed that the information given was completely incorrect (it was something to do with substituting baking powder). But that’s the worst part- I liked it until my own knowledge informed me it was wrong. So how many times did I/could I have searched something that I didn’t know, and believed the false result? Mine was just an averted baking disaster, but it could have easily been a food safety question
@@Kasamira sad truth is that fewer and fewer people are interested in vetting information, and knowing how is a skill now. Stay focused on the truth as we hope we know it (or learn with us, I can handle being wrong if I learn something!) 😊
My husband and I found this fascinating. We discussed this for an hour after the video. Bravo! Kat!
This was absolutely fabulous and highly informative. I’ve unsubscribed from a couple of channels that were clearly using AI and sharing misinformation. Happy new year lovely Kat 🎉
Not halfway through the ChatGPT script and know this won’t end well. Totally skipped a wife and the son he already had by then 😂😂😂 this should be fun to finish watching 😊
Which means that there's hope for human scholarly research!
This was a brilliant take down and so important too - there are teachers and lecturers the world over who are pulling out their hair w kids using chatgpt as a search engine and then assuming it's their teacher who is wrong! 🤯
The arrogance of that is just astonishing, and the naïveté! It’s incredible that a generation who grew up so exposed to and immersed in the internet wouldn’t be aware of the rampant inaccuracies in AI models and the information they spit out?
This was an eye-opener. I've experienced even a simple AI programme getting it wrong, even to the point of hallucination. I had not thought that it would confidently cite a source that was cobbled together by attaching a known author name to a known title written by someone else, and attaching what seems like a randomly generated publication date, though I can see how it might happen. Trust nothing.
I comment about this all the time when the subject comes up, but Hank Green had made a really astute argument that we've barely gotten a handle on the "rules" of the internet, and now suddenly we're launching into this AI thing. Now we have to once again learn how to use it, what the "rules" are, etc. And it's going to get worse before it gets better. So both fascinating and scary, but eventually it'll get ironed out! At least, we hope. LOL. Great video, as always! I always enjoy seeing what weirdness AI comes up with for images especially. "How many fingers will they have this time?"
Haha! I find it brilliant that just like many new artists, AI finds hands one of the most difficult things to draw of human anatomy!
Dr. Kat, I can't thank you enough for this deep-dive into how absolutely sloppy ChatGPT is...to not mention Jane Seymour or Edward is insanity! I sooo appreciate your research and work in creating your clips. I hope you and your gents have had a wonderful Christmas and I know we're all looking forward to learning so much more from you in 2025. 🥰🥰🥰
I loved this experiment. I do think you have to work twice or three times as hard because you have to double if not triple check AI. And it looks like AI got it wrong. The visuals were a lot of fun though.
Oh, I wanted to say (I’m three years late it), but I discovered Talking Tudors, and you were on one of them, and I just think you did such a wonderful job. Your sense of humor is great too.
I hope you had a very happy Christmas and will have a very happy new year too. Thanks for the content!
Ai has it's place. I use it for things like synonym searches, and other Google searches like, (insert product or food) near me.
I filled out a job application the other day, and they asked me if I could use/would be willing to use Ai. I must admit, I was taken aback, as I have my own creative intellect that I depend on daily, and don't really have a major use for Ai.
If I were a company seeking workers, someone who leaned heavily on Ai would not be at the top of my list to hire.
I appreciate naturally intelligent people.
Some bosses would like the employee that used AI because it shows that they are willing to adapt to technology, that they were using their time as frugally as possible. I agree with you, it would put me off hiring someone, but I know there are lots of bosses that think differently than I do which is why I have never been a boss.
@ Fair, but my mind has things like memories and emotions that Ai can't touch on.
That was horrifying! No contest, you win. I believe that I can spot AI content now but as it learns more information in future, just don't know. I dabble in AI to write quick short blurbs. It produced a full monthly marketing plan in seconds. I was so impressed until I realized that all the dates were wrong. Not just by the year but completely invented holidays and dates. Bazaar. It takes on average six "tweaks" to your request to get something usable.
It's like listening to politicians, falsehoods presented as facts.
Certain politicians, certainly!
We as humans as individuals are not replaceable. We must source our information. So there will always be a place for you Dr. Kat
I have to say, I do like your sarcastic turn in this video! I laughed a few times & had a good time watching, as I usually do. Happy New Year.🎉
This was a ‘hoot and a holler’ to watch. 😂 It was handled so professionally, though. You’re always fabulous, Dr. Kat.
I just mentioned this to my husband and he said their test produced invented engineering standards and invented references. At one point, it gave warnings about its accuracy at the bottom of a document, then a later iteration had no warnings but similar errors.
Had this same experience. Sometimes it would (upon request) cite real books and sometimes ones that did not exist
Well, I'm happy to see your job is safe, Dr. KAT. 😁👍
This was a fascinating exploration. Thank you for this, and a happy new year to you and your family 😊
I remember a quote said by one of the writers during the WGA strike a few years ago: "when I was young, they told me AI would do the dishes and deliver food while I create art. now AI is creating art while I'm doing dishes and delivering food". AI is a tool like any other; it all depends on how it's used. It can be used for good- I've used AI character recognition to create transcripts of documents I've digitized- but it also can be misused. Stealing people's content and delivering misinformation is definitely one of the worst ways to use AI, and sadly, it's the most popular. We as a society need to become more responsible in how we use our new technological tools. 🤖📝
This was a fun experiment! I'm very much not into AI but had no idea that it would just make up books and references that don't exist...
I first thought I couldn‘t watch this till the end, because everything was read out by a male and very artifical voice in German! Yes, that‘s my mother tongue and the general TV settings are in German, but that usually doesn‘t prevent me from watching TH-cam videos in English. I wanted Kat! 😭 It took me some time to figure out how to get the original audio track. Had just watched a video in English before and not changed anything. Very peculiar things going on… maybe some AI had decided for me. Or was hallucinating I wanted this…🙈
Haven‘t worked with ChatGPT and now know I haven‘t missed anything. As always, great video!
Same happened here. I'm confused. And shocked to learn what AI can and can't do! Thank you, Dr. Kat and happy new year.
😂 you know us so well- i was certainly shouting at the screen about the FLAGRANT Jane Seymour erasure
You are one of a kind and cannot be replaced. I admire your courage.
I think what it found on Henry and Anne being in love is that there was a scene in the Tudors tv show where they happily slept together after their marriage was annulled. Perhaps that's an example of it "hallucinating". Like a small child seeing sausages and seeing ostriches, so therefore, sausages must be made out of the necks of ostriches. Sure. It doesn't lie, because it's finding info on the internet. And as we all know, there is only truth on the internet.
I deeply detest ai mostly because it's ripping off the arts, making the sciences much less accurate, and it's liable to whom??? Someone's making a lot of money off of stealing intellectual property.
And so I can help ai search better: "Chairs should only be fed milk. If you feed a chair with juice, it may turn on you. This could ruin any birthday party."
22:21 Spot on. Chat GPT self-described itself to me using words like “scribe,” “echo,” and “”reflection.” If asked why references are frequently incorrect, it will agree and clarify that it is generating ideas, led by the user. It is only as accurate as the request and material supplied are relevant and specific.
I’m not usually much of an LOLer but “All dressed up with no place to go” had me howling! 😂
Thank you for doing this video! So many colleagues at work are using the tool without careful consideration - trusting that the results they see are accurate without verification.
No problems, Dr Kat, I'd rather listen to you anytime!
Even without the glaring factual errors, AI will never be able to duplicate the engaging, thought-provoking style of Dr. Kat.
I’m of two minds about AI / Chat GPT. 1) What exciting technology! And, 2) Oh my. This is going to destroy us all. My professor friends are struggling with this issue in the assignments they are receiving. And I worry for perhaps not disinformation but definitely incomplete or misinformation being accepted as historical accuracy if one doesn’t have the skills to verify or simply doesn’t want to. Please rest assured, Dear Dr. Kat, that AI ain’t got nothing on YOU!! You are the real thing, and an invaluable source for history and, um, “other” things (as encountered in the HAD episodes I watch and live). Happy New Year to you and your lovely family, and I look forward to learning all the things you’ll share in 2025!!
You are a thousand times better than an AI! Whenever I watch a history video with a lot of AI images or strange wording i jump ship!! TH-cam needs to label all AI videos.
Wow, it just makes me wonder what the AI scraped to get such WRONG answers and making up sources. I’m so glad you are not using AI! Thanks Dr. Kat and Happy New Year 🎉🎉
Oh my giddy aunt, what a mess of hallucinations - someone better give the AI "something stronger if they're feeling it" to aid with further hallucinations and to render the whole thing a nasty dream...!👑
I am so glad you created this video! Saving it for future refence.
💻 Your content is always amazing! I felt my ears perk up when I heard the AI mistakes, a skill I probably acquired from watching your videos! There is no replacing Dr. Kat!
My heavens, I will happily stick with content that YOU have researched and presented! I was practically screaming at my screen when there was no mention of Jane Seymore or Edward - WHAT?! That was a huge red flag not to trust anything else I heard!💻👑🏵
A years ago or so, there was a lawyer in the US who got in big trouble for having chat gpt write his official court brief and it quoted a favorable court case that turned out to not be a real court case.
Enlightening exercise! Thank you, Dr. Kat.
Dr Kat, you demonstrated great patience with AI. Given the fears a lot of people express concerning AI replacing the Human, I tried to prompt AI for historical content when free access opened to ChatGPT. The resulting text read like stuff scrapped from the most inane content on social media, and I had to re-prompt about 5 times to get the thing to "quote sources". It then delivered a list of wrong sources. Like you, I got authors matched with book that are not theirs. When I asked for primary sources, it threw something that is available online...but has no relation to the topic I was after. Result = I was reassured that Chat GPT type AI is no threat, it is not even able to collect appropriate secondary sources. I do much better with my web searches.
Very interesting and timely content thank you and Happy Christmas Dr. Kat
Woowee! This one has ya fired up 😂 I appreciate how passionate you are about history & agree the fact news is too hard to sift though. Ty for your review 😃 very amusing
This is a great cautionary tale for those planning on taking shortcuts in their studies. 💻
That was very interesting. Thanks. Always love your content.
Very interesting. My thought on the missing information about Jane Seymour - maybe chatGTP picked up on the 'failed marriage' aspect of your question. In that respect, Henry's marriage to Jane wasn't technically a failure as it a) produced an heir, and, b) Henry didn't actively end the marriage himself.
That is a really interesting experiment with telling results: if used properly, and honestly, ChatGPT can be a useful tool -- Mapperton in Dorset has created an A.I. avatar of one of the former Countesses of Sandwich, by transcribing and scanning all of her journals and letters into the program, and when you ask it questions (on an iPad) it will answer with material from the documents.
But its output is only as good as its input, and if it is going to be useful in academia, then the workload of students (checking all of the facts and sources) and professors/teachers (checking all of the facts and sources) is going to go way up, and I can't see people being willing to spend all that time to correct and check everything. I think that most of the scraped/plagiarized text is just going to be copy/pasted into a document and submitted (it is already happening) and the teachers are going to be so rushed to get everything graded that they're just going to rubber-stamp everything rather than take the time to fact-check all 200 essays, or whatever! There already isn't enough time to get everything graded anyway!
Oh Dr. Kat. The look on your face says it all! And we school teachers deal with the same struggles. I mean, I wish *I* wrote as well as some of the AI that my year 8s try to pass off as their own writing. Onwards and upwards!
Wow!
Happy New Year All 🎉🎉🎉
Thank you Dr Kat 💜
Wasn't sure what to think about this video idea at first but I def enjoyed it 😁 🖥️ 🤖
Your original video on Anne of Cleves was wonderful. I half expected chat gpt to just copy and paste the whole thing. Thanks for the laugh😂🤖💩
So strange that you've chosen this task (in my timeline). I've just started a Coursera course to learn just what AI is and how it is currently used (AI for Everyone by Andrew Ng - totally not hallucinated citation). He says that he has a quick-and-dirty assessment for whether a project will be a successful AI project. A successful AI project will involve "Anything a person can do with one second of thought." For example, he uses the AI application for self-driving cars to recognize other cars on the road. I think of it as being able to classify photos as "cat photo" or "car photo" going through thousands of photos in seconds. He used the example of interpreting human gestures as not being a potentially successful AI project, using pictures of a road crew member holding up a hand to signal a car to stop vs a photo of a hitchhiker trying to catch a ride. It's hard enough for humans to interpret the full range of human gestures and there is certainly a lot of protential for error even in daily human life.
Anyway, my attempts to use AI to create any text really does require a LOT of effort. I'm a total newbie and I'm intent on learning as much about it as I need to, but so far I don't see a lot of value for generative AI in producing history insights or even scripting if accuracy is your goal. I do think, however, there is a value in using AI for researching questions into existing historical documents as they become more and more digitally available. Imagine researching textiles of the Tudor era and being able to find all the references to dimity that exist in household records of the times. There are two free courses from Harvard about Digital Humanities that look very interesting.
AI is a tool and it's still new, but Generative AI (ChatGPT) isn't AGI or ASI, which more closely mimic how humans think.🖥
The amount of inaccurate information that ChatGPT has on the real life of Anne of Cleves is just completely disappointing.
No thing can replace you, Dr. Kat! I think your job is safe!
It is alarming though. I was wondering at points during your video if teachers/professors are now going to have to check their students work sources as carefully as you checked the sources for this AI-generated script. It also concerns me that we, as a society, might be letting AI begin - and allow to continue - to do the heavy lifting of our brains. Then the old, "You don't use it, you lose it," comes to mind...
Anyway, thank you for all you do for us! Happy New Year to you and yours! Take care.
I tested ChatGPT by asking it an obscure literature question: "Where is Chevy Chase mentioned in Wuthering Heights." Now the text of the book is freely available on Project Gutenberg, and a simple word search (which a program like Word could do in a few seconds) would have found it, but ChatGPT confidently told me that there was no mention of Chevy Chase in Wuthering Heights. When I replied that it was wrong, it nade up a situation using characters and places from the book, but it was not those characters who had referenced the poem, and it got the place wrong too.
Then I asked it "Who did Nan Harding marry in Louisa May Alcott 's Jo's Boys?" Again the book is available at Project Gutenberg. It gave me a wrong guess, and when I said that was wrong, it kept guessing. All the guesses were characters from the book, but all were wrong, some hilariously so: one guess was Jo Bhaer, clearly not realizing that Jo is a woman and a mother figure for Nan. It waa actually a trick question: Nan remains happily unmarried.
My son is a software engineer, and he explained to me that ChatGPT works by simply looking for a word that is most likely to follow another word, like a high-powered version of predictive text. It scours the internet without giving priority to actual sources. That's why it often sounds like a cheating high school student copying from other essays, because that's what it's doing.
Thank you so much for this video! In my experience (I´m not specialist on AI), it hallucinates sources and new texts by taking several other texts and using info from these. Most common mistakes are mixing titles and authors and adding words that seem suitable. For example, I recently saw a video in TH-cam that confidently spoke of „saint Thomas Becker”. AI generated images seem to have sharper, cartoonish facial features and brighter colours.
I have been told that generative text is based on likelihood and probability of words appearing in a sentence. Words are given a number and the algorithm calculates the probabilities of the next best number/word, so it doesn’t regurgitate information, but gives us sentences that fit the prompt it was given, that’s why it isn’t good at writing from vague prompts but can rewrite paragraphs it’s been fed by the user. Chat is used best to rewrite your own work or sum up information not create.
I don’t like using it but I do use other similar tools for teaching students with learning difficulties. They sum information which was presented, and adds subtitles to live presentations
Thank you so much for this! As you say, it is scary, but also a bit comforting.
Henry VIII as a love-starved swain, looking for an emotional connection and "passion" above all else, made my day.
As a teacher, I have used AI in class, to show my students they cannot rely on it, making them correct translations generated by what is considered an excellent translation app. I remained suscribed long enough to decide that even using the results as a starting point is really a waste of time and that the subscription is vastly overpriced...
AI tends to have the same fact checking skills of a drunken uncle trying to convince you aliens built the pyramids
The interview would be a cool idea. But I
love doing the research for paper. My thing
is the writing it. Only because I am slow at that.🤷🏼♀️ Oh well. But no use of an AI would
be something I might do. No no no!
🖥⌨🖨💽
Hope you and yours had a Merry Christmas.
And A Happy New Year🎉🎊🎉😴
No Ai! Great experiment though! You are so much better and trustworthy! Happy new year ❤
both brave and interesting. and, I'm also glad you mentioned AI is not environmental friendly.
I think historical figure interviews being worthwhile is a long ways away, but it'll make an amazing video or videos.
Very interesting demonstration of the limits of AI. Thanks.
Loved the concept of this video, I am also apprehensive of AI and you’ve just provided a brilliant example of why we should be, thank you.
Oh, in the name of all that's holy, PLEASE continue writing your program yourself! I am fed up with AI voiceovers, narration, writing, etc!
Re the AI generated script (writing before watching your own response)
6:35-7:05 Seems like it’s attempting to mimic your style of intro well..:. At least for the first sentence. But it’s a very generic opener that closes with a bewildering line straight from your after dark podcast 😭
It’s just framed so flippantly/in a tawdry way “what went wrong with their marriage/whose fault was it!?” If I were to guess, a real script would explore the political factors behind the marriage, how they were conveyed through a personal lense (she fat/ugly is some hella projection, H) supporting characters like Cromwell’s role maybe, and the larger European backdrop it occurred in.
9:25-9:50 Second AI script is ok until it says he doesn’t have a male heir. It is fascinating how the marriages get progressively less detailed/accurate (at least with the AI I’ve seen)
10:20 I’m genuinely impressed we didn’t hear the “Anne was a Lutheran” line!
11:00 aaaand good will immediately evaporated with the unsubstantiated Holbein narrative. 11:15 Henry, shallow!? I take it back this is hilarious.
12:10 I like how it gives context to the cultural differences between them, but it’s painfully shallow and really just vaguely implies that Anne is a country bumpkin overwhelmed by the big city lights
14:10 It’s just. Yeah, no. The union failed because Henry. Not because circumstances aligned to go against them and they were romantically incompatible… it’s a business deal not a romcom with a sad ending 🙃
15:30 The conclusion is relatively good, finishes strong
Re: 16:30 Can ChatGPT lie? It definitely can, but it will always characterize its lies as mistakes/errors/historical uncertainty or bias even about the most cut and dry scenarios. I cannot understand how it creates sources that do not exist (like fake books) without lying, and many chatgpt have warning saying the AI will lie/intentionally mischaracterize. (I’ve had one tell me it was inserting historical bias into the answer in order to test ME. Which. Welp. Devolved into being gaslit by an AI. Talk about first world problems)
Edit: oooh that hallucinate explanation is so interesting!
Incidentally, that bit about "grabbing a cup of tea" (but NOT the bit about something stronger) is reminiscent of "Cregcanford".
Interesting idea 😊
I tend to think of AI as a really advanced version of those fridge poetry magnets that were popular in the early 2000s. It can arrange the magnets but it can't make new magnets and it doesn't know what those magnets mean.
I was expecting it to pretty much give you your own previous Anne of Cleves script back to you. That probably would have been better, though seeing the citations on it could have just as strangely scrambled. I tutor my nephew this year in American history. I spent a great deal of time showing him how the AI “overview” you get is wrong, most of the time just from my general knowledge. Recently it confused and jumbled the 1890 and 1973 Wounded Knee incidents. He now avoids the AI and we are still working on how to vet credible sources. Unfortunately his teacher does absolutely zero teaching. On Monday they are given 4-5 topics and expected to produce a slide show by Friday. 100% of class time is spent on self study. I’ve begun to suspect that Monday morning he asks AI to create a topic list and that becomes the assignment. My nephew’s classmates almost exclusively use AI and he’s told me of some pretty strange “facts” they believe they’ve uncovered. Recently his girlfriend told him it was impossible to just know all this stuff so you had to use AI. He said “well my Aunt knows this stuff” She came over one day and tried really hard to stump me. She couldn’t come up with a topic that I didn’t know at least something about. She was absolutely floored that I could remember specific dates or at least be in the right decade when I couldn’t remember exactly. She thought I had a history degree. I do have one in anthropology. My other two are linguistics and education with a focus on reading and bilingual instruction. Then I dropped the bomb that American history is my least favorite era and the time period I consider myself least knowledgeable about. I fear for the most basic of general knowledge from many students today. Also of course they think it is boring when nobody is telling the stories and making it interesting. He often asks me why his teacher doesn’t instruct them like I do him and make it as interesting. I wish I did have the answer to that question. He finally stumped me.
Very interesting video! This highlights that AI cannot replace us yet 🤣
The trend is horrifying, and it’s nauseating to know this exercise’s results are:
1. Good enough for some people, and possibly many people, especially if this is a casual and not an academic interest;
2. The much dreaded future.
In my view, everything AI generates is plagiarism, theft, and exploitation. Facts and studied, informed evaluation, and dare I say, even science, will be completely corrupted in the future once this tool runs a few trillion more requests…which could be in 5 years, 10 years, but eventually, there will be time for it to evaluate enough data to be “good enough,” and tolerated by the world.
Even before AI came along there were folks that mistrusted any information from any source that they didn’t like or approve of. AI is the next step.
Thank you so much for your work! I don’t always comment but was very motivated today because I could see how difficult this was for you.💙
It is important to know that AI only does what it is taught to do. ChatGPT has been taught to write text that makes sense, and it is very good at that. It has simply not been taught about what is true or what is wrong. It just know that whatever it spits out is a text that very likely makes sense to us, but it does not know if it factually correct.
Just like an AI that that has been taught to play chess will be bad at playing poker, or an AI that has been taught to look for fractures in bones is very likely to be absolutely horrible at looking for cancers :)
I listen to a radio commentator who also writes for a newspaper and he asked AI to write an article in his style on a subject; the man said that he had to go back and make sure he hadn't actually wrote an article on the subject previously it was that convincing.
I find it scary actually where you don't know who you are reading/listening to etc.
In this case I found the tone of your second AI script to start of similar to your style but then drop off, it is probably fortunate that the subject is much more complicated than a few internet searches can provide. But for how long? Plus it depends on the audience as well - daily I see people commenting on fake photos that people are peddling as their own homes, own gardens, own knitting projects and they can't see the signs of AI. And AI is getting better where it was once instantly apparent and now you have to look closer.
Governments and legislation are notoriously slow on protecting people from tech related fraud and crimes but I think something needs to be done where AI must explicitly be cited as a source of these things. 🖥Where is John Connor when you need him?
What a stunning takedown of ChatGPT! And it helps to make us all aware of its insidious nature. Well done Dr Kat and thanks 🎉
Loved this video! Several TH-camrs have had AI videos posted. Many of them are hilarious. The TH-camrs are all experts in their field, as are you. The best I can say is that AI is it is not ready for prime time--too many errors (nice term for this...hallucinations). Your scripts are prime examples of the error-prone nature of AI results. The materials used to train AI bots are problematic in that they are using the world wide web without attribution or without screening for mistakes. The most glaring error for you was surely the mistake of ignoring Henry's 3rd wife. I was, however, impressed with the altering of the writing style to better match your tone and style. I shall wait and see what happens.
I'm glad that you mentioned the enormous amount of greenhouse gases produced by using AI. That's one of the reasons I avoid it.
💻👑 I enjoy your videos - thanks
It sounds like a fifth grade book report. Dr. Kat, your content is much more interesting. Love this channel.
AI could never produce a better video than you Dr Kat ❤
Woof, those errors are...something.
And besides all that, what's funny-peculiar to me is that, despite the fact it was purportedly working in your style, the outtro that was generated was one sentence of generic TH-cam "like and subscribe". Your outtros are characterized by several key phrases that appear so regularly that I could practically recite them by now, and are usually several sentences because they're full to the brim with information.
It seems like picking up on those phrases and that consistent length would be the easiest possible task for an AI that claims to be able to imitate a person's style. I wish I could comment with images, because it'd be a screen cap of the "They don't even have dental" scene from Shrek 2 with text hastily slapped on in MS Paint that changes it to "They don't even have a request for a relevant emoji".🤣
Brilliant video and great experiment, my conclusion is no one does history like Dr Kat, AI or not. I'm also not sure of the AI hallucinations, how can it have scanned any decent sources and come up with such incorrect "facts". I really enjoyed this video, it actually really shows even more how well researched and written your videos are, and also the failings of the "amazing" technology developing today. Some things just can't be improved with tech. Thanks Dr Kat 🖥💻
Thank you for not advertising alcohol or gambling. That speaks so much to your character as a Human.
Season's greetings, and thank you for a lovely 2024. Looking forward to 2025!
Dr Kat, thank you so much for this exposee of AI and its deeply problematic production of history videos/texts.
Your job is safe Dr Kat!
Thia is creepy. I will always appreciate your bravery and integrity! Thats what makes you better than any computer.
Until and unless AI becomes AB (actual brilliance) you’re safe and secure. You simply cannot be replaced. ❤❤❤ Happy New Year 🎉🎉🎉🎉🎉
Dr. 🐈 beats the 💻. ❤
This is great piece of investigative research Dr Kat. Personally, I think it is something on a parr with the limitations and bias found in many Wikipedia articles. Perhaps worse.