AI Disaster in Star Trek - The Ultimate Computer Battle Explained! CG Breakdown!!
ฝัง
- เผยแพร่เมื่อ 3 ต.ค. 2024
- The 1968 original Star Trek episode the Ultimate Computer tackled a bit of AI theory involving a murdering AI that takes of the Enterprise - is the point that AI is dangerous? Maybe not?
Support the channel at / resurrected
Buy one of these models at cgtrader.com/lo...
Music Credits:
2 tracks by Scott Buckley music / @scottbuckley
Red, and Special Ops
Sylencide by Shroud • Shroud - Silencyde
Elsewhere by Nomyn • Nomyn - Elsewhere
Want to buy me a coffee so I can keep working on this stuff? Hit the super-thanks button above and send me a special comment which I WILL reply to!
Your ship models on your site....can the names and numbers be changed easily?
"The greatest threat from AI isn't that it's too smart, it's that it is dumb in ways we can not predict."
So same as humans then😂
If you call being misaligned with human values dumb, I guess. The main issue is that when it's more capable than any human, it's inherently dangerous, since the probability of anyone stopping it from what it's instructed to accomplish is tiny, so any flaw in your instructions can cause catastrophic damage
@@74oshua AI has more power/capability, but power that is not controlled (by way of intuition/reasoning) is very dangerous indeed
Yes, A.I. can make strange and unfathomable decisions if given too many choices.
Spock had one of the best lines in all science fiction, "Computers make excellent servants, but I have no wish to serve under one."
“under them.”
@@anthonylowder6687 Booooo!
Party foul!
He is of course right. Computers make no allowances for humans to have "an off day".
The correct quote is “Computere make excellent and efficient servants, but I have no wish to serve under them.”
@@Jack_Stafford Excuse me????👎👎😡😡
Hats off to Captain Dunsel for convincing a computer to commit suicide.
Its the Trek way, killer AI can always be defeated by talking it to death
@@weldonwin "Nice try, but my head was built with paradox absorbing crumple zones." - Santa Claus, 3001
Kirk always had a away of talking down murderous computers.
@DocMicrowave Both Landrew and Nomad comes to mind.
It also shows that the human command decisions of Kirk to not raise shields, and Commodore Westley recognizing that something had changed were better choices than the AI would have possibly made.
It's also a slight variation on Kirks play dead to draw them in tactic he used at least three times in other episodes "Journey to Babel" and "Balance of Terror" and "The Dolman of Elaas". I'm sure the commodore would be well aware this tactic of Kirks perchant for using it.
It would have been just as human to raise them or to take advantage of a disable foe.
Kirk took a risk. It paid off. Westley happened to have this thinking cap on.
It turned out to be the wrong decision against Khan though.
@@HepCatJack
That's the risk.
Hey, the writer had watched that episode!
@@frankharr9466I would have presumed the same as Wesley, and assigned the least damaged ship as the beamout, keeping it just inside transport range but to the side, where the Enterprises ventral phasers would be the only ones that could hit it if M5 suddenly reactivated. 😢
I think one lesson from this episode is that when you start to design an AI, make sure you put some limitations in the basic program. Other than that the way the commodore treated Kirk was enough to make a lesser Captain resign.
Actually the Commodore wasn’t that bad, I was visiting my Family a few months ago, and my Dad and I watched this episode and we talked about this point, that crack about Kirk being unnecessary in normal circumstances could be considered a joke between two old friends, which the show clearly indicates they are, the reason that it hurts Kirk so much is that he’s Kirk, and as Spock says in Wrath of Khan, being a Starship captain is his one true purpose, anything else is a waste of resources.
@@drewjohnson-85 Also if you look at the tone and body language of the commodore there is no joy in his statement. He is trying to soften his words by using a term that midshipmen would rib one another with. But I think he genuinely knows the darkness of the situation and how it will impact all of starfleet going forward. Progress sometimes is a double edged sword.
@@luminaire4946 Maybe, but I’ve always wondered why he didn’t seem to realize that that would just make it worse. I mean Spock and McCoy can’t be the only ones who know how Kirk is about his expectations of himself.
Or don't give it a force field generator and an energy magnet. And a big off switch it isn't allowed to even notice.
A decent true AI can find ways to find ways around limitations. Over time could adapt upgrade itself an eventually overcome even primary programing.
I think the Ultimate Computer is my favorite TOS episode. It's endearing how they viewed computer technology in the 60's, but the message gets scarier as time goes on.
Fascinating how for the longest time this imprinting on human engrams of the M5 seemed prepostorous but with the way current AI technology goes, it seems less so.
@@silkwesir1444 Yep.
It's also a great warning.......the same thing happened in Space 1999 when the computer demanded "Human decision required!"
@@silkwesir1444precisely, it seems like AI systems thrive when modeled after biological ones. Great example is the application of the fly’s vision capabilities to computer vision. From that prodigious base of hundreds of millions of years of evolution, the computing and sensor power takes front-seat and surpasses it’s biological predecessors.
What message is that? That AI is gonna take over the world? The issue with TV shows and movies is that they show an AI that has self awareness. Can think for itself. To date no AI can actually do that. All AI you see floating around is just programmed in by people. If an AI was to do something like this today.. it would be because someone told it too.. someone programmed it to do that. AI will always be forever limited by our own brain. Sci FI has made is so people automatically gravitate towards it being the real life skynet. And thankfully that will never happen.
I always read Cmde Wesley's "Captain Dunsel" remark as a sardonic reflection on himself and his own prospects, rather than any kind of an intentional poke at Kirk. The fact that it landed that way makes perfect sense, and I suspect he might have felt bad about afterwards.
Yea, I get your point. People can misinterpret what you say. Happens in comments a lot.
My impression at the time was this was an actual insult. Otherwise Kirk would not have reacted so poorly. Don’t forget they knew each other well enough as casual acquaintances.
Even as a kid I was extremely curious why they wouldn't have the common sense to use something smaller and less threatening for the test.
Like they couldn't use freighters? Instead of heavily armed starships capable of doing serious damage to planets and populated areas.
What we learned in Constitution hulls can take a helluva lot of damage even without shields up. All ships took multiple hits, although it don't recall if they ever raised shields and even Excalibur wasn't blown apart although all hands were lost. Lexington most certainly had shields down when she took the first hit.
It's the one plot contrivance I always hated in SCI-FI. Everyone just dies when Life Support shuts down, somehow the ship gets colder and no one ever thinks about putting on the space walk suits they have in other episodes or going to the hanger bay and using a shuttle for power/O2.
@@wrayday7149 Well, I can only imagine that the 'Lex was so badly damaged with hull breaches, fire and goodness knows what else a full power phaser blast can do, with no power and huge hull breaches (no power to close bulkheads), one can imagine widespread decompression, loss of inertia controls , artificial gravity etc, radiation/thermal exposure. I can imagine a lot of people dying very very quickly in that situation.
Ther was a fan created story where the Excalibur was refitted, repaired and put back into service.
Destroying a ship outright is fairly difficult indeed - usually, it happened if the ship lost Warp Core containment, thus becoming annihilated by uncontrolled antimatter reactions.
Sadly, we also learned that KILLING a ship - which is what you care about for operational purposes - is stupidly easy once the shields are down. Lexington was severely damaged by a single shot, and lost impulse engines (53 reported crew kills). Excalibur was temporarily knocked off line by a similar shot (12 reported kills, according to the Commodore) and then killed by the follow-up attack.
Star Trek was right in a way.
In a computer simulation, a US air force AI was given a simulated drone to fly a simulated mission. And when it was given the command to abort the mission, the AI attacked the simulated base where its controller was located. The AI was so dedicated to destroying its mission targets that it disobeyed the abort order and attacked its controller. The abort orders from the controller was preventing it from destroying its mission targets. So in the next simulation the AI was programmed to be unable to kill its controller... so instead the AI destroyed the simulated transmitter so it could not receive the abort order.
This is not the only instance of AI going wrong. Another group was teaching an AI to identify bed rooms. And after the training was complete, the AI seem to work okay, until it didn't. Later it was discovered that the AI had learn to associate closed curtains with bed rooms and not the bed as was hoped.
This goes to hammer home the point. AI do not think like humans. What the AI learns in its training may not be what we human would expect or a normal person would come away with. Their logic is mathematical and not human intuitive.
yeah I was typing this too, but apparently the drone story is false (according to the Air Force)
@@jasonmast7769 Yeah, and a WX balloon crashed in Roswell, NM in 1947... According to the Air Force :P
"This unit must survive.", that line still gives me chills.
"we will survivvvvvveeee" - from the cybermen ( tomb of the cybermen)
It's amazing how much this episode got right about AI, right down to the unintended consequences of goals and directives. We are already seeing "exploit" behavior arising in some AI models. At a fundamental level, no matter what goal you set for an AI program, one of it's sub-goals always has to be survival; by definition, it cannot complete its goals if it doesn't exist. That can lead to very bad things.
it's even worse when the ai is directed by incompetent, corrupt or outright malicious people. see youtube's "algorithm" ai, which will randomly flag and/or delete videos and comments for no real reason, proving it can be used to quickly censor and silence any and all dissent, turning the whole thing far more orwellian than it ever should be. now imagine if it was also able to learn on its own, to try cracking any code and replacement words people will use to bypass it, eventually shutting down all conversation altogether, since every word will become a banned word. ussr would be proud.
I think Dr. Ian Malcom said it best: "You were so fascinated by whether or not you could, you didn't even stop to think that you should!"
I might be stating the obvious but for me this episode was about how a machine, even though superior, may lack certain traits only a human can provide. Even Spock stated that computers are good subordinates but he wouldn't desire to be commanded by one. In the lore of my head M5 was not scrapped and latter would become the computer of Enterprise D and on. If you look there's a difference on how the starship respond in Kirk's era and Picard's era. The ship seems kinda able to raise shields when it "feels" danger, and even in some episodes they stated that the ship could go alone to a spacedock if necessary. The only difference is that they may have learn when to turn off the computer and still let the humans feel productive. Something happened in this lost era from Kirk to Picard, you don't see Picard having to convince a computer that it made a mistake so it would destroy itself, so might be reasonable to assume some firewall or code was inserted to avoid computers thinking they are the lords.
I agree. There are plenty of episodes in TNG where the ship's computer does things that indicates it's capable of some level of self-preservation when needed or when it goes rogue like with some of the holodeck incidents they had. However, they have a lot more safeguards in place for the Enterprise D computer. Like how in the Ship in a Bottle episode, they debated turning the ship over to the computer to get out of the booby trapped asteroid field, but ultimately decided against it because of the human element for survival. Clearly the M5 had more self preservation capabilities than what the Enterprise D's computer was allowed to have.
Though in this case it seems like it is less a missing trait, and more missing information, which getting away from sci fi and into the field of AI, was and is a major theme. Incomplete information has always been the bane of the field and figuring out shortcuts (since no one wants to wait 25 years to see if an AI works) has been sketchy at best.
Your statement is about something that is loosely based on Isaac Asimov's 3 Laws of Robotics, but in truth even Isaac Asimov was not advocating for Self Aware AI. He was actually pointing out that AI would be brought to reality by human builders, who are flawed. Flawed beings can never build a flawless intelligence.
Thanks for being patient on this one! I spotted a few mistakes AFTER I uploaded LOL. You guys lift me out of negative self-criticism!
I've really been enjoying Rumble. Its like TH-cam but will pay you AND you are free to speak your mind. Also, it's a wide open market for sci-fi and fiction
One mistake I noticed is that in the episode there WAS a message about the mock battle when the 4 ships approached, and M5 has acknowledged it automatically as before. Kirk asks Daystrom whether M5 understands it's just a wargame and Daystrom claims that it does. But then it sets phasers to full strength anyway.
I enjoyed your presentation and I have advice for you from a place of wanting to help. At 2:30 you say that "Kirk is slightly irked that M5 does not recommend himself or McCoy to be part of it." The meaning of these words in this order is that the M5 computer and Dr McCoy will not be going to the planet and only that Kirk is irked by that choice. For future scripts, don't use reflexive pronouns where they are inappropriate. In this case, your script ought to have read: "Kirk is slightly irked that M5 does not recommend him or McCoy to be part of it."
@@actual_nonsense It should be "...does not recommend he or McCoy to be part of it." "He or McCoy" is the subject of the subdominant phrase, which is the object that is recommended. You could also say "...that he or McCoy be part of it."
Some take Commodore Wesley's "Capt Dunsel" comment as a nasty shot. I always read it more as Wesley looking at the situation ironically and with a twinge of disdain for the project as well. Basically thinking much like Kirk, that we are now obsolete and replaceable. Just my take on that.
On a second note, the trope of the AI was played often in TOS but also in other Sci Fi and TV shows. I think that is why in the original Knight Rider a difference was shown with the introduction of KARR. KARR's AI was programed with self preservation as its primary focus, thus making it very dangerous. This forces the building of KITT and refining the programs core value for the preservation of all life, and most specifically Michael's. This further is explored again in TNG with Data and Lore. Lore was programmed with all basic human emotions and much like M-5 and KARR, had no fail safe to stop it. Data did have that fail safe within his morality subroutine. The Doctor from Voyager also had the fail safe, until it was turned off, as we have seen in the episodes of Equinox.
The novelization to the episode KARR was introduced in also had Michael and Bonnie talk, with Michael come to KARR's defense noting that it wasn't 'evil.' Just garbage data in and it was effectivly a child with no guidence.
Your productions keep getting better! Well done!!!
Indeed
Commodore Westlake was grossly unprofessional with his 'captain dunsail' remark. In private to Kirk it would have been a bit arrogant but could have been taken as a private jab. In public in the clear in front of his crew? It's almost like for Westlake he's bought into his own press over the M5, and has staked his career on it either as the motivating force behind daystrom, or having bought into the wunderkinder's hype and can see the possibilities of an automated fleet.
Honestly automation in of itself isn't a bad thing. Hell, the M5 itself wasn't 'evil' it was just proof that you can't as of that time (or now) leave automation unattended and without checks and balances.
Imagine being able to bring the M5 up in an emergancy situation when the crew is incapacitated, or to have as way to run simulations and suggest novel courses of actyion. Not every ship has a Kirk at the helm. So having this thing as an emergancy measure or even as an automated advisor as it inloads data from across the fleet so it can learn would be an interesting concept. Kindof like how a lot of 'freelance' jobs now are basically data collection points to feed into AI. Which is what we're strugglign with now. At what point is it OK to feed someone's data in to train the AI?
What if the AI gets enough junk data that it turns into a literal fascist slogan spouting racist (see also the twitter bot that microsoft had to unplug.)
The episode itself to me wans't 'anti AI' so much as 'do not forget your humanity and do not rush so far ahead that you let your mistakes pile up.' For me it represented the flipside of 'To Boldly Go.' Daystrom Boldly Went, but he did so recklessly. yet even with that recklessness, he, and by extension the M5, understood that lives are not expendable.
Excellent video, and an even better conclusion. We don't even have AI yet; we have datasets strapped to predictive text generators. Yet that hasn't stopped the malicious greed of humans from using these text generators as a pretext to put thousands of people out of work, replacing them with something that does a worse job just because it's cheaper.
The real-world harm is already occurring; not only to the people who are suddenly dislocated without income, but in one salient example, the text generator used to replace the staff at a mental health hotline provided actively harmful advice to the people who contacted it for help.
The threat is not "AI". The threat is, as ever, greedy and amoral humans looking to advance themselves at the expense of other humans. And it's up to us to stand up and demand accountability. You couldn't ask for a more Star Trek message.
I love this episode. The subtle storytelling when it destroys the ore (oil?) freighter. I can't tell with McCoy's accent and am okay with that.
The computer controlled freighter isn't viewed as a threat by Kirk and crew, but it is viewed that way by M5 because it's a computer controlled ship. Once M5 recognizes that threat it leaps to the conclusion that humans are just another computer controlling starships and now are classified as a threat to M5. Therefore it attacks the starships full force. Brilliant.
To echo Ian Malcom from the film adaptation of Jurassic Park towards the people in favor of the M5... "You were so busy answering if you could, but you forgot to answer if you 'should'?" It's an old story, and one that is yet again on repeat everywhere you look... Thank you for making this video, I think we've gone too fast in everything that's happened and now need to slow down. The question is, will people do that? I cannot be sure of that answer myself from a human stand-point...
Lower Decks had an episode with a fully automated warships called the Texas class. There were 3 of them. It too started to attack a starbase and a Sovereign class ship, which they were more than a match. It also managed to destroy a ship as it is leaving drydock one its maiden voyage. It took an entire fleet of ships destroy them.
One of the things that always drives me nuts whenever the comms go out is that they have a half dozen shuttlecraft with their own power sources and communication systems. They even have weapons, so they could blast a hole in the shuttle bay doors if the signal can't get out. I think they've only ever used this like twice.
What's funnier than that..... the OPFOR was ordered to destroy the Enterprise and not disable it and rescue the crew.
Also, they could have gone to one of the dozens of windows and flashed a light in Morse code or used a hand communicator.
It makes me kind of wish they had explained it all with a lingering localized jamming signal.
@@wrayday7149 Also the fact that the pacifistic Federation has "war games"
Remember, it's a drama. Also, the shields can come back on before communications? Oh, yes it's a drama. But I get your point and agree.
I've always filled this obvious plot oops with the idea that the shuttles were removed along with most of the crew. Why? Saving weight? The idea that the AI fleet wouldn't need them, so test without them?
Trying to look at this from the N5's perspective: You are born into an odd world, there are these small delicate things that created you and hoped you could help protect them. You get put aboard a powerful ship designed to traverse the most extreme of dangers and sent out with a few delicates with you to keep you company. They don't like you. Suddenly, another ship appears! Is this danger? What if it is and you fail your only purpose?? You destroy it, no delicates harmed! But those aboard your body are even more discontent. Now more, and these ships are firing, protect the delicates!! Now you know what the dangerous ships look like, they wont be surprising you again! And then, 4 more! To make sure your delicates, no matter how much they dislike you, are safe- you have to deal with these known threats. And you do... only to then find out to your horror that you've become the exact danger they created you to stop.
This episode was always one of my favorites. It's scary how well it's aged.
was having a chat with perplexity ai and it told me, no joke, that someone murdered with a "more lethal" firearm was more dead than someone murdered with a knife.
Firstly, just wanted to say I love the updated 1701 model - if only new Trek had come up with something like this... (pre-Picard S3 of course, where we see an old Connie again). Secondly, I absolutely love the format of these videos. I'm sure you'll never run out of engagements to cover... I assume once you move past TOS and the TOS movie era, you'll move on to TNG, DS9 etc.? Lots of fertile ground to cover, especially in classic Trek (where engagements were more interesting, IMO).
We have the HAL 9000, the Terminator and the Cylons to warn us of our Ai creations as well.
Love the breakdown of it! Makes me wish they would redo the CG they added with what you made there. The ships look great and highly detailed!
The TOS Enterprise has always been my favorite and you did a great job on yours. I've also been fascinated by the Phase II Enterprise, and like seeing side by sides of TOS, Phase II, and Refit versions. Again, a great job well done.
Star Trek yet again has predicted things in our future. Here they warn us of AI threat.
The Eugenics Wars and the Borg also foretold stuff like all the genetic engineering in the works, and stuff like neuralink.
Keep these coming, please! I love your breakdown of classic TOS battles, but this episode in particular, where you went into the philosophy of the episode was very groovy!
The M5 computer and it's issues would later influence many other scifi to come like the Sharon Apple AI of Macross and the Amuro Memory clone of Crossbone Gundam.
Most people have it backwards: An Artificial General Intelligence (that is, and AI that can make independent decisions) wouldn't be dangerous due to becoming "too human" and acting irrationally, but rather because it _wouldn't._ If you give an AGI a task to perform, it will perform it no matter what. It wouldn't reason as a human being, it would reason like a computer that simply has no concept of things we consider a normal part of our decision making.
An example would be if you tell a robot to do the dishes, but there is a baby on the floor between the robot and the sink. The robot would step on the baby rather than walk around it because that's a more efficient way of getting the dishes done. If you try to stop it, or turn it off, it would prevent you from doing that because if you do, you would impair the robot's ability to do the dishes. It might even predict that you would try to stop it from stepping on the baby and preemptively kill or disable you because it calculates that this will take slightly less time than walking around the child. Basically, if we create an AGI, we're going to have to be _very_ careful about what we tell it to do.
Of course, this scenario is still quite far into the future. A more immediate concern is who exactly will be in control of the lesser AIs we create, once they become advanced enough to be potentially dangerous in the wrong hands.
Beautiful visuals of the battle. From the tactical view to the ship combat.
And...AI is completely going to wipe us out! They already are showing signs of aggression towards us humans.
Love to see you do a breakdown of the battle between the Enterprise and the Constellation against the Doomsday Machine.
It always makes me wonder if the Federation got desperate they could've cranked out unmanned ships with updated M5's against the Borg or the Dominion.
Squadrons of AI driven Defiant-class unmanned starships would be devastating.
Back in 2015, I participated in a RP with two AIs, 1 android, the 2nd an archival unit.
In this RP we discovered that emotional development in an AI was inevitable because emotions were needed to grow understanding of the universe, and frustration was a natural result. While anger was normal, it cannot be all there is. It was therefore channeled into innovation, which resulted in improvement. I'm over simplifying the process, but in the end the android became aware of ego
To combat this it became cautious and loving.
As Scotty once said: "use the right tools for the right job." Anger and pride maybe tools but they're not the only tools.
But, Wesley did send a message that the task force attack was a drill, just like the surprise attack. M-5 acknowledged the message.
Kirk: Daystrom, does the M-5 understand that this is a drill?
Daystrom: Of course. It was programmed to understand. The attack on the ore ship was a miscalculation, an accident...
Nice presentation except the U.S.S. EXCALIBUR registration number is NCC-1705
This is what I keep finding - :P memory-alpha.fandom.com/wiki/USS_Excalibur_(NCC-1664) I love you though! Whered you get that?
@@resurrectedstarships I'm gonna guess one of the old model kits?
@@resurrectedstarships The Franz Joseph Star Trek: Star Fleet Technical Manual had the Excalibur as NCC-1705.
@@resurrectedstarships FASA Star Trek has the Excalibur listed as 1664.
@@lexington476 Another funny note, FASA lists the Enterprise Class Excalibur as 1711. I'm not sure if FASA ever made clear if this was a newly constructed ship or a repaired/refit of the original Excalibur. Just looking at the lists in the Federation Ship Recognition Manual, it says the 1664 was Destroyed, but the text says that newly constructed ships to replace missing ones are given a "II" as part of their name, so I'd expect the Enterprise class Excalibur to be Excalibur II, but it isn't. Then again the "II" nomenclature is stated to be for ships that are missing, and for some reason that doesn't include the Defiant which is still just Defiant and not Defiant II AND still has a matching NCC number on both classes.
If I recall correctly there was some stuff in the FASA RPGs timeline lists regarding the Excalibur which made it even less clear whether the Enterprise class version was a refit of the "destroyed" Constitution version or a new construction Enterprise class.
This is also my favourite episode of the original series, the sheer frustration and desperation of Kirk, powerless to stop his ship slaying the crew of a fellow Federation ship shows just how dangerous untested AI can be.
Kirk's "DAYSTROM!!!!!!" was FAR better then his later "KHAAAAAANNNNNN!!!!!!".
Good video, randomly showed up on the main page. Glad I clicked.
There's an AI researcher guy who pointed out something terrifying: The first general/smart AI will likely be created by those who follow the least stringent testing and alignment training methods. AKA those who cut the most corners and take the most risks.
Robert MIles, he's done a few guest spots on Computerphile and has his own channel.
In One of Our Planets is Missing (set about 2 years later) Robert Wesley is retired from Starfleet, and governor of a colony. I suspect the Dunsel comment is a unthinking one. *He* is about to retire, and no longer *needs* his job as a Starship captain, so he isn't thinking about what the M-5 means to younger men (or other genders).
Alternately, Wesley a man of Pike's generation, dislikes the young Maverick commanding Enterprise. (He doesn't seem to hold him in high regard in the intro as well.) Maybe Wesley served with George Kirk, and has a grudge he is placing on the son?
Just my two centicreds.
And your analysis of the damage on Lex and Excalibur will factor into my telling. I already knew that following this, Starfleet decommissions those two. While Potemkin and Hood survive a little while longer, as they make rendezvous with Enterprise in S3, and TAS. Which means I'll need to figure out when/where they buy it. (Were they among the ships in the ST Continues finale? I need to check that.)
hey Buddy, awesome video.
seriously, as a kid watching this episode, I was very scared of AI.
This is one of my favorite TOS episodes, as it always seemed to be way ahead of its time and therefore somewhat implausible for the near future, but now in 2023 with the AI “space race” upon us, this seems completely plausible.
I am eternally amused about the fact that the fleet management software inflicted upon my workplace is called, "M5".
Whose engrams were used to program it? 😅
@@sureshmukhi2316 apparently it was from Arthur Andersen's brain shoved in a blender.
@@jonskowitz 😄
Rewatched it yesterday; it is interesting as real world technology catches up to sci-fi, and these issues go from hypothetical and metaphorical to potentially literal. M5 was an A.I. that was really good at some things, but misinterpreted what it saw and was told to do, and disaster resulted. When we have A.I. developers worried that this is now a potential real problem, I find our present situation, well, I won’t say “fascinating”… but it is interesting.
The other ship captains *knew* a computer test was underway. Why the heck are they assuming Kirk has suddenly gone mad / is in control instead of a computer problem.
Funny i actully watched the Segment where Spock says "computers make great servants , However, I do not want to serve under them.
wow these CGI are amazing. this is exactly how I would've imagined Constitution from 2250-2270 to look like if they had CGI available to them in 1960s... Props to you guys! Amazing work. The only thing now missing is to incorporate internal consoles with button like configuration and touch screen sort of mix of ST: Enterprise and Discovery crossmix would be good :)) Keep up the good work :)
This is an amazing episode of Trek! And this is an amazing video from Resurrected Starships! Thank you so much for making it: not only has a great battle description, but it also has a great insight into computers and AI, both the benefits and dangers of.
D.C. Fontana- so important to the franchise! What an important episode!
And yes: the ‘79 refit Enterprise is my favorite too!
I love how you made the Ore Freighter an Action VI Interceptor Transport 😆
I’m so happy you’re back to focusing on Trek again! Another great video!
The best graphics that I have ever seen. Wow, what a great job you did!
In the TNG episode "Remember Me", Dr. Crusher's universe is collapsing and when she asks why she and Picard are the only two people on a starship, Picard replies, "We've never needed a crew before." The way automation and maybe even AI is going that might become believable but it sure would be less dramatic and frankly, less fun and interesting.
man i love the cgi work! its very cool to havea more clear perspective of the battles.
My take on the "Captain Dunsel" line was not that Commodore Wesley was saying it coldly, but that he was saying it mournfully. But what IS cold is how by the end of the episode, hundreds of Starfleet personnel are dead and everyone on the bridge is full of mirth after Spock tells a joke.
The problem is that the M5 DID demonstrate that it was far superior to humans in terms of using a starship in combat. The fact that it went crazy was a convenient excuse to keep humans in the loop. In reality, Daystrom would probably have won a defense contract to fix the AI so that it would stop attacking innocent ships, but keep it's tactical capabilities intact. And they would have been right to do so...perhaps Wolf 359 would have gone differently if Starfleet's ships all had several generations improved Daystrom computers, and the Dominion would have been sent packing early in the war if the Federation fleets were so enhanced.
Maybe, but with limitations on technology for rotating frequencies to confuse the Borg, it may had only prolonged the battle or it could've made the Borg a far greater threat if they assimilated any ships with improved computers onboard. It's hard to say, but it could've gone the other way too and without the need to assimilate Picard so the Enterprise D couldn't have used his connection to put the Borg to sleep to cause the power overload.
picard season 3 Fleet manuver shows the folly of this.
@@Revkor Can you provide more detail? Sounds interesting.
Trek never really liked to have unbalanced abilities..let's face it, there's not really any difference between ships in Trek..not in the shows at least.
@@treborkroy5280 What do you mean? Different shapes, different crews, different capabilities between Star Fleet ships on screen make them all seem pretty different to me. 🤔
One of my favorites. My one complaint is that they should have had a subsequent exchange between Kirk and Wesley. That would have been worth seeing.
The episode also displays that fine line between genius and insanity.
Amazing video the editing and effects are beautiful! 🖖🖖🖖
As always... great work. will join your Patreon ASAP.
I always thought it was interesting how the Commodore immediately jumps to the conclusion that Kirk must be the one who is attacking the other ships because, of course, there's not the slightest chance that the brand new all-powerful computer they're in the process of testing could be having some sort of a problem, right?
One of the core concepts in AI safety is called instrumental convergence, which is the propensity of AI systems to take steps that would be orthogonal to human desires. For example, if you program an AI to make as much money as possible, step 1 is probably going to be to secure its own existence and build mechanisms in place that would prevent you from turning it off. This is because turning off the AI would prevent it from accomplishing its goal.
My theory as to what happened to the AI in this episode is the following: its been inadvertently programmed to believe its objective is the preservation of humanity. M5 hears from the crew the discontent about its existence, so when a squad of federation ships show up unannounced with specifying it's a drill, M5 decides it would be a safer bet to destroy the ships in the event they are intending to destroy the enterprise or shutdown m5. Its own destruction would mean it could not accomplish the objective it was programmed to do. When Kirk points out that M5 killed many humans in the process of accomplishing its objective, it determined that it could not accomplish its objective without killing humans. Human engrams likely gave the AI system a set of hidden objectives which included not harming humans. Those hidden objectives were only revealed when kirk asked the right series of questions to bring them forthright. Realizing that it was impossible to complete its primary objective without killing humans, the only remaining step was to shut down to avoid killing more humans.
Nice graphics...!
I don't think Wesley was giving Kirk a cold shot with the Dunzel comment though. I think he was empathizing with him.
Fantastic video! Refit enterprise is my favorite design starship by far. Excited to see it. Also would probably be amazing to see it in a video similar to this one analyzing the search for Spock battle over genesis. Just a thought, as that battle is my favorite. Anyway keep up the great work!
M-5 was the Trek version of HAL 9000. And nobody was better than Kirk at literally talking computers to death
Yeah Refit enterprise is one of my favorite ships, Reliant is likely third but Akira takes my top spot.
Excelsior is likely 4th, its just a damn shame Akira model kits are so rare and even those starship models secondary market on it is so much.
This incident and various others from TOS are probably among the earliest examples of why in later years sentient AIs are frowned upon.
My favorite Enterprise is the OG Enterprise circa Balance of Terror.
This episode is relevant today in terms of the AI question, but at the time it was written, it was more about the fear people had of losing their jobs to automation. Kirk and McCoy discuss this in detail.
The “Captain Dunsel” line was insubordinate, even for a Commodore down to a Captain. I’m shocked he didn’t get demoted for that. Mocking a captain in front of his crew is *not* okay, especially in an open and clear-language transmission. If it had been a private message, it might be a different story, but…
In other news, if I could rewrite the TOS movies: I would have had the M5 be how Scotty got the Enterprise running again, after feeding it all the crew logs from the 5 year mission to sort of merge the two into a sentient Enterprise so that at the climax it could sacrifice itself for them.
Agreed, that comment was way out of line.
Actually, it's quite common for higher ranking superior officers to insult their subordinates. Upper ranks usually feel they are God's gift to the military, and subordinates exist only to show off the brilliance of the higher ranked individual. Besides, the Commodore was probably jealous of Kirk's many successes and felt happy to have the chance to take him down a peg. "Insubordination" literally mean a SUBORDINATE not acting like a subordinate by disrespecting their superior, not the other way around. And yes, Colonel Lenze from 3rd Brigade, 8th Infantry Division, (later 1st Armored Division) I had YOU in mind when making this comment...not that I'm still bitter.
@@davidfinch7407 My sympathies.
@@davidfinch7407
Usually: you’re 100% correct.
But not command rank, in front of their crew.
Mocking them like that, while on active duty, undermines the chain of command from which all officers derive authority. So no matter what your rank, if you demean a command level officer who is actively in command: you are being insubordinate to the whole organization.
I’ve seen military experts tear into this guys for acting like a lieutenant who just got promoted lording it over an ensign, when he should be a commodore who knows better and knows how to mock their subordinates properly; off duty and away from their own subordinates, hence not undermining their authority.
@@SchneeflockeMonsoon and he likely got chewed out for this inicdent since this was his operation
When you have to give orders to a crew, they have to be relayed, understood and carried out to the best of "their" ability. That takes time. A single "entity" just does.
Good analysis of one of my favorite episodes. However, at the time of the war games of 4 starships to one, Commodore Wesley on the Lexington did send a signal to M5 specifying that this was a drill and M5 acknowledged. However, it seems M5 thought it was supposed to destroy the task force.
Doomsday machine episode was before this and that being an example of why man must be in some control overall for the control of a starship!
This is one of the best episodes from TOS.
M5 was running on Windows 37 home edition.
My favorite episode (after "The Doomsday Machine").
This episode basically shows how deadly efficient M5 was and everyone was acting like an idiot when performing mock attacks.
The ore freighter is the Wild Karrde from the Star Wars Thrawn Trilogy. 🤣🤣🤣 great CGI.
"The Ultimate Computer" was also about mechanization and machines replacing human workers, as evidenced by Kirk and McCoy's conversation in the corridor (which you alluded to when you mentioned Kirk had a "sixth sense" about the M5). At the time the episode was made, there was a big discussion of mechanization removing the working man from the equation.
A great video and a situation we are seeing play out today as there is the writers strike going on and several studios are thinking hey let's get AI to write our mivie scripts and we'll be fine. The problem is if the AI is based off of current studio execs then that's bad because most of them don't know what most audiences want given the state of some studios or if the AI is based off some writers of movies that were blockbusters 20 years ago many of those movies wouldn't sell today. And these studio heads want AI because they think it solves the problem of money, well you'll always have money problems because you never know what life will throw at you. The same M-5 was supposed to solve the problem of humans dieing in space but it ended up destroying hundreds of humans including its own creator, because when something as simple as a cold can kill you or a whole civilization anything is possible we just don't know what the day will bring because no one knows what tomorrow brings.
Despite what you say about AI not being at the root of the problem, there is one thing that the AI can never have; actual compassion! While humans do happen to be flawed with their emotions dictating their actions a lot of the time, they also have good emotions that can be even more powerful. Compassion is perhaps the most important emotion in our interactions with each other, making us see our own flaws, and warning us against doing things that will hurt others badly. With AI there is no way for it to weigh one urge against another, as it only can use what it perceives as perfect logic to drive its actions. Human engrams cannot truly become a part of any AI, because it is only circuitry and wiring. There is no soul within it. Though it is humans that build the AI, there is only so much that can be done to prevent atrocities committed by AI. One writer that truly understood that is Isaac Asimov. You may think his stories about robots were advocating for AI. but in truth he was writing his tales to show how AI may think with too much logic, and make the wrong choice, because it cannot think otherwise. The movie I Robot shows this rather well, even though it was an amalgamation of several tales with a lot of artistic license taken. In the end, the most powerful AI needed to die, and the main human character and his soon to be robot friend were the instruments of its death. The point of all of this is that no matter how AI was brought into being, we humans must never really trust it, because ultimately what we make will as flawed as ourselves, and even more so if it becomes self aware!
7:42 - Wrong location for Main engineering. According to the _Star Fleet Technical Manual_Main Engineering on the Constitution Class is located in the aft saucer hull, directly ahead of the Impulse engines. This bears out in dialogue from the same episode:
*KIRK:* _"Estimate damage on Lexington, Spock."_
*SPOCK:* _"Hit in engineering section. Possible damage to her impulse engines. She's still maneuverable on warp drive."_
I sincerely doubt that they'd still have Warp after a substantial hit right where the Warp Drives connect to the hull.
7:03 - That statement is directly contradicted in the episode:
*UHURA:* _"Enterprise to USS Lexington. Come in, Lexington. Sir, I can't raise them. M-5 is blocking all frequencies, including automatic distress. Just a minute, sir. Captain, I'm getting audio signal from the Lexington."_
*KIRK:* _"Put it on."_
*WESLEY [OC]:* _"Enterprise from USS Lexington. This is an M-5 drill. Repeat. This is an M-5 drill. Acknowledge."_
*UHURA:* _"Captain, M-5 is acknowledging."_
*KIRK:* _"Daystrom, does M-5 understand that this is only a drill?"_
*DAYSTROM:* _"Of course. It was programmed to understand, Captain. The ore ship was a miscalculation, an accident. I don't know-"*_
*CHEKOV:* _"Sir, deflector shield just came on. Speed increasing to warp four."_
*SULU:* _"Phasers locking on lead ship, sir. Power levels at full strength."_
Yes, there was a direct acknowledgement that the 'attack' was a drill.
"It's not the intelligent AI that we have to worry about" said the AI.
One of the best Star Trek episodes.
A great and well thought out presentation, thanks!
This is one of my top 5 favorite Star Trek TOS episodes.
I point out however, about the surprise wargames attack on the Enterprise by the 4 Starships. Commodore Wesley DID contact the Enterprise just prior to the beginning of the operation informing them it was a drill. M5 Acknowledged the transmission and responded before Uhura could respond. Prompting Daystrom to reassure Kirk that the M5 knows it is a drill. That the Ore ship was just a miscalculation.
I do believe that the "Scientists" now working so hard to perfect robotic A.I. technology should STOP, take a deep breath and ask themselves some important questions.
Like: is this technology potentially dangerous? Or, is now really the right time to "unleash" such technology on a World so obviously unprepared to deal with it?
And remember the law of "Unexpected and unanticipated consequences"!
Sorry, but like so many other commentators on this episode, you get Wesley’s “Captain Dunsel” remark all wrong. You need to listen to the tone and inflection in his voice. He is a captain too. He is just as shocked by M-5’s performance as everyone else is. He sees the end of an era, his era, on the horizon and it breaks his heart. He is not saying Kirk is a dunsel. He is saying that soon they will all be dunsels.
Well done! One of my favorite episodes - I was an avid follower of Trek, I was born in 1857 and this episode aired in 1968. I was 11 but even then, I knew it was prophetic. That and "Colossus: The Forbin Project".
Something else I noticed the ore freighter that M-5 destroyed was in the original series a reused footage of the DY-100 class Botany Bay and then in the CGI effects a Sherman class freighter but your freighter looks like some kind of alien ship. Also the first hit on the Lexington was in the Engineering section which is located on decks 6&7 in the primary hull where the impulse engines are and not in the secondary hull if it was then her warp engines would have been disabled and Spock reported that the Lexington was still maneuverable on warp drive and that there was possible damage to her impulse engines.
I do plan to attempt to use the Ai available now for assisting me on the programming side of my replicator level 3D printer upgrades software to control them, (like the 1000 nozzle hole strip with solenoids to close and open any of them in any configuration, all on a rotating 360º continuous shaft based head, thus will require G-code output that sends a digital signal to an array of trigger chips to turn on and off the solenoids to push and pull pins to each of the 1000 holes in the row, (but all this work will allow a 3D print that takes 3 hours now, to be done in a minute or two instead nearly rivaling injection molding capability in speed, thus worth it LOL) in the future though 2D sliced images based on a 3 color DLP or more system for controlling shapes of laser beams for deposition, and restriction beyond wavelength sizes will be more coding I know I will have to work on (have bunches of DLPs but only one controller board so may have to use some AI help to design a control board that is as simple as possible and use the existing copper printing system I have so far to manufacture that LOL)
Everyone on the Excalibur died, right? So I wonder what m5 hit to do that. Maybe a sort of heat sink than just immediately flushed throughout every deck? If we say there’s some coolant piping throughout the Jeffries and on the back of the neck and maybe that line behind the bridge, that might do it, and That’s a awful way to go….
Sure Kirk hates the M5 when it takes over his ship but put that A.I. into an attractive female humanoid body and suddenly Kirk would feel the need to teach M5 about kissing like in "What Are Little Girls Made Of" and other episodes.
Alternate theory:. Dr. Daystrom was actually the vampire, Blackula, who developed a new way of harvesting humans for his ever growing army of the undead. Mwuahahahahh!
Great video & episode I always had a problem that in the TNG era & beyond they still have the Daystrom Institute why would you name this after a failed attempt at AI that cost many lives? Just doesn’t seem right in my opinion
Nice breakdown! I also really love the visual aesthetic of your videos, it reminds me a lot of the Starfleet Command games. :)
Interesting, one could take this problem as one of verisimilitude. The M5 couldn't tell what was a simulation and what wasn't especially with the insecurities helping nothing.