This 100%. The more descriptive you are, the better the results. LLM is going to be as good as its understanding of the task, if you give it basic input you'll get basic output.
My take on this is that LLM powered tools work far better as collaborative tools than automation tools. You don't want them to do stuff for you, you want them to work with you to automate and search in the small and you provide immediate and continuous feedback to direct the process. It will get things wrong, it will guess wrong. If you stay in and close to the generation though, it's like having a bunch of high school aged interns at your disposal.
Thanks for sharing like this Sam... ngl - I often get suspicious of motivations when content creators disclose a companies involvement... but this was super compelling.
Thank you. I honestly thought the video would end on the low note in the middle. But watching it make that flow chart for me changed my view completely
Sam, in essence, your initial prompt lacks depth, akin to asking a physicist for lunar travel advice and expecting a simple solution. It's crucial to refine prompts to avoid overwhelming the model with endless possibilities. Remember, the model's outputs are probabilistic, sensitive to initial conditions, and can become chaotic with higher temperature settings. This is exacerbated when we use the models own outputs as later inputs. Regardless of the rules it has it will screw up if the initial prompt is so simplistic that its combinatorial explosion potential is very high. Your prompt provided minimal context for generating extensive content, you are begging for errors. Professionals would offer more detailed prompts, not simplistic requests. This app and ones like it assume some baselines such as enhancing expertise, not catering to beginners or oversimplified queries.
@@davidoffberlin I am not an AI no... what idiot would set their Ai off on a game of randomly commenting on videos about AI? That sounds a little like a waste of tokens to me, cheap as they are these days. Ohh look I also used the words 'It is crucial to ...' in there too... and other words from... ummm the lexicon. What do you think David? am I an AI or just educated and therefore easily confused with an AI? Do I use AI... ohh yeah you could say that yes... maybe a good argument could be made that my personal lexicon after several years looking at outputs has been contaminated. I will agree thats possible. But generally no I'm not AI, I am not artificial. Having said that its an odd term AI... we do not after all say that an aircraft is artificially flying... its really flying!
Is it me or does the diagram at the end look wrong? Eg ACIT 2811 appears in the "with prerequisites" when it doesn't have any (according to the diagram). Compared to the earlier diagram, the dependency between ACIT 2811 and COMP 2311 seems backwards. This seems to be the problem with a lot of this sort of stuff, it seems superficially "wow" but on closer inspection isn't actually right, isn't correctable in a reproducible way (ie fixing your scraping or graphing code) and ends up making the user a proof-reader and fixer, a process that is only marginally less work than doing it all manually in the first place.
In what formats can you then export it?... - What manual modifications can you do after the initial creations?... Can you re-import an export that has been modified (in a correct way of course)?...
5:47 personally I always build the data structure, or at least the first draft, for real first. Once you have that, and it's easy, there are numerous tools out there that'll generate the ERD perfectly for you and, if you're using something like SQL Server and know your way around the system tables, you can use the results of your create table/index/foreignKey/whatever scripts to generate your data dictionary. This is the fastest way to develop and document data structures in my opinion. Tools like Visio can even connect directly to your database and create an ERD in seconds... I've pretty much given up on AI for now. There's not enough value added to justify the overhead and I have scripts to automate most things I'd ask an AI to.
woah! super lucky the algorithm gave me this video...just started making some state machines for my game and was having trouble coming up with the flow chart...this tool made it wasy easier.
Really been enjoying your content Sam. Your tweet regarding this is brilliantly summed. We do the critical thinking and decision making and LLMs do the mundane tasks. Couldn’t agree more.
Thanks for watching my content, I appreciate the comments too. Let me know if you end up using these tools and find out something interesting that I missed
Unbelievable. This is my second video of yours and I am just starting to get invested in your content, and I find out that you're a program head at my school!?!?! I am just starting term 3 of the CST diploma and I am in the AI option. What are the odds? I cant wait to keep watching your videos and if you're ever at the Burnaby campus I would be beyond thrilled to meet you.
I think model is being rendered in real time, not being called multiple times. But this is totally mind blow. And very very handy! Thanks for introducing me to this tool. 🙏
I had a play with eraser and found the functionality that was the most limiting was that after having a flow graph generated and making a couple of edits there were a small number of items a matter of presenation that I wanted to rectifiy manually rather than consume free tokens to alter. However this thought chain proved that this system is flawed as there was no obvious way to manually edit flows in the ‘normal’ way after generating a flow.
I gave it a very simple prompt of some components and how they link to each other in a software architecture. It gave me only two boxes, with the same text that was none of the elements I mentioned...
I feel like a lot of people react to the first 2 mins or so of the video where things fail. But Sam demonstrates how to use this properly in the second half of the video. And that it is useful if you don't leave your brain at the door but ask it to produce narrow and focusdd outcomes which can be mundane and boring to do manually. I believe the challenge is that companies are pressured to announce things like "we solved fusion with AI" then people use the product and go "well this is more like: explain what fusion means" and can't really appreciate that it's a great tool to do tasks a bit better because they we sold the dream
Of course you need to give it a context prompt. So that knows excatly what to do for you. You tell a designer give me a diagram for an X feature - guess what - that is the 1st diagram you get. Prompt details are eseential - you are giving instruction to your team.
I still don't know how I feel about all of this. I would bet that a large number of companies would not approve of pasting schemas into a 3rd party app just to generate a diagram. How do they use that data? Do they then train their models on it to improve? For small projects this seems okay but larger ones it just doesn't sound like a good idea using.
Nice video Sam! This has changed the way I think about using AI tools. This tool itself seems pretty neat. That default of string for primary key is kinda horrifying though lol
Maybe I’m missing the point as I haven’t done a comparison, but why are people using Eraser when it’s a clone of Excalidraw which is free and open source (and includes this AI functionality)
HI mate, since your AWS videos are little outdated, would like to request react node file upload mini project that use AWS S3 and Lambda function and DynamoDB.
Exactly! Pretending that AI creates new possibilities and visualizations by turning an SQL script of an existing database into an ERD is just nonsense.Even database tools that have been around for more than 10 years can do that for you.
Anyone have advice for an AI that can return valid SQL from a prompt and deliver it programmatically to a backend that produces a front end result from running the SQL.
@@SamMeechWard fair enough and I agree and I have thought how to keep it quarantined from the server and pre validate it. But really think if done correctly a lot could be done with it in my field. But thanks for the reply :)
Yes, but they are specific to the database or sql dialect. There's not tool that can take any sql code, or orm code, or object oriented data model, and create an ERD. This is general purpose for all languages and paradigms
Good answer by Sam. My big plus with such an AI tool is being able to 'talk' to my ERD. Otherwise I'd have to change the ERD myself. In Architecture, the older tools can be used for the as-is diagrams, and the AI tools for designs you'd want to modify by natural language
really? you gave the thing the vaguest of descriptions, then based your criticism on that. would a human do exactly what you had in mind with this kind of instruction without some kind of follow up ? (yes AI has a lot of shortcomings which I was looking forward to seeing in a video like this, or things like can i iteratively update the diagrams with chat etc)
Show me a single tool that can take any database schema or data model from any language or framework in any format and produce an erd We’re not talking about the same thing
This seems like such a bad use case for AI. Even if it did work, and it did create diagrams that makes sense, how much work does it actually save? I dunno, maybe I'm just an old head who doesn't see the value in having an AI that helps with something that takes 0.1% of your work.....
2:56 ok you just asked him to create a diagram for a e-commerce site! That what he did. You didn’t gave him any context. So the only thing you can complain about is this application hasn’t been design to ask you questions in order to better understand your intent. And this has nothing to do with the LLM himself. We are mid 2024 and people on TH-cam reviewing AI application still making the same mistake: a AI function in an application is not just the LLM, or even just one LLM. Between your input the generated output, here a diagram there are many components involved which are not an LLM! So in short it’s a split responsibility: you didn’t provided the app any context of exactly what you was expecting and the app is not doing mind reading so should have been designed to ask you more questions in order to fine tune the context of your app.
Yes, but they are specific to the database or sql dialect or using the specific IDE. What if I use a brand new programming language and a brand new orm that was just released today? No ide is going to be able to take my code and turn it into a diagram. This LLM is general purpose for all languages and paradigms
This video shows exactly whats wrong with AI, the AI diagram is cool and all, but its wrong and you dont even notice it. For example, ACIT 2811 has ACIT 1620 as a prerequisite, but its not even listed there.
Sam: Gives the vaguest description to the AI for what he wants
AI: Gives vague model
Sam: "The AI doesn't really know what I want"
Turns out it was just a skill issue
@@SamMeechWard I call them user bugs.
PEBKAC
Thankyou 😂😂😂😂 I said the samething like what the hell..
This 100%. The more descriptive you are, the better the results. LLM is going to be as good as its understanding of the task, if you give it basic input you'll get basic output.
My take on this is that LLM powered tools work far better as collaborative tools than automation tools. You don't want them to do stuff for you, you want them to work with you to automate and search in the small and you provide immediate and continuous feedback to direct the process. It will get things wrong, it will guess wrong. If you stay in and close to the generation though, it's like having a bunch of high school aged interns at your disposal.
Exactly right. Highly motivated and slightly stupid interns.
I think we're going a bit off the rails if we got people designing their infrastructure with DiagramGPT.
100%
@Dom-zy1qy We think of this more as a tool to help you brainstorm your infrastructure and, more importantly, communicate it to others on your team.
I really enjoyed watching this video, realised the true potential of Better Prompting and LLM tools.
Thanks for sharing like this Sam... ngl - I often get suspicious of motivations when content creators disclose a companies involvement... but this was super compelling.
Thank you. I honestly thought the video would end on the low note in the middle. But watching it make that flow chart for me changed my view completely
Sam, in essence, your initial prompt lacks depth, akin to asking a physicist for lunar travel advice and expecting a simple solution. It's crucial to refine prompts to avoid overwhelming the model with endless possibilities. Remember, the model's outputs are probabilistic, sensitive to initial conditions, and can become chaotic with higher temperature settings. This is exacerbated when we use the models own outputs as later inputs. Regardless of the rules it has it will screw up if the initial prompt is so simplistic that its combinatorial explosion potential is very high. Your prompt provided minimal context for generating extensive content, you are begging for errors. Professionals would offer more detailed prompts, not simplistic requests. This app and ones like it assume some baselines such as enhancing expertise, not catering to beginners or oversimplified queries.
What he said ☝️
A very well articulated point.
Agreed! You read my mind.
Prompt engineering!
@@davidoffberlin I am not an AI no... what idiot would set their Ai off on a game of randomly commenting on videos about AI? That sounds a little like a waste of tokens to me, cheap as they are these days. Ohh look I also used the words 'It is crucial to ...' in there too... and other words from... ummm the lexicon. What do you think David? am I an AI or just educated and therefore easily confused with an AI?
Do I use AI... ohh yeah you could say that yes... maybe a good argument could be made that my personal lexicon after several years looking at outputs has been contaminated. I will agree thats possible. But generally no I'm not AI, I am not artificial. Having said that its an odd term AI... we do not after all say that an aircraft is artificially flying... its really flying!
Is it me or does the diagram at the end look wrong? Eg ACIT 2811 appears in the "with prerequisites" when it doesn't have any (according to the diagram). Compared to the earlier diagram, the dependency between ACIT 2811 and COMP 2311 seems backwards.
This seems to be the problem with a lot of this sort of stuff, it seems superficially "wow" but on closer inspection isn't actually right, isn't correctable in a reproducible way (ie fixing your scraping or graphing code) and ends up making the user a proof-reader and fixer, a process that is only marginally less work than doing it all manually in the first place.
This will be a banger for students that are working on their thesis 😄
Great f'n video. Kinda enlightened me on using the AI tools I use on a daily basis more efficiently... Cheers Sam!
In what formats can you then export it?... - What manual modifications can you do after the initial creations?... Can you re-import an export that has been modified (in a correct way of course)?...
5:47 personally I always build the data structure, or at least the first draft, for real first. Once you have that, and it's easy, there are numerous tools out there that'll generate the ERD perfectly for you and, if you're using something like SQL Server and know your way around the system tables, you can use the results of your create table/index/foreignKey/whatever scripts to generate your data dictionary. This is the fastest way to develop and document data structures in my opinion. Tools like Visio can even connect directly to your database and create an ERD in seconds... I've pretty much given up on AI for now. There's not enough value added to justify the overhead and I have scripts to automate most things I'd ask an AI to.
woah! super lucky the algorithm gave me this video...just started making some state machines for my game and was having trouble coming up with the flow chart...this tool made it wasy easier.
Really been enjoying your content Sam. Your tweet regarding this is brilliantly summed. We do the critical thinking and decision making and LLMs do the mundane tasks. Couldn’t agree more.
Thanks for watching my content, I appreciate the comments too. Let me know if you end up using these tools and find out something interesting that I missed
Nice!
Pretty soon, that's all we'll do - put in requirements, and facts.
The machine will generate the software, tests, and all the rest of it.
Unbelievable. This is my second video of yours and I am just starting to get invested in your content, and I find out that you're a program head at my school!?!?!
I am just starting term 3 of the CST diploma and I am in the AI option.
What are the odds? I cant wait to keep watching your videos and if you're ever at the Burnaby campus I would be beyond thrilled to meet you.
I think model is being rendered in real time, not being called multiple times. But this is totally mind blow. And very very handy! Thanks for introducing me to this tool. 🙏
If it mind blow then my aii system created the advanced schema just write input .it create a mongodb schema er diagram
What about the custom GPT in GPT4 from Eraser called "Eraser Diagram"?
I had a play with eraser and found the functionality that was the most limiting was that after having a flow graph generated and making a couple of edits there were a small number of items a matter of presenation that I wanted to rectifiy manually rather than consume free tokens to alter. However this thought chain proved that this system is flawed as there was no obvious way to manually edit flows in the ‘normal’ way after generating a flow.
I gave it a very simple prompt of some components and how they link to each other in a software architecture. It gave me only two boxes, with the same text that was none of the elements I mentioned...
The company (eraser ai) also offers Website wireframes; however, it doesn't appear to support AI for generating these (yet?)
Does this tool work for ISO 10303-11 EXPRESS-G diagrams for modules and resources?
I feel like a lot of people react to the first 2 mins or so of the video where things fail. But Sam demonstrates how to use this properly in the second half of the video. And that it is useful if you don't leave your brain at the door but ask it to produce narrow and focusdd outcomes which can be mundane and boring to do manually.
I believe the challenge is that companies are pressured to announce things like "we solved fusion with AI" then people use the product and go "well this is more like: explain what fusion means" and can't really appreciate that it's a great tool to do tasks a bit better because they we sold the dream
Almost quit the video also because he started critisizing the tool after giving it very vague information LOL! But I had faith in him!!
Great video. I'm wondering, did you get chatGPT to scrape that site? I'm confused what happened?
same here
6:42 He wrote his own script to scrape the site
Great demonstration of how to effectively use AI tools. Thanks for sharing!
Exactly my vibe. Does it get the schema and datatypes,.. correct? If you’re going off pen and paper or a document.
what if it was fed the description and 1) an Epic... or 2) multiple prompts of user stories?
This video was so good that just saying that is an understatement!
Of course you need to give it a context prompt. So that knows excatly what to do for you. You tell a designer give me a diagram for an X feature - guess what - that is the 1st diagram you get. Prompt details are eseential - you are giving instruction to your team.
speed increase for sure. Still need to validate it though. The 2 graphs at the end are different.
I still don't know how I feel about all of this. I would bet that a large number of companies would not approve of pasting schemas into a 3rd party app just to generate a diagram. How do they use that data? Do they then train their models on it to improve? For small projects this seems okay but larger ones it just doesn't sound like a good idea using.
Nice video Sam! This has changed the way I think about using AI tools. This tool itself seems pretty neat. That default of string for primary key is kinda horrifying though lol
Depends on the db, it didn't know I wanted a relational database with an auto incrementing integer 🤷♀️
Maybe I’m missing the point as I haven’t done a comparison, but why are people using Eraser when it’s a clone of Excalidraw which is free and open source (and includes this AI functionality)
HI mate, since your AWS videos are little outdated, would like to request react node file upload mini project that use AWS S3 and Lambda function and DynamoDB.
You can generate an ERD from a db structure. You don’t need AI for that.
The point is that you can't 'talk' to your generated ERD in that context :)
Exactly! Pretending that AI creates new possibilities and visualizations by turning an SQL script of an existing database into an ERD is just nonsense.Even database tools that have been around for more than 10 years can do that for you.
“it was a pain in the ass, because I didnt want to use python and all the good graphing libraries are in python” is an extremely relatable experience
How did chatgpt scrape the course page?
That is ChatGPT Plus. It can browse the internet for content. Then use the content of the page as prompt to create the list.
hey Sam, this is what they teach us 1/2 Year in the university - AI is out Of control. 😇. Let's make our own ai model.
The i in llm stands for intelligence
i'd like to see something like this evaluate the game Dungeons and Dragons
the chatgpt give me wrong result of the course extract
:
Does it have BPMN 2.0?
Anyone have advice for an AI that can return valid SQL from a prompt and deliver it programmatically to a backend that produces a front end result from running the SQL.
I would be very hesitant to run any sql code that's generated by ai
@@SamMeechWard fair enough and I agree and I have thought how to keep it quarantined from the server and pre validate it. But really think if done correctly a lot could be done with it in my field. But thanks for the reply :)
Hello, how do u do know? with the gpts?
Theres tools that can generate ERDs from your database schema that have been out in the wild for 20 years lol
Yes, but they are specific to the database or sql dialect. There's not tool that can take any sql code, or orm code, or object oriented data model, and create an ERD. This is general purpose for all languages and paradigms
Good answer by Sam. My big plus with such an AI tool is being able to 'talk' to my ERD. Otherwise I'd have to change the ERD myself. In Architecture, the older tools can be used for the as-is diagrams, and the AI tools for designs you'd want to modify by natural language
really? you gave the thing the vaguest of descriptions, then based your criticism on that. would a human do exactly what you had in mind with this kind of instruction without some kind of follow up ?
(yes AI has a lot of shortcomings which I was looking forward to seeing in a video like this, or things like can i iteratively update the diagrams with chat etc)
Earned a sub! More GPT tools please! 👌👌
Awesome content Sam!
And amazing moustache! Is it real? it's too nice...
arrrrrrborrrrrzzzzz!!!!!
Yes it's real. I made it myself 🤗
nice points, the AI will never be a mind reader, and will attempt anyway based on assumptions that are likely wrong
mm, i just created a diagram in this platform but i cant edit it manually, pff, i consider for while better use only excallydraw
thanks. very good one!
Thank you so much
I like how you think
Amazing stuff!
This is how I’ve been using copilot
most of my commands to co pilot contain the word "boilerplate"
Great Video, Great Mustache, papa Sam
Pretty interesting! 🧠
Sam, what is with that mustache. U like it?:))
Doesnt work on mobile
Got it, is for the viewsss
5:15 if you have the sql code we've had tools that spit out the diagram for DECADES
you dont need AI to do that lol
Show me a single tool that can take any database schema or data model from any language or framework in any format and produce an erd
We’re not talking about the same thing
Hey look, Mermaid + ChatGPT.
Your prompt was far too vague to expect a detailed response. Be more specific and you’ll get more detailed desirable results.
Lovely ... #HariKrishnaSahu
thanks
💡
Do real cool stuff using bun+hono and quasar 😝
Then I would have to learn quasar 🙃
This seems like such a bad use case for AI. Even if it did work, and it did create diagrams that makes sense, how much work does it actually save? I dunno, maybe I'm just an old head who doesn't see the value in having an AI that helps with something that takes 0.1% of your work.....
Not using your imagination! Think how would you use this with other tools.
Saved me so much time already
2:56 ok you just asked him to create a diagram for a e-commerce site! That what he did.
You didn’t gave him any context. So the only thing you can complain about is this application hasn’t been design to ask you questions in order to better understand your intent. And this has nothing to do with the LLM himself.
We are mid 2024 and people on TH-cam reviewing AI application still making the same mistake: a AI function in an application is not just the LLM, or even just one LLM. Between your input the generated output, here a diagram there are many components involved which are not an LLM!
So in short it’s a split responsibility: you didn’t provided the app any context of exactly what you was expecting and the app is not doing mind reading so should have been designed to ask you more questions in order to fine tune the context of your app.
🔥🔥🔥🔥
❤❤❤❤❤❤❤❤❤❤❤❤❤❤
Hey, Freddie Mercury! ✌
Improve on your prompting
I still don't feel convinced how this could be helpful. Maybe explain a different way
now we lost the job! In next century history will write these days as destroying humanity.
so it's a skill issue, good to know
subscribe for more skill issues
TIL.
You have to add that information in... that explanation makes no sense...😂😂😂 I provide the heavy details in my ai
Yeah, show it full diagram and then it will print it
Another demo that demonstrates how AI is useless. IDE tools can generate these schemas for decades
Yes, but they are specific to the database or sql dialect or using the specific IDE. What if I use a brand new programming language and a brand new orm that was just released today? No ide is going to be able to take my code and turn it into a diagram. This LLM is general purpose for all languages and paradigms
@@SamMeechWarddon’t worry… it’s just another bitter coder scared of AI. He knows he’s got a few years left before a 10 year old can code
@@J3R3MI6 my 5 year old nephew is coding! 🤯
@@FickleHeadYou can code too! It’s great fun even if it’s unemployable at some point
@@FickleHead so cool dude 🙏🏽💎 I can’t wait to have kids like that one day. When I was 5 I was playing with Hot Wheels lol
Ai is a very buggy if statement collection code. It is almost useless
This video shows exactly whats wrong with AI, the AI diagram is cool and all, but its wrong and you dont even notice it. For example, ACIT 2811 has ACIT 1620 as a prerequisite, but its not even listed there.
Sam you are very weak in prompt engineering. Garbage in garbage out. In prompt engineering the more detailed your prompt is, the better the result
Ai is so overrated
*LLMs are overrated