วีดีโอ

Please could you share this code in Python with me? I am studying programming and I want to see how ChatGPT implement double-entry accounting logic. Thanks.

It's adding information and changing the narrative of what is contained in the documents. I am a Christian, and I wrote and uploaded a sermon. The sermon specifically pointed to Christ accomplishing certain things in a person's life. The voice-over totally removed the narrative of the text and pointed to the fact that it's not about spirituality but about our own inner drive. The problem is it kept saying that this is what the document was saying. Now, if I had uploaded tons of PDFs that I didn't have the time to read for myself, I would have thought that what NoteBookLM was saying would have been an accurate representation of what was in my uploaded documents.

Does it understand other languages?

Sooo cool 😁

You are a literal Grandmaster.

How to contact you for pdf to excel project conversation?

So is the best way to use data in a secure way. By using ollama and llama3 instead of openai. And then adding csv’s or excel in streamlit and query that information locally?

Loving the series❤

As an accountant in my daily life I also tend to compare the models but unfortunately I only have paid subscription on ChatGPT. I find Claude very promising though. I feel like combination of ChatGPT and Claude would be nice

How can we implement this solution to work with loads of data, without uploading it manually but using a API

Upon trying to run, "!pip install pdfplumber -q" ERROR: Exception: Traceback (most recent call last): ... ... (tons of red) Couldn't progress

Have been using Claude projects to build a banking salesperson proforma web app (front and backend). Couple findings worth sharing. I find it’s best to put in the project prompt a chain of thought prompt block. I also periodically update the files created back into the project knowledge and start a new chat within project to keep its context fresh. And the every major iteration I ask it to generate a readMe file summarizing the project with a mermaid flow chart illustrating components. Also had it output python code extracting the project file hierarchy (that I feed the results of back into the project knowledge).

Your videos are awesome bro. I'm trying to move from a staff accountant role to a senior account role.

Thank you for this video!

Please Ihave many invoices each one with different structure of line items, sometimes you find first description qunatity ...

Very good

i passed out while learning regex💀

Amazing video! Is it ok to send confidential data to openai API? Does it store it or reuse it for training?

Hello, can you please help me find my social security number?

AI will not replace me AI will not replace me

Noiiiiiiceeee

Kinda off topic, but relating to your last video using ChatGPT to parse form fields..... I have been having success with using the API in Jupyter lab, with GPT-4o, by first base64 encoding the image, and sending the base64 as an attachment to the prompt, and in the prompt (example)'"Extract the permit records from the provided image. The first line of a permit record starts with 'Permit' and the last line start with 'Notes'. If a value is not present, provide null. Use the following structure: {\"Permit\":\"\" and I gave it 15 or so columns in json that were 'unstructured' data but a table nonetheless= a page can have 5 records, that have 15 columns, and each 'row' on the page is a mini table with other nested tables..... it will pull all the records out as json. you could replace the 'named field' = First Name : John Doe with 'semantic analysis' dropping even expressions if you chose to. You can add 'multiple' pages to each prompt as attachments. for each group of 5 pages it was "created": ____, "model": "gpt-4o-2024-05-13", "object": "chat.completion", "system_fingerprint": "", "usage": { "completion_tokens": 4096, "prompt_tokens": 1399, "total_tokens": 5495 25 page cost me something like .52 cents in API usage (I dunno, I was having it create fictious articles too, to see what it would do 'about the data ' article wise) too, so the token count may be off for the final $ amount. better yet? this works on Llama 70b, on Groq, with jupyter lab and a local RAG to vectorize the page/image on the fly..... so this is coming/here for local RAG

TY 😀

Wonderfully explained, thank you so much.

This is very good, I have a project to extract the PDF document which has multiple headers and multi line data. And I’m following your process and GPT advance also but I stuck in some cases. Would you like to connect with me Sir, that’s will be very thankful for me.

hey! nice video, simple and straightforward. Is there a way that we don't have to insert every time the html file for a specific company and instead just type the ticker symbol and we can download the data from it? I saw something with Edgar tools but they don't have all the rows. Thank you very much!

This is great, was having GPT coach me through a PyTesseract & spaCy variant with Named Entity Recognition. Do you have a GitHub or something with this code?

That prompt is massive, lol. The model processes it with every. single. message. the user sends. Even when you just enter "2". No wonder OpenAI is losing $500 million a year.

I found that by opening a pdf file with Mozilla Firefox and inspecting it with the developer tools you can collect its text (with the help of JavaScript) after the web browser has converted it to HTML and maybe save it for further processing with someone programming language.

You know, I started watching your videos years ago with pdfplumber and forms. And to be brutally honest, I had not even created a conda env, or even known how to install packages. I really had no choice but to learn how to do it, and 'getting stuff out of a pdf'was the reason. Largely because on the average month right now, I see 300 different pdf files, with different formatting, that can be 1 page, or 1000's of pages. Pretty much all unstructured text (without field names) still meant to create a table, or a 'row' that is a record, that is mini table of unstructured data (if that makes sense) Using AI locally as well as Chatgpt and now Groq 'Llama3 70' it's become really , in some cases easy to say, ' I exported this with this block of code (paste code block) and here is a sample of the text output (paste a page or two), IN THE NEXT jupyter lab cell, how do I parse the txt into these columns (paste the columns) Attached is i screenshot, tagged with the key elements I am looking for in the pdf page thats now text' Annnd it will spit out code. You may have to hammer away at it, and keep pasting 'error outputs' after running code, ..... but it WILL figure it out. I think the key here, is not all python packages are equal.... pdfplumber is an unsung champ, but pymupdf and other packages have strengths. going back to 'I think the key here', what I mean is, knowing what to import to open the pdf to maybe 'how do I get it out of this, into this' so I can parse it more reliably, and even automate it. If you google = 'Invoice or Bill Custom Parsing using Kor (LangChain Extension), Generative Language Models & Prompt Engineering (Part 1)' And take a look at what he is doing, it's similar to what you are doing here. The difference is you can use this with local models (ollama, LMstudio, Jupyter Lab ect). I see you (so am I) poking at the 'well, they say these things (AI) can 'see'...can they? With just 16 gigs of RAM, you can run the 7b models locally, and (try out the google search results) with some of the recent ollama ready models, they are pretty nuts with vision capabilities. I'm really starting to wonder if the future just isn't every body training either their own 'form/pdf' data on smaller LLMs like Mistral 7b, Llama3 7b, or really, any of the 'Phi' models from Microsoft. If it's just named entity type stuff..... a BERT model could be retrained on this. It's just become so cheap to do it, and the models are small enough to where even if you had 50 different trained models ( first class, Open Source AI models like Mistral/Llama/phi) on specific form data, the over all file size (and that it is 'local' = free) of them is moot, becaue if you can drag/drop a pdf file into a context window (or command line) and have the model output a json formatted response? Lol, thats a win, even if 50 models take up 600 gigs on a m.2 hard drive. It's less than $100 for a Terabyte m.2 ........ lol, with the rush for everyone to upgrade and play with AI, has the idea of 'parsing with expressions' evolved into 'prompt templates, vectorized data, and custom trained AI models'?

Great video, do we need to pay to buy tokens outside of the standard ChatGPT subscription for the API key to work ?

It would have been easier to have pymupdf crop the page, and export it as text, preserving the line spacing of the text to a txt file, and read in the text file with pandas to treat each line as a row, using line spacing to map data to columns.

The reality is, most pages that have 'in demand' data in tables are using heavy javascripting on the page . 'Selenium-Wire' (different from just 'Selenium')will catch the important parts though. pandas will read the html tables too. Rips right through it.

Hi PA, thanks for the videos. Can I ask, are you also CA or CPA? Or are you learning accounting concepts while programming?

Question from a aspiring data scientist: How did you go about finding and fixing the problem that occourd in the last video? When it didn't work for me I felt hopeless and like my project wont continiue for a long time like the last time I ran into a problem. How did you manage to find an alternative/working way? I am very impressed by your skills of not only finding and fixin the problem but also putting in effort of making it avalible for everyone and even going about making an instructional video explaining it. Today I'll celebrate you my friend. Sorry for my bad English, Greets from the Netherlands

I am curious what OpenAI model you are using in your API call. The reason is that when I screen shot a block of your PDF and submit the screen shot, I am getting 100% accuracy in the data extraction. I am using the 4o model.

How about a link to the PDFs you are working with so that we can run our own approaches and see if our tools will get different results.

Whoa this is so cool 😲

Nice could you please provide code link

thank you for the guide.

thank you!

Superb detail and very straight forward. Thank you for showing me your workflow and insightful guide to parsing semi-structured data

Thanks man I'm having a really hard time with this video called "It's super easy to use Python to scrape webpages for dynamic tables!" If I run the script in your WebScraping.ipynb it gives out errors I have to find to fix. Is there an updated version?

is it possible to read read pdf from online location like google drive, sharepoint using python without download pdf

promo sm 😩

Hey man, you helped me so much you have no idea. I’m struggling to add a feature to my code tho, could you help me out?

Respect! Great video, super explanation.

its not scanned pdf file