Read CSV File Data Into An Array Of Structs | C Programming Example
ฝัง
- เผยแพร่เมื่อ 25 เม.ย. 2022
- How to read CSV file data to an array of structs in C (i.e. a file where each line is a record of comma separated values). Source code: github.com/portfoliocourses/c.... Check out www.portfoliocourses.com to build a portfolio that will impress employers!
This is great! I and my friend are going to learn more C this summer. Now we’re dealing with finals. I shared the channel with my classmates and my friends. Your channel is a gem!
Awesome, thank you so much for sharing. And good luck on your finals! :-D
Kevin, I can't say enough about this style of teaching how to program. I have a small library of books that come nowhere near this depth. I was suckered by the word "advanced" in their titles. I hate wasting time practicing writing B.S. snippets. I want/need to spend time writing code with utility, and that's what you do! I feel this knowledge will stick with me, and it definitely gives me a much better understanding of C. Thank you!
I'm so glad to hear these videos are helping you out Wayne, and thank you so much for the positive feedback and words, that really means a lot to me! :-)
I looked through your video list and couldn't find one where you went back to this subject and made a more robust and generic CSV file reader. I would suggest that if you actually do so, you use a dynamic array of whatever objects you decide upon storing for each line. Maybe teach people to use the C11 strtok_s() function instead of the older strtok() and perhaps teach about unions and enums so you can select the type being stored. Perhaps when storing a string use the C23 function strdup() to copy the string being stored and demonstrate free() at the end. Just a thought.
this is exactly what i needed for my programming class, thank you
You’re welcome, good luck in your programming class! :-)
This tutorial is actually good stuff. Well done sir!
I'm glad you enjoyed it! :-)
Excellent video thank you for your efforts and knowledge sharing!
This is pure gold! ... Thank you!
Thank you, and you're welcome Gerald! :-D
It actually worked! You are amazing.
I'm glad to hear that it worked for you Lauren! :-)
thankyou, i finally got the video, which I needed desparately.
Thx mate! Helped a lot!
I'm glad it helped, and you're welcome! :-)
what if students characteristics were separated by '
'???
Thank you sir , great content 💯
Thank you! And you're very welcome! :-D
This example works great as long as all 4 variable types do not change. Try using this code using all string type variables. %c, %49[^,], %d, %lf
will fail very badly. Neither will the unwanted commas be parse or omitted from the output.
Yes of course, this is just an example, if you want to read a file that’s formatted differently you will need to change what placeholders you use, what your struct members are, etc. :-)
a file called "file.txt" with content:
U,Virat Kohli,23,95.6
U,Serena Williams,22,83.2
G,Wayne Gretzky,19,84.2
must be saved directly in the main folder of the project. If using Code Blocks then this text file must be in the same location of main.c
Yes, if you're using some sort of IDE / Code Blocks you might need to save the file in a working directory (what that is will depend on the IDE). :-)
How would you go about using this code to make another simple program to then have options to sort by average, name or age?
I am thinking of how I may use this: probably an MCU connected to a WiFi router that downloads a CSV file from the vendor server on weekly bases, for example. Then the CSV file is automatically interrogated by the MCU to read the new features added by the vendor for the customers device. Then the MCU automatically dump the new settings of the product into the EEPROM of the MCU and the product is then automatically updated without the customer knowing what is going on, in the background. I love this feature.
Sounds cool, I love your enthusiasm for these things! :-)
Hi I have a similar project but my file is separated by one or more space(a). I want to store mine into struct array as well, how do I go about modifying this to account for the potential extra delimiter.
Thank you for your great work sir 🙏🏻🙏🏻
Great question! :-) And I think you could replace the commas in the scanf with spaces like this, and it should work even if there is more than one space separating the different fields of data:
read = fscanf(file,
"%c %49[^,] %d %lf
",
&students[records].type,
students[records].name,
&students[records].age,
&students[records].average);
this is amazing
Helped a lot
I'm very glad to hear that! :-)
He makes it look so easy. Ugh!
If you have any questions let me know. :-) I want to make a tutorial-style video covering scanf in more depth in particular.
@@PortfolioCourses, thanks. It isn't the programming per se that flummoxes me, rather C's terse syntax and the way it handles pointers. When one is used to Pascal and Visual Basic, it's quite different and quite a challenge.
Thanks again.
@@Mnogojazyk I hear you on all of that, that's why I really enjoy Python a lot as a programmer.
Great code! Was wondering if there's a way to clear the structs for repeated use ?
Thanks! :-) And yes, we could use memset() to set the entire array to 0: th-cam.com/video/Njsn5HAnAnk/w-d-xo.html. Or we could write a loop that would manually go through the array, setting number members to 0 and char array members to an empty string.
If your csv contained an array of strings like [one;two;three;four] how would the corresponding code in the typedef struct change? And how would this change how we read that part of the file into the struct?
That's a great question Jon, but I think I would need to make a whole video to explain those scenarios properly. :-)
I think it would depend on whether the file contains only an array of strings on each line, or whether it contains an array of strings on each line along with some other information.
Because if it's only an array of strings on each line, I would be tempted to not use a struct at all. Maybe a 2D array of pointers to strings could be used instead to store the data in the file. Or if we use a struct, maybe we could "name" each field/string and store them into a char array. We might store the string data using dynamic memory allocation if the length of the strings can vary and there is no maximum, or we might store the string data on the stack if the strings have a maximum length.
If it's an array of strings on each line along with some other information, then I would use a struct. Though again we could store the strings in either dynamically allocated memory or in char arrays on the stack.
And in terms of reading in the strings, we could use the same sort of technique we used here, but with multiple strings. Or we could read in the whole line with fgets() and then use something like strtok() to go through and copy out each string: th-cam.com/video/nrO_pXGZc3Y/w-d-xo.html.
I think maybe one day I'll do more videos on reading CSV files, including some challenging cases involving string data.
A grand Thank You 😊
You’re welcome! :-)
Hi! great video, i have a question maybe you have an answer. If im reading with fscanf like you but there are some fields(columns) I dont want to save, do I have to put 2 commas together? It would be like in your example if you dont want to save the students age, would it be: fscanf(file, "%c,%49[^,],,lf
", ...) ?
I would assume that there is still some data in the columns, so you might want to just use a placeholder like %d or %49[^,] or whatever is appropriate, and temporarily store the data into a temporary variable (but in the actual array of structs). That would likely be the easiest/simplest way to deal with the situation. :-) There may be other ways of course, but that's the first idea that comes to mind.
Great video.
I get a bug that the fscanf returns 0 all the time. I check the file load, and it is working fine. Any help would be appreciated.
Thanks Yu! 😀 The original code in the video is available here: github.com/portfoliocourses/c-example-code/blob/main/csv_to_struct_array.c. Without seeing your code I'm not sure what the issue could be. It could be that the file is not in a compatible format, rather than an issue with the code.
Tysm!
You're welcome! :-)
Hi, if there was a header line, how would i go about removing that first?
Great question Tommy! :-) Before the do-while loop you could have this code below to read the first "header" line of the file. It reads each char one at a time until the first newline character
is encountered which signifies the end of the line.
char c;
do
{
c = fgetc(file);
if (ferror(file))
{
printf("Error reading file.
");
return 1;
}
} while (c != '
');
Hi there, I have a CSV file that has commas and new lines within the fields between the comma delimiters. What should I do?
Great question Julien! :-) This would involve some more difficult parsing of the file contents. It's possible to do this 'on our own' by using some algorithm to parse the file, but at that point I would consider using something like regex to parse the file: www.geeksforgeeks.org/regular-expressions-in-c/. Or using a library that is purpose built to parsing csv files: github.com/rgamble/libcsv. Maybe one day I can do a video on this and go over different approaches.
you may have a look at "strtok" in "string.h" to solve it.
Thanks for the video! Could you make a video on how to copy data from a .csv file into a 2D array of strings next? Thank you
Hmm that's a good idea for a video, I will add it to the list of ideas. :-)
@@PortfolioCourses thank you!
You're welcome! :-)
What would happen in case that it’s not the average, instead of this, let’s put 5 or x tests ….
In this case you have to put each int? Or you can use an array with the numbers?
Great question Daniel! :-) You could do either... you could have 5 int members, or you could have an array of 5 ints, either approach would work.
Thank you
You're welcome Sang! :-)
I did the same program as yours and when I run the code it says the file format incorrect. why is that?
The code is posted here: github.com/portfoliocourses/c-example-code/blob/main/csv_to_struct_array.c. At the top of the file in a comment there is an example of what sort of file content format it will work with. If you use this code, with that content, it will work. 🙂 I don't know why your program isn't working, there are many reasons why it may not work. Perhaps if you look at your program and compare it to the file that I linked to, you will be able to identify the difference.
Can we not take the integer values as a string while using fscanf? like can I use %49[^,] for every value instead of %d,%lf and so on?
I suppose you could read them as a string, but then you would need to convert them to int (or double, etc.).
Oh! thanks a lot
Great Video btw!!
Thank you! :-)
what if I want my output in a order like in a table format.
At the end of this video, letter frequencies are output in a table format. It might give you some ideas for how to do what you want to do: th-cam.com/video/1OZMcC0euic/w-d-xo.html. Good luck! :-)
How can I read this one
"Please, Please, Please Let Me Get What I Want",The Smiths,112
All Star,Smash Mouth,220
Lucky,"Colbie Caillat,Jason Mraz",196
It's a CSV file of songs
song name, singer, length
but some song names have commas in it
and singers also have spaces or commas
How can I deal with this kind of file?
This is more challenging, we need to rely on the quotes to help us identify the "field" of information in this case. I may make a video on this more advanced sort of parsing one day. 🙂
what if we don't have the commas (,) suppose we have space between the records of each line how can we do this ?
Great question Fatima! :-) You should be able to just replace the , characters in the format string with space characters, and that should match for one or more whitespace characters: stackoverflow.com/a/12835596.
Thank you for the great videos. i have a question what if the char array (name[50]) is at the end of the row?
I tried fscanf(file, "%c,%d,%lf,%49[^
]
" ..... but it didnt work. i could not find a solution.
Hmm maybe try taking out the last
at the end of the string?
@@PortfolioCourses Thank you for your answer. I tried that and a lot more and googled it without success. I could not find a solution .
what happens if i dont put
at the end of the fscanf
Adding the
has fscanf read and discard any whitespace characters that follow until a non-whitespace character is encountered: c-faq.com/stdio/scanfhang.html. This has the effect of removing the
from the file input buffer, so that the pattern matching can begin again with the next int on the next line. We could actually move the
to the beginning of the format specifier string and it would work too. :-)
hello! amazing video, just wanted to ask about an error i kept getting (format error) and its in this code:
counter = fscanf(data_file, "%d,%69[^,],%d,%d
",
&persons[lines].id,
persons[lines].name,
&persons[lines].age,
&persons[lines].weight);
the percentage mark in %69[^,] is red and im not sure why but it might be the problem, and I don't know how to fix it
and this is the format of said file:
#,Name,Age,weight
1,Joel Hunter,57,52
2,Cody Johnson,21,74
3,Thomas Lewis,52,49
4,Willie Ochoa,41,87
5,Samuel Davis,38,63
6,Raymond Nichols,27,55
7,Matthew Bentley,63,90
nvm im stupid, i needed to skip the first line. TY
I’m glad that you were able to figure it out! :-)
Realizing this video is over a year old, hopefully this will be seen and read. Some questions:
Early on after opening file.txt and checking for opening error, should there be "exit(1)" instead of "return 1" since no function had actually been called? Same query applies to the error check after a file read.
While entering the format specifiers for fscanf should "%c,%49[^,],....." be "%c,%49s[^,],..." instead?
And, finally, how could one handle reading a .csv file in which one, or more, vaules you want to read is empty, i.e.
U, Virat Kohli,,
U, Serena Williams,21,83.2
G,Wayne Gretzky,19,84.2
I'm sure this situation will negate the file record read check where you check the variable if (read == 4) since fscanf returns only the number of formal parameters which were successfully assigned values. My research is telling me this isn't something fscanf can handle. In this case will I just have to read the .csv file line by line and parse each line to extract the data, i.e. using fgets() and strtok()?
If we had called a function from main and wanted to exit the program during the execution of that function we would use exit(). But using return 1 in main will exit the program with a status of 1 the same as exit(1).
Re fscanf() it works without the s, so I don't understand the question.
And regarding fields not being there, I would not recommend fscanf() at that point unless you want the program to just terminate when the incorrect number of fields are there. You could likely still use fscanf() to solve the problem, but it would require a substantial re-write.
@PortfolioCourses I see. I must have had some bad syntax because when I tried the code and induced some errors, I wasn't getting the error msgs from the program displayed. Once I changed the syntax from "return 1" to "exit(1)" I saw the error msgs. But, don't quote me on that. Been some hours since trying it.
Also, I wasn't getting the desired results until I added the s after 49. I'm using Code::Blocks 20.3 compiler btw.
Been banging my head on this csv file and the consecutive ,,, is stopping me in my tracks. Trying to incorporate your teachings of reading csv file data and populating an array of structs.
Anyway, thank you very much for your time in reading & responding! Meanwhile, I'll stay tuned and soak up the knowledge you share.
I think I have learned of the proper function for my needs - srttok() or strtok_r(). It will tokenize the line while returning pointers to the beginning of each field within the line of csv data.
sir , I have a problem with a c programme , I want to print some names(1) of persons in the beginning of the file and other names(2) down the first names
it works in the first code run but when i want to add more names of the two kinds , the name(1) goes down name(2) of the first code run , not in the beginning of the file..
like that : name1/name2/name1/name2
but I want name1/name1/name2/name2...
and sorry for this big comment lol..
Hmm I’m not sure what’s going on based on your description Abdelhak. Maybe if your post your code in a comment with what should be the correct output me or something else may be able to help you.
@@PortfolioCoursesokay i commented
@@abdelhakmezenner2855 OK I left a suggested approach as an answer.
is there a limit of characters in printf? if so, how can I fix it? my program is not printing whole data
it works for 26 students, after that number the first lines are missing
okay apparently it works depending on the compiler i use
Yes, it does depend on the compiler: stackoverflow.com/questions/8119914/printf-fprintf-maximum-size-according-to-c99?rq=1. :-)
HIi, if i just want to read certain data at certain location only, how can i do it?
Great question Ong! :-) This video may help you with that, it shows you how to read a specific line from a file: th-cam.com/video/w0mgn6OLKUs/w-d-xo.html. You could combine the ideas from both videos if you need to read certain data from a CSV file.
you can try to parse each line with "strtok", from "string.h".
then you pick only the fields you want!
@@marc-andrebrun8942 I agree strtok can help too, this video might help them with that: th-cam.com/video/nrO_pXGZc3Y/w-d-xo.html
Where is C programming used nowadays ?
Im going into advanced C topics. I did cryptography, sockets, threads. Can C be used for data analysis ?
C programming can really be used pretty much anywhere. :-) It's popular in systems programming, embedded systems, and other lower-level kinds of programming. C could be used for data analysis but other languages like Python are popular for that now.
@@PortfolioCourses C for data analysis, that would be awesome!! I will take a look before start python.
@@serebbi Cool!
My csv file has a header which is all symbol and string, so when I run the program it give me an error " File format incorrect." How could i solve this problem sir
Or how can i make fscanf function start reading from the second line of the file ?
Great question John! :-) You would want to read the characters from the file up until the fist newline first, before reading the data from the file. So above the do while loop in this example, you could have something like this maybe:
char c;
while ((c = fgetc(file)) != '
');
That should continually read characters on the first line of the file, and then stop when a newline character is read. At that point the file pointer should be pointing to the second line of the file, and you could then start reading the data with the do-while loop from there.
@@PortfolioCourses Thank you sir it worked 👍👍👍
@@johnadam9785 That's great! 😀
how can i sort on the basis of age in ascending order
So you can apply any sorting algorithm, such as Selection Sort: th-cam.com/video/YepJ7fDmyjI/w-d-xo.html. But you'll need to use .age to access the age member specifically of each struct when performing comparisons. And you know this is an excellent idea for a video, perhaps I will make a video on this topic soon! 😀
@@PortfolioCourses i am stuck on this since 2 days and have an assignment due today and there is no video which explains a sorting of a file using age in ascending order and I really need help.I have 5 hours can you help me with that?
@@swagatpandak7325 Sorry, I won't be able to make a video on that topic in the next 5 hours. 😞 You could use these videos to help you though.
- this video we're both commenting on should show you how to read the data from the file into an array
- this video will show you how to sort the data in the array, the only difference is you'll need to use .age to access the age member of each struct in the array specifically: th-cam.com/video/YepJ7fDmyjI/w-d-xo.html
- after the data is sorted, this video will show you how to write the data in the array back to the file: th-cam.com/video/rbVt5v8NNe8/w-d-xo.html
@@PortfolioCourses no problem i got it figured it out from your github.Thank you
@@swagatpandak7325 OK, great! 🙂
What if each letter, name and number has quotations around it like “U”, “Serena Williams”, “21” and etc ?
You could do something like skip over the first character in each field knowing it is a ", and maybe using " to recognize the end of each field (skipping over the , before moving to the next field). Or you could store the strings like we do the string now, except after the string is stored, trim the first character and the last character from the string knowing they are " characters. Numbers could then be converted from string to a proper number type with functions like atoi() if we like. This could be a much trickier problem though if the data for our fields allows for things like: "data \" escape character more data". Where we have \" in the middle of our string, using \ as an escape character to have " in the string instead of ending the string. At that point we may want to use something more advanced like the regex library to help us parse things. :-) So it's definitely do-able but it introduces some new problems, maybe one day I can make a video on this too.
why when i compile the code its only print
Error opening file
and also i try when i delete this function it also print the next one which is
File format incorrect.
i have try a lot i dont know what is the problem can you help me
Hmm, I'm not sure why Bakr. 😞 The source code is available here: github.com/portfoliocourses/c-example-code/blob/main/csv_to_struct_array.c. If you try to use this exact source code, with a file in the exact format as in the video, does it still give you the error?
@@PortfolioCourses yes it’s a cvs file I put in fopen(“data.csv”) it’s right ?
It’s okay to put you a link to see for the file I want to open to know
@@bakramazon191 Yes, if the file is called data.csv you would put data.csv.
@@PortfolioCourses yeah it’s work but I have a question first if my type not a letter it’s a numbers it’s will be the same ? And second why when I complied the first line was not printed
@@bakramazon191 The example code includes examples of reading in characters/letters (strings) and numbers, so you would want to use the same approaches as used in the video, but you may need to adjust which fields are read using which approach depending on how your data is formatted. In the example the format is a character, followed by a string, followed by an integer, followed by a double value (i.e. a number with decimals). Your file might have a different format. I don't know why it would not print the first line. There could be many causes for that, for example a counter variable in a loop used for printing the data starting at index 1 instead of index 0.
I have 10.25.1998 (date of birth) and I need to break it into a day, month and year. How to do it ?
Great question! :-) strtok() may help you out: th-cam.com/video/nrO_pXGZc3Y/w-d-xo.html. As strtok() gives you individual pieces of the string, strcpy() may be helpful too to either save the entire string or copy pieces of it into other char arrays, etc.: th-cam.com/video/RQ8O13utXQw/w-d-xo.html
That said, you could do it in a REALLY simple way with:
char date[] = "10.25.1998";
char month[3];
month[0] = date[0];
month[1] = date[1];
month[2] = '\0';
char day[3];
day[0] = date[3];
day[1] = date[4];
day[2] = '\0';
char year[5];
year[0] = date[6];
year[1] = date[7];
year[2] = date[8];
year[3] = date[9];
year[4] = '\0';
Hi,
i have similar project but ihave space between the file data not comma i replaced space insted of comma but i got file format incorrect:
int read = 0;
int records = 0;
do
{
read=fscanf(file,
"%d,%14[^ ],%f
",
&allStudents[records].std_num,
allStudents[records].std_Name,
&allStudents[records].std_Grade);
if(read==3){
records++;
}
if(read!=3 && !feof(file)){
printf("file format incorrect");
return 1;
}
if(ferror(file)){
printf("reading error
");
return 1;
}
} while (!feof(file));
Hmm, in that code there Bilal it still looks like you have commas in the scanf:
read=fscanf(file,
"%d,%14[^ ],%f
",
Like there is %d, for example at the start. Have you tried replacing those commas with spaces instead?
@@PortfolioCourses thanks a lot I appreciate your help ..
I have replaced them and everything works.
@@BILALMAHFOUTH Excellent, I'm glad to hear it worked! 🙂
@@PortfolioCourses Thanks i had the same problem!
@@redabelcadi You're welcome! 🙂
I have 3 char array
read = fscanf(file,
"%d,%49[^,],%49[^,],%49[^,
",
&students[records].n1,
students[records].name,
students[records].email,
students[records].dob);
if i did like this the do-while loop is keep on running
Hmm, it looks like the end bracket ] is missing from the last placeholder? Could that be the issue?
@@PortfolioCourses yeah sir I corrected it but even though it showing file format incorrect
I pasted my hole code csn you see it ?
@@aravinthananth6770 No, I can't see it. If it's showing file format incorrect, you might want to check to make sure there is no extra blank 'newline' at the end of the file.
@@PortfolioCourses I guess this is happening because the rightmost column of the csv file is a string.
Who actually pronounces char as car?
Hahaha… there is actually no definitively correct way to pronounce “char”, but “car” is one of the valid ways to pronounce it: english.stackexchange.com/a/60175.
Who actually complains about such things unless they're soft in the head 🤔
i wrote that code and i get that error
1 Michael Jordan
2
7895664 \Users\User\AppData\Local\Temp
nr:0file format incorrect
why would it be?
while(!feof(yp)){
newyazar=yekle(head);
read = fscanf(yp,"%d,%29[^,],%29[^,]
",&newyazar->yazarID,newyazar->yazarAd,newyazar->yazarSoyad);
Most likely it is because the format of the content in the file doesn't work with this format string. I also wonder if that last %29[^,] should be %29[^,
] because otherwise the newline may be read into the string.
thank u man u are the best one!!!!
Could you please give your email sir, I want to share a file with you
What file do you want to send me? At this time I am unable to answer questions via e-mail, but I do try to reply to TH-cam comments whenever possible. Sometimes people copy and paste code segments into comments so I can see them.
read = fscanf(file, "%d,%49[^,],%d,%d
" the percentage mark in %49[^,] is red and i dont know how to fix it..
Hmm, I'm not sure why that would be based on the code you have shared Idan.