Sockets and Pipes Look Like Files (Unix/fdopen)

Patreon ➤ / jacobsorber
Courses ➤ jacobsorber.thinkific.com
Website ➤ www.jacobsorber.com
---
Sockets and Pipes Look Like Files (Unix/fdopen) // This video discusses files, file descriptors, and file-like things (like pipes and sockets).
Related Videos:
Sockets/Client: • How to build a web cli...
Sockets/Server: • Program your own web s...
***
Welcome! I post videos that help you learn to program and become a more confident software developer. I cover beginner-to-advanced systems topics ranging from network programming, threads, processes, operating systems, embedded systems and others. My goal is to help you get under-the-hood and better understand how computers work and how you can use them to become stronger students and more capable professional developers.
About me: I'm a computer scientist, electrical engineer, researcher, and teacher. I specialize in embedded systems, mobile computing, sensor networks, and the Internet of Things. I teach systems and networking courses at Clemson University, where I also lead the PERSIST research lab.
More about me and what I do:
www.jacobsorber.com
people.cs.clemson.edu/~jsorber/
persist.cs.clemson.edu/
To Support the Channel:
+ like, subscribe, spread the word
+ contribute via Patreon --- [ / jacobsorber ]
Source code is also available to Patreon supporters. --- [jsorber-youtube-source.heroku...]

[@reptilicusrex4748]

Thanks for making these videos. The content is excellent.

[@RenaissancManEng]

I love your videos. You are very efficent and concise. ... BTW What editor are you using?

[@puniat]

He uses VSCode as his code editor.

[@jonshouse1]

I prefer to go a level lower and use raw I/O for files, that way files and sockets are both file descriptors. FILE* buffered I/O works, but is from an era of small file reads and writes on systems will small memory. These days it is better to either read in larger chunks or to mmap the file and treat it like memory. Unless you have a workload like log files with the occasional small write it probably better to ignored buffered I/O completely, the underlying O/S cache mechanism will do most of the job for you.

[@maxaafbackname5562]

Why is using FILE I/O "bad" when reading lines, like in this example.
When using low level I/O you must buffer it yourself in some way.
Maybe using mnap() is more efficient, but not low/level I/O?

[@jonshouse1]

@@maxaafbackname5562 I did not say bad, I said "I prefer".
"When using low level I/O you must buffer it yourself in some way." ... No you don't ! the "f" functions and the "raw" functions both require a buffer to copy data from kernel space (filesystem/device) to user space, under fread is a call to read(). The only difference between fread() and read() is that fread() reads a chunk of data and sits on it in case you do another consecutive fread(), this really only matters if you do tiny consecutive reads or writes. mmap() uses read()/write() at its core but also has the overhead of managing file size changes (truncate etc). mmap is not useful for small IO. Basically what I am saying is unless you do small text writes at staggered intervals then most other workloads are just as efficient using unbuffered I/O on a modern O/S and have the advantage that everything become file descriptor only so the code is less complex.

[@benhetland576]

@@jonshouse1 AFAIK the "raw" functions aren't actually part of any standard C library at all, so going with the FILE* is the only truly cross-platform solution. They are part of the POSIX and some other standards though, so I'm really just being very pedantic now. In practice the open/close/read/write family and their int file descriptor has been available since the dawn of time_t on every *nix OS out there, and most others as well, only with slight variations in the names of headers needed. Such has been the state of matters even since the pre-standard C era.

[@jonshouse1]

@@benhetland576 Interesting, thanks. I did not ever consider that. Every OS i've used had open/close/read/write so I just assumed they where a standard base and other more complex functions where built on them. I know under the "fread" is read() as strace shows that. I never considered if read() was part of the standard, I just made the assumption.

[@nandhakumar1772]

Hello Jacob, would like to understand locks types (atomic, mutex, or any )

[@user-mr3mf8lo7y]

Line #49, sprintf, without formatting options and variable(s). Is this something better than strcpy? I recall sprintf empties out before writes into. Was that the intention somewhat?
Cheers,.

[@n0kodoko143]

Awesome explanation! Thank you!

[@beardymonger]

unix: everything is a file, *almost*
Plan9 (made by some of the same guys): let's fix everything that was 'almost' in unix, i.e. everything *is* a file, networking is builtin :-)
Sad that plan9's legacy is mostly UTF8 and the proc filesystem in linux.

[@benhetland576]

Yes, Unix used to have two kinds of things in it. It was either a file or a process. Enter /proc, now everything is a file!

[@danielskrypnik5181]

Is there a video where you explain C/C++ compilation process step by step?

[@ohwow2074]

Watch Cherno's C++ series. He's a game engine developer. Has really nice videos.

[@notjoemartinez4438]

Went down this rabbit hole by running the file command on random .sock "files" in my /tmp directory then trying to open them vim 😂

[@AnantaAkash.Podder]

Excellent Video... Thank you Man...❤❤

[@maxaafbackname5562]

Even if stdin is redirected, it is still not necessary a pipe...

[@mosha2533]

Iam here after Instagram reel

[@waltsullivan8986]

You're missing all the out-of-band events and signals that networking provides by treating sockets as simple files.

[@JacobSorber]

Very true. Probably a good topic for a future video. Thanks.

[@m0Ray79]

You actually need only one argument. "-" for stdin, string starting with "" for http and any other string for a filename.

[@erbenton07]

tip: instead of using while(feof use while(fgets

[@m0Ray79]

Pinging a server to get its IP address? Mmm, that scent of Windows...

[@erbenton07]

Don't use while(!feof...
use while(fgets,....
feof is basically useless

[@AdvaTced]

lol, The most important thing you missed here, Which is the file descriptor fd of the socket that have been opened by us as the socket of the client, This guide/explanation is dead when you reached the 7:53 and when you said "we just get from the `http_get` function, What actually is inside this `http_get` is that we're opening a socket and sending an HTTP 1.1 / get request via ***socket**** so we do open a fd which is the socket id that retrived and than we're openning the fdopen file stream of this particular process file descriptor for the particular client. you just didn't show anything about it, and it's a big miss, And this is the WHOLE POINT of this video.
what you have done there is to ESTABLISHING a connection to the webserver and retrive data from it, and you streamed The data that have been received to a Unix domain socket... AND this Unix domain socket has an open fd for this particular process! You didn't explain that. THIS IS SUPER important.
I'm giving 6/10 to this video.