CORRECTION: The resource kit was apparently a lot more available than I realized. I knew it was in the book, but I didn't imagine the book was *that* common. I never saw it at any of the shops I worked, but that's a small sample size.
I worked as admin in a small lab, the Resourse Kit seemed to be standard must have at the time. The books filled up half a shelf, and occasionally helped, but that's not why you bought it.
I remember the NT Resource Kit well. I was an NT admin back in the 3.51 / 4.0 days and that kit was indispensable. The book was also riddled with errors - documentation that was flat-out wrong. "It works like this!" No, Microsoft, it does NOT work like that.
technically i think the grammar is correct, but regardless it does intuitively sound backwards indeed Windows Subsystem for Linux the Windows Subsystem (the subsystem that exists within windows) that is used for running Linux
@@shallex5744 yeah, I get that’s what they were going for, but it’s still backwards. Anywhere else, the noun adjunct to “subsystem” describes what the subsystem does, and “for” indicates what subsystem is meant to be a part of/run on. It’s not a Switch emulator for N64, it’s an N64 emulator for Switch, for example. Even if the grammar is technically ambiguous (it probably is, English is not especially precise), WSL is still strictly backwards to the standard convention used in any other similar context.
It's been a long time since I had to fool with it, but iirc "enterprise support" didn't get you far and in our case, lead to migrating to an entire MS shop against the wishes of everybody who didn't have their hands in the piggy bank. Which if you boil it down, that seems to be the intended result anyway.
Enterprises which wanted to use POSIX software on Windows NT were very rare and those that did used one of the variety of 3rd party solutions on the market and not the POSIX subsystem. These 3rd party solutions mostly targeted software companies looking to provide their product to Windows only shops. Most POSIX software running in Windows back then was 3rd party software from these companies and the terms of the POSIX solutions allowed distribution of their software with the POSIX runtimes.
M$ does not support. There were some major bugs reported to Microsoft in their SNA implementation (another embrace and extinguish, but then again, who really wants SNA). Bugs were send with fixes and reproduction instructions to their enterprise support. 6 months later a reply came that they found it was a bug and that they will look for a solution. That's how enterprise grade their support was.
@@bloepje SNA Gateway was a particularly bad product. Many of my employer's customers at the time unfortunately didn't discover this until replacing the reliable Novell SAA with it. It was so bad but also so key to my employer's customers accessing our primary product that eventually my employer was able to strike a deal with Microsoft. They loaned some of their developers to Microsoft to help improve SNA Gateway. Besides the poor quality of that particular product my experience with Microsoft support was ok, but at that time my customers were also very large enterprises or large government agencies which had large expensive contracts with Microsoft in order to get a high level of fast support.
I've always kinda knew this subsystem existed, but wow I didn't realize it was this cursed. Crazy to think that presumably many many hours were put into designing the subsystem, only for it to be basically unused. I wonder what it was like for the Microsoft developers back then.
NT has a native subsystem capability that was leveraged for this. Microsoft simply added another subsystem, although it certainly took a long time to get it going. Maybe there were plans to improve it, maybe there wasn't Microsoft replaced it with "Windows Services for UNIX", which was SUS compliant (probably not getting away with that half baked POSIX bs anymore), and then WSL1, and then WSL2 (which is literally a linux VM running inside hyper-v with some clever stuff to get it communicating with windows) Btw, NTFS is also case sensitive, fwiw, windows just...kinda kludges around it
I worked with the NT beta versions and release versions from 3.1 onward, porting a product which ran on various Unixes. We initially thought we'd be able to use the POSIX subsystem, but it was quickly obvious that this was nothing but a box ticking exercise. Our product which ran on NT, on MIPS, alpha and x86, was completely win32 with no use at all of the POSIX subsystem.
I worked on a project in 1995 where management had the vision that we should have a common codebase for Unix and NT. The team actually tried to use the POSIX subsystem whit the same success as showed in the video... Our solution was to write our own libraries as a version of pthread for NT and not to tell management. 😁
@@vkvishnu Nothing remotely like that. My work was on a PostScript interpretor for industrial printers (film recorders, A0 sized printers etc). Our product already worked very well on a range of UNIX workstations (SGI, Sun, Digital etc) and porting to other UNIX machines was easy enough because they had a common (POSIX + X11) API, but the Windows port was very much more complicated than expected because the POSIX subsystem was basically completely useless. Therefore we had to write a load of windows specific code.
The UNIX API begun in the early 70s, when the system was ported to C. POSIX provides a standard for this API, as many UNIX systems got further and further apart from the AT&T sources over time. Currently POSIX means at the very least you can run console apps on Linux, *BSDs, and Mac
tbh I'm not surprised this is such a mess, I've never had POSIX accomplish what it was supposed to. Anything beyond basic apps has always required a pile of ifdefs even for stuff like memset/memalign/malloc. They didn't even add endian swapping functions until *2014*... for a standard which is literally centered on standardized IO.
@@ShinyQuagsire I think you are conflating Windows adherence to the POSIX standard with the actual POSIX standard. The reason why WSL exists is because of Windows attempting POSIX compliance.
Back in the 90s I used to work for a company that implemented Windows NT and for a while had the largest NT network in Europe. I was aware of the POSIX subsystem which I understood was present to allow sale of the product to US Government entities that had strict policy on procurement where certain features had to be available regardless of use-case. Another feature was C2 security which Windows NT complied with although security capabiltiy has been enhanced since then. I recall anecodally that some software vendors had made their previously Unix targetted software available on NT using the POSIX subsystem but never saw it myself. I had access to Technet and MSDN so had the resource kit and SDK as well as development tools and there was loads of interesting stuff included. For example the source for the built in screen savers using OpenGL library. At that time Cygwin didn't exist and even doing development on Unix/Linux relied on the runes aligning to setting up Windows for this wasn't vastly more complex!
Yeah, I remember M$ bragging about that C2 compliance from the very start during the presentations of NT 3.1, and how hard work it was to get that certified. Later I found out that this approval was only valid for a certain subset of hardware/software components, and anything with a network (even just LAN) was _not_ C2 compliant. Even back then running without some form of network was pretty useless.
Considering that at that time Microsoft has already have enough experiences with developing XENIX, the half-baked POSIX implementation were deliberate moves by Microsoft.
I always wondered what the POSIX subsystem option was in Visual Studio's linker options for C++ projects, I guess it still has some support even in Windows 11 and VS 2022, how interesting.
The idea the resource kit was obscure or hard to get a hold of is just incorrect. If you wanted the book along with it every bookstore carried it in their technical book section. It was advertised and well documented in magazines of the era. It was available for free via the Microsoft BBS and the Microsoft FTP site. It was very commonly installed on NT systems.
I worked in IT in this era. I remember the 9x ones. The NT one was rarely, and it didn't note that the POSIX tools were included. The book itself doesn't even list that; I had to check the disc myself. The book just lists traverse checking has to be disabled, and nothing else.
My brother and I had the NT 4resource kit though he had to order it. A few years later I was able to get the 2000 resource kit but that was a lot more common..
I love the NCommander In Realtime series. Your livestreams are a bit long for my attention span, so I do like these re-narrated shorter versions, since it lets me see what curse-ed things you're doing... and I totally love the mindset. "This thing is available, I wonder if it's usable, it's probably not but they said it worked so we gonna try it". I always knew the POSIX compatability was kinda trash but you've shown it's total trash, thank you for sinking hours into this, great to watch!
POSIX on Windows is so funky, the default C library (msvcrt) fools you into thinking it might be POSIX compliant until you actually try to use it, add the 10 official implementations of POSIX, each for a different version of Windows, third party libraries like Cygwin and MKS, as well as Microsoft simply integrating a Linux kernel into the OS.
@@MaxOakland macOS is weird since it combines Old Macintosh, NeXT, Unix, and Mach operating system heritages. Macintosh gives you the GUI style and file-systems, NeXT gives you the premier programming API, Unix gives you the more standard C API and file-system structure, and Mach gives you some of the kernel and executable file format
Fun fact : Linux isn't actually POSIX-compliant. Stallman thought some of the decisions of the norm were stupid and never actually got the certification. You CAN make it essentially compliant with the variable POSIXLY_CORRECT or POSIX_ME_HARDER, but it's not official, making macOS the only fully POSIX OS outside of "real" Unix like AIX or HP-UX.
A couple of Linux distros (specifically, EulerOS from Huawei and Inspur K-UX, both RHEL derivatives) got UNIX 03 certification at one point, proving that Linux indeed _can_ be made "fully compliant" if needed.
Reminds me of the time when Java applets were a thing, Microsoft shipped their very own Java Virtual Machine. It was so bad that Sun sued them and eventually won.
Uh, assuming it did pass the NIST test suite, then the problem was with one or more of: the standard, the test suite, or people's expectations. Undocumented, 'soft' standards are a problem.
Agreed. If the government wanted to be able to build rogue on Windows NT, the requirements should have included the pieces needed for it. I can't fault Microsoft for not choosing to go above and beyond what the specified requirements were.
POSIX itself is an example of malicious compliance though, since it was all about taking the UNIX API and whitewashing it as a standard for all OSes despite it being obviously intended only for Unixes. They even made the acronym rhyme with Unix, just to rub it in. Which is why POSIX is so limited, once they removed all the bits that were too Unix-y, there wasn't much left. Microsoft made a good call by implementing just the standard to point out the ridiculousness of the standard.
Perhaps *that's* what MSWin's "Malicious Software Removal Tool" is for. Because I install it on a MSWindows system, yet MSWindows is still there afterwards.
The POSIX subsystem is definitely not "batteries ready" UNIX, but you should consider the amount of heavy lifting Microsoft did to make the NT kernel compatible with UNIX architecture. For instance, process creation is very different in Windows and UNIX because Windows has no notion of forking. NT therefore supports an efficient implementation of forking, using copy-on-write, even though Windows didn't need it. Furthermore, NT had to be designed to permit both a case insensitive (for Windows) and case sensitive (for POSIX) naming conventions. I'm fairly confident that NT would have a significantly less complicated kernel if POSIX was not a requirement. They probably wouldn't even have gone with the server/client architecture of having different subsystems if not for POSIX. At the time NT was released, the only subsystems were Windows, OS/2 and POSIX. If they dropped POSIX support, I'm pretty sure they could have found an easier way to do just Windows and OS/2. Even though CYGWIN is a "batteries ready" UNIX, it doesn't take advantage of any of the NT kernel's built-in support for POSIX. This means in principle if you have a UNIX program that does a lot of forking (like a web server), it should run much more efficiently on NT's POSIX subsystem than on CYGWIN. In essence, Microsoft went to the effort to make the cake (everything difficult about POSIX, they implemented) but they didn't bother to frost it or decorate to make it look appealing. It was of course designed for government contractors who presumably had the ability to code up whatever bells and whistles MS didn't provide. One wonders if some government agencies were making use of this back in the day, and we just never heard about it.
Looked up the WeirdNIX contest in old Usenet posts. One of the examples in Donn Terry's 1986 announcement of the contest pointed out that the existing specification considered a single dot to be a valid return value for getcwd().
Here I am, (in Greece)helping with decorating the New Year Cake while taking breaks to watch your contribution to Computer Necromancy. Excellent, fact-packed video. May 2023 bring more of this. You are a gem.
This might be interesting. The subsystem I've heard of existing ever since I learned about NT4 in my teen years, yet never heard about anyone using in the real world.
POSIX subsystem was never mean to be a replacement for Unix but a runtime time environment for applications that had been designed to be posix compatible those applications didn’t make use of all the extended c libraries , Unix tools etc because by design they were meant to be posix compatible. What most commenters and the author should remember is that before nt no other major operating system had ever offered completely different runtimes running on the same kernel out of the box. This allowed specific customers to run these applications side by side with general windows applications. Remember that every other posix compatible os offered it natively but could not run other runtimes requiring customers to have multiple systems to achieve the same results. And the windows resource kit visual studio and related documentation was included with every enterprise level msdn subscription which back then was included in major enterprise agreements with Microsoft. At the first company I you just walked into the it administration office to get these cds. So a lot of the downsides mentioned in this video are really only applicable if you were a hobbyist to smal enterprise in which case this function wasn’t designed for you.
100 times this ^ POSIX isn't there to run some nerd's *nix nethack, it's there to run POSIX compliant software. That's all. I think folk confuse _most *nixes are POSIX compliant_ with _all POSIX machines should be same as some *nix_ POSIX is designed to be _Unix-like_ , not _to be_ Unix. It's very basic, and that's the point of it. [ and aye, as a software dev since the 80s, every company I've worked for from aerospace, thru shipping, to pure IT, has had a full MSDN if they were an enterprise business, it's the way it was - I've got folders and folders of the stuff in my spare room :) ]
3:51 It kind of reminds me of early "compatible" computers that could run MS-DOS applications, but no real software because programs didn't use DOS/BIOS calls for display etc. Now, I also remember there was a company who made something that rounded out the POSIX subsystem and made it truly useful. MS _did_ provide the critical foundation -- stuff that must be implemented in the OS itself -- of having a 'fork' function and providing for case-sensitive file names.
Windows has the equivalent of fork(), but it is expensive to use. For example, back in the days when Git was becoming really popular, Mozilla were looking to switch to a DVCS (I think they were using SVN or something). But they had a requirement that whatever it was would work well on Windows. And Git, at that time, made too much use of fork/exec to work nicely on Windows. So Mozilla went with Mercurial instead. Git has been modified since then, to tone down its appetite for spawning processes, to cater better for Windows installations. On Linux, of course, there was never a problem.
Cygwin has a spawn() function that basically emulates fork/exec with native Win32 calls in the event that that's what you really want to do. But it does provide an implementation of fork that's kind of hilarious in the lengths they had to go to. Obviously it's incredibly slow, but the best part is that because of address space randomization it can sometimes just fail if Windows by chance decided to stick a DLL in the wrong place first. It will retry a finite number of times but there's always a remote possibility that it will just have to give up eventually.
@@lawrencedoliveiro9104 On early Unix systems fork() was also an expensive operation. Since every writable memory page in the parent's address space would be copied to the child. Hence there was vfork() which would share the address space and suspend scheduling of the parent until the child called exec() (or exit()). In more modern systems writable pages are shared and marked Copy on Write.
The real punchline is that Microsoft developed and sold a very popular commercial UNIX versions of the 1980s: Xenix. Even considered pushing it as a replacement for MS-DOS... But then AT&T started selling UNIX directly, and Microsoft recognized pretty quickly it couldn't compete with the company it was already paying licensing fees to, and decided to work on a different successor to MS-DOS - OS/2. As for Xenix? That got sold to a company that was co-developing Xenix with MS: The Santa Cruz Operation, or SCO...
@@watchm4ker I would love to get more history on the NT name, some say it actually came from a version of Windows developed for the Intel i860, code-named N10 ("N-Ten")
@@Rouxenator That, I don't know. I know it used to stand for New Technology, at launch. And that it's... Basically the original version of OS/2 3.0, after Microsoft decided to abandon THAT partnership.
@@adg1355 Only for 1.x text mode programs. Which was actually super useful because OS/2 1.x was a better DOS than DOS, much better for software development. Many dev tools were bound to work under both DOS and OS/2 1.x.
11:27 Remember that Windows didn’t have a decent terminal emulator anyway (and as far as I know, still doesn’t). Microsoft didn’t like the traditional idea of sending inband escape sequences to trigger terminal functionality like cursor addressing, text attributes etc. So they developed their own terminal API that required special system calls to control all these things. And, surprise surprise, nobody could be bothered writing software to use these special system calls.
They do not. And haven't had a CLI text editor since 'edit' in the 16 bit days No wonder WSL is a thing, CLI workflow in the most basic sense is basically non-existent
@@RadikAlice Yes they do have a terminal (console host, using pre-11 nomenclature), a bunch of improvements was added to conhost in 2019, and now Windows Terminal is default on W11
The Windows subsystems for OS/2 and POSIX were supported up to and including Windows 2000. I used the POSIX subsystem on Windows 2000 back in the day, and was fairly satisfied with it, except that it didn't support fork(2) (the call returned -1 with ENOTSUP / ENOTIMPL). Since I needed an implementation of fork(), I created one in native Win32 code (which was pretty ugly but worked). BTW, most of the tools and documentation for Windows POSIX were available on the Internet by the time I used it. The Windows SDK was downloadable from Microsoft as well.
No random and srandom, so no proper C Standard Library support or a really broken build environment. With no networking and very limited interop(ooh shared filesystem), They really didn't want anyone using it. Meeting the bare minimum for government contracts, just like any business would. WSL 1 pretty much ended up doing The Posix subsystem the right way, aiming for actual binary compatibility not just poor-quality source compatibility.
WSL1 isn't binary compatible, it is syscall compatible. It also wasn't the first time it had been done either. Circa 2002 out of MS Research a similar project was done which could run Linux ELF binaries on a Subsystem. To demo the ability they were running an Apache webserver and had Firefox accessing it. It was never going to see the light of day with Balmer leading the company. WSL1 is more like WINE in that the syscalls were handled by Windows and translated on the fly to an equivalent set of Windows API calls. WSL2 is more like And Linux or CoLinux where the kernel is actually running in a VM and is isolated in its own Hypervisor. Each have their own strengths and weaknesses, but WSL2 with how it visualized the file system and hardware is more resound and can keep up with Linux kernel changes in a way that WSL1 can't. The magic is how they've bridged WSL and Windows so that they are both working together with only a few seams if you know where to find them. For most of my Linux needs, Windows is the best Linux system I need and it is easy to switch between different Distros. POSIX on the other hand. It was available for very custom work and not expected for more than supporting those very bespoke customer solutions. The subsystems were initially designed to support more than the Win32 API, but with IBM and DEC usage being significantly anemic, OS/2 and VMS compatibility wasn't a priority. NT 3.1 and 3.5 were intended to offer more, especially on the POSIX side of things seeing as Dave Cutler and many early NT engineers had built VAX for Digital before coming to Microsoft. The bones were there and have been from the beginning -- NT kernel was going to run Win16, Win32, DOS, OS/2, and POSIX applications, each in their own environments -- it was soon clear that Win32 was the Subsystem to design around. I think it would be interesting for Microsoft to revisit supporting more subsystems. There'd be a great MSR project to make modular environments to host things like a Node/Deno container, one for .Net CLR, Android, or a modern LISP machine, with a Win32 host machine providing a way to keep those other systems running in isolation. It would look a little like Docker Containers in some ways, or a NT-based Kubernetes OS. I think Windows 10X was sort of heading that way with Win32 apps running in their own container, and I think it is still a concept worth going after. WSA and WSL demonstrate that it is possible, but it still needs refinement.
FWIW I dont think there was any malicious intention in "sandboxing" the posix subsystem. In Windows the POSIX subsystem sits above the kernel just like Win32 and OS/2. I dont think this was anything malicious just Dave Cutler drawing a natural boundary around the kernel and implementing what was needed. I worked on this operating system and I can tell you that there was a TON of stuff going on in Windows at the time and it was simply the case that OS/2 and POSIX were very quickly not worth spending developer resources on.
No, that's completely fair. I think it got left out of the actual cut, but in the original notes, I actually did compare it to the OS/2 one, which can access NetBIOS, and IPC structures, and has more integration. Like, you could actually use the OS/2 subsystem as a migration path from OS/2 server to NT, and Microsoft's own pre-NT 3.1 early NT toolchain was OS/2 based and ran through the subsystem.
@@NCommander Totally. Around this time we were building Window 95 with direct X, then merging all of that code into the NT codebase to produce Win2k, figuring out how to service the OS, doing the IE thing, I mean the origin of Windows was a rocket ship. It was a very diff env from today, tbh noone particularly cared about POSIX. Linux was nothing even close to what it is today (a trajectory which changed dramatically when Google invested in Android) and it was definitely not a high priority. The situation has changed 1000% and WSL is a much more committed attempt and IMO absolutely critical for Windows' - survival given the importance of Linux today dev machines need to be good to work with Linux and OSX has an advantage there. But IMO Windows has made great strides here e.g. passthrough GPU virtualization for WSL usage is a major deal for machine learning workloads. Anyway im rambling.... Awesome video, awesome channel, thank you - subscribed!!
On a technical level, it would've been impressive if it was completed and documented properly. The NT Kernel has support for subsystems besides Win32. Would've been fun to actually use it. Ah well.
I bought a copy of Services For Unix back in college where I used Unix and NT workstations in the engineering computer labs and ran Windows on my dorm PC. Writing portable c++ code back then was frustrating, especially not standard Unix filesystem structures for headers/libs.
Nice! Wasn't this a big deal at the time? This is crazy stuff! As an economic history researcher, I swear this story deserves a paper, if there isn't one already. Subbing right now! ❤
4:56 tar is not a single format and I'm guessing you were trying to unpack a GNU format tar file. With GNU tar, you might have better luck creating archives with one of "--format=posix", "--format=ustar", or even "--format=v7".
Windows terminal stuff has always been atrocious and it was the first thing I thought of when you mentioned Rogue. Even now it's quite bad, though it has improved significantly. It's still decades behind
The problem with IE4/Active Desktop is just how resource-hungry it is compared to the original incarnation of Explorer. Plus, on 95 at least, the shipped IE 5.5SP2 runtimes make it leak memory. To each their own, but anywhere the original is an option, I take it.
Back in the day I used active desktop to have a live stream of some kittens playing in a cat shelter stream from a webcam as wallppaer. It used flash and you had to restart explorer everyone once in a while to fix the memory leaks.
@@lucass8119 It's actually really useful for debugging when you are doing iteration on Windows but the target device uses a case-sensitive filesystem. You can set the folders as case-sensitive and catch those bugs early before trying to deploy and debug on the target hardware.
UNIX and companies that sold UNIX were their own worse enemy. One of the reasons it was the POSIX standard and not the UNIX standard. X-Windows is another such standard that was crippled from day one. Many UNIX based companies also were not POSIX compliant. Microsoft was hardly alone in that regard. The world wanted a graphical OS that didn't cost $2000 per year per work station. IBM, HP, Solaris and the others charged ridiculous prices for UNIX and hardware. Linux at the time was a hobbyist toy and wasn't an option. Today people get software almost free but in the 1990's it wasn't cheap and Windows NT cost 1/10th of other OS's. Not a small thing when you have 1 million employees who need computers. POSIX might have been a government specification, but the reality was it was only used by a few people in IT. 95% of computers were used for email, Word, Wordperfect, Lotus and Excel. Managers and IT departments wanted a single type of OS to support, or at least that seemed the vision. The reality was/is that different use-cases required different hardware, different OSs and different IT support. Even at the time people were aware of the limitations of NT and POSIX. They didn't care as long as the legacy box that only mattered to the old guys running mainframes was ticked. Remember, UNIX was really only run on mainframes or very expensive workstations at the time. Google wasn't even a twinkle in the eye of Silicon valley. It was only later in the 2000's when Linux was mature enough to save UNIX from obscurity that UNIX compatibility became important. Today things are much different. So it isn't really fair to judge MS from the 1990's. POSIX wasn't all that important and it looked like UNIX was going to die a slow death. If not for Linux, it probably would have.
Would love to see a comparison to the later Unix subsystems in NT: SUA/Interix and Windows Services for Unix (SFU). And maybe a side look at Microsoft's own standalone Unix implementation, which also influenced these efforts: Xenix.
And I thought the RS/6000 shipping with reference implementations of core features was bad (which it is) but this makes that preferable, I bet that if Linux and open source hadn't taken off in the business world we'd still be dealing with this kind of unsubtle yet subtle enough middle finger towards UNIX as a whole
“Unix” is a trademark that only proprietary products can afford to license. And all those products are either dead or dying right now. What matters nowadays is not “Unix”-compatibility, but “Linux”-compatibility.
@@lawrencedoliveiro9104 While I understand your point, the wording made me roll my eyes and kinda want to say _something_ out of spite It feels insufferably pedantic, but maybe that's just a gut reaction from a bad first impression
It’s like they assigned a bunch of project managers to ship a bunch of components and when it came time to integrate it in a reasonable fashion - some higher up must have said “eh…… what’s the ROI?”
I saw mention of the posix system in Windows NT back in 1993. It never even occurred to me to go check it out, since I had access to Unix. What would have been the point?
@@NCommander I've been in software for 40+ plus years, an I have been burnt many times by emulating other peoples environments. Its always a moving target, and its really amazing how common it is for programs to skirt the rules of an interface and take advantage of undocumented quirks of the interface. This is one reason why I am not a fan of either Wine or the Linux on windows subsystem. If I want the Unix/linux environment, I'll just go use Unix/Linux. If I want a program to port to a different environment, then recompiling it for the new environment is the way to go. For programs that cross multiple platforms, that's why god made cross platform porting kits. Put it this way: a program designed for porting via a kit had portability in mind from day one. A program written for a given interface (Windows/Linux/Mac) was designed to be bedded in cement from day one.
I’m not sure if the complaints about requiring those “obscure” CDs make sense from a 1990s perspective. Pretty much nobody really needed the POSIX stuff (developers and users alike), and disk space was at a premium, so relegating those to separate CDs/products seems reasonable to me.
Very cool video...didn't know a posix subsystem existed for windows NT...I only found posix on c functions that looked like yea that might exist on say unix...and it would say depreciated...and I was like ...yea I am using that...for it was something simple I was familiar with...just needed to refresh on the parameters, etc. I guess not knowing about this was a good thing...kept me from attempting it when I was moving from DOS to Windows NT...and didn't know much about how to get there...I found the win32 library interface and I was like ok...winner going with that...worked out ok...well less painful than anything like this.
You mentioned a book "The Unix Haters Handbook" in a video of yours, one of the solaris ones maybe, anyway, I had to tell you this book is absolutely hilarious. I haven't laughed this hard reading something in years. Thank you.
I have a feeling that the relevant FIPS standard was written with a close guidance from someone with Microsoft's interest at heart (or in his pocket more likely). So reaching the bare minimum for compliance was easy, even though the implementation was a complete joke and an insult.
I really like your channel, it has V I B E S. IDK how else to put it. Something about the music and the calmness of your voice and the highly interesting subject matter. Really great, thank you! :)
Wanting to say compliant on the box but doing as little as possible to make it so, and making it so damned hard to have any real practicality as a move to make such compliance a thing of the past.
SFU is much more complete and better than this. Not only it provides much more Unix utilities and POSIX API but later versions even contains X11 libraries. SFU is much more close to something made properly than this thing ever was.
Well in the those days there was a lot of arcane stuff to get a Linux system up and running as well (i still remember modelines for the display driver or wading through long lists of kernel options "fondly"), but at least there was some documentation (read those READMEs) and it didn't have this feeling of malicious intent.
As a counterpoint, you should try out OpenSTEP YellowBox NT. It's basically all of OpenSTEP (fka NeXTStep) 4.2, both GUI apps and command-line, but running under NT 4. (I once had it installed under XP, as well.) It's pretty complete for 1996-97, but good luck building modern stuff. Ironically OpenSTEP 4.2 (including OpenSTEP Mach, the full OS which was derived from NeXTStep and later evolved into OS X) is worse at this than NeXTStep 3.3 in some cases, as the POSIX compatibility is more broken.
We've been having a long discussion on it on Discord, and how it works (which is abusing cygwin). It's on my to cover list as a stream if nothing else.
Good 'ol malicious compliance. I think I'm preaching the choir here when I mention embrace, extend, and extinguish as MS's modus operandi back then, hell, they coined the term.
Boo at Microsoft for doing the bare minimum but what'd you expect from them back then ;) The Resource Kit, however: I don't know anyone that used NT 4 that did not have it installed. If you used NT Workstation at home you were pretty much sysadmin-level to begin with, had a work MSDN subscription, and knew what was out there.
This video gave me flashbacks when Microsoft decide not only to implement ODF in the most half backed way inimaginable but also publish it's own OOXML format as a standard in the most useless way to make difficult the implementation by the competition 😬😵
Yes, my thoughts exactly. And didn't they send their own people to the standards meeting just to vote for them, and never show up again? I remember that causing a problem because they couldn't get a quorum or something. IAC, what they shipped was the original version without any of the feedback fixes needed by the eventual standard.
Microsoft itself does not support ISO 29500 (OOXML) properly. Microsoft -Office- 365 still only implements it in “Transitional” mode, since “Strict” mode apparently causes too many compatibility problems.
For a while back in the early days of those standards I remember discovering that Word's implementation of the ODF appeared more true to the official standard than OpenOffice Writer's was. This was because Writer had already progressed to a later version of its native ODF format by the time it became standardized, which had more extended/updated features. I cannot recall that it had any way of saving/exporting to the older but standardized version. Word's way of supporting ODF was rather cumbersome though, as one had to install and deal with it as a separate plugin. Conversely OOXML has always been a mess from its very inception, allowing for including BLOBs in proprietary formats that only Microsoft itself ever has a chance of fully supporting.
Is this the reason Windows has system32/drivers/etc/hosts? I've always thought this implementation seemed very at odds with everything else in Windows.
The original Windows NT 3.1 TCP/IP stack was a direct port of the one from BSD 4.3 by Spyder Software. I've always assumed it was a holdover from that, although having a hosts file is de facto standard (and might even be in one of the RFCs as required).
I was interested in that thing but never actually had time and motivation to try this out. Now after your video I know that it's not a bad thing that I didn't try. Good thing that it was replaced by SFU which was much more complete and useful. Also I used Cygwin and it was pretty convenient and useful as well. I was even able to run some Unix desktop environments with it and Cygwin/X.
I'm sort of tempted to get a VM setup with the POSIX environment on my end so that I can try and run neofetch in it so my guess is that I would have to get bash working in this environment
This may not be the case, but I read that the reason NTFS supports hard links (that is, more than one directory entry pointing to the same file) is that they had to implement this POSIX subsystem. Hard links are _so_ useful. For example, prior to a large scale batch rename, I can clone a directory tree without duplicating the file contents. (This uses cygwin btw.)
Don't even need cygwin. Lookup "Link Shell Extension". NTFS supports both hard and sym links. And it has for quite some time (NT4!). That plugin has some cool functionality they built on top of the base functionality, so it's definitely worth a look.
@@ccoder4953 hell you don't even need to install anything, the standard command prompt windows ships with has a mklink command for doing this though having it as a right-click context menu is _very_ handy if you make links a lot
@@lawrencedoliveiro9104 Actually, we're both sort of wrong. According to Wikipedia: "Symbolic links to directories or volumes, called junction points and mount points, were introduced with NTFS 3.0 that shipped with Windows 2000. From NTFS 3.1 onwards, symbolic links can be created for any kind of file system object. NTFS 3.1 was introduced together with Windows XP, but the functionality was not made available (through ntfs.sys) to user mode applications. Third-party filter drivers - such as Masatoshi Kimura's opensource senable driver - could however be installed to make the feature available in user mode as well. The ntfs.sys released with Windows Vista made the functionality available to user mode applications by default. " As far as the special permissions thing: "The default security settings in Windows Vista/Windows 7 disallow non-elevated administrators and all non-administrators from creating symbolic links but not junctions. This behavior can be changed running "secpol.msc" the Local Security Policy management console (under: Security Settings\Local Policies\User Rights Assignment\Create symbolic links). It can be worked around by starting cmd.exe with Run as administrator option or the runas command. Starting with Windows 10 Insiders build 14972 the requirement for elevated administrator privileges was removed in Windows "Developer Mode", allowing symlinks to be created without needing to elevate the console as administrator. On the API level, a SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE flag is supplied for this purpose" So, the policy can be changed and it appears Microsoft will soon make the change the default.
Hi. What I remember, but back in win2k world and appropriate MV VS, there was an option to configure a project output to be for Windows POSIX subsystem, as just one setting. I tried it literally once or twice, it worked, but there was absolutely no practical usage.
6:04 Being able to traverse a directory (e.g. CD into it or one of its subdirectories) is a separate right from reading (listing directory entries) and writing (creating files) -- I think Unix-likes use the "execute" bit for it, and Windows NT and NTFS do the same in their ACLs. However, looking up this permission for every ancestor directory in addition to the permission of interest for whatever file you're working with takes time, and since the concept didn't even exist on old-technology Windows there was little confidence software would get it right. So all users were by default granted the right to bypass these checks: if they have permission to a file, they have permission to a file, regardless of some execute bit on some parent directory. Needless to say, that's not POSIX-compliant. While the government would have to revoke that right (compliance!), it's irrelevant to what you're doing so feel free to leave it on.
What if you strip the NT kernel down to a DOSlike prompt, and try to set it up such that the POSIX subsystem is the only available command set? I don't know exactly how close you can get to this, but it seems like a gloriously cursed idea to me.
In theory it's easy to replace explorer.exe with anything else (for example cmd.exe or some other terminal app, maybe even written by hand to have an illusion of full screen text mode) and stick to that. Having a "pure" text-mode out of NT would be tricky though and I'm not sure if possible on retail systems, given how system is bound to GUI and various low-level service processes that controls user session & login.
You cannot run POSIX subsystem (PSXSS) without the Win32 subsystem (CSRSS). You can, however, have a pure command line installation of Windows (e.g. the Embedded toolkit allows you to build one), where you can easily replace the command interpreter with sh. Might be an interesting idea to combine with NT Embedded.
@@dr.shuppet5452 It would be probably the most limited "UNIX" system ever 😀But could be interesting with Cygwin (for older Windows) or WSL for newer ones...
@@dr.shuppet5452 You could have text mode Windows installs, if I remember right, all the way until Server 2019, where iirc the option to not include the desktop in the install was removed. Text mode Server 2016 is a very limited experience, as the majority of modern Windows-centric server utilities, or at least the ones I've used, have major GUI components with either minimal or no terminal interface. Maybe, to commit to the cursed idea as much as possible, the system PATH variables could all be cleared, assuming the system uses hardcoded directories instead of PATH during boot.
@@dr.shuppet5452 Is that a hard dependency, or more of the fact that you wouldn't be able to launch anything? In theory, you probably could create a native mode application (runs before csrss starts) that starts psxss and messages with it, not that it would be a sane endeavor at all, given the absolutely nonexistent documentation on writing native mode applications, and the RE work that would be required (unless you could find relevant bits in the many leaked sources over the years) (internal details below, probably shouldn't look if you want to contribute to Wine or ReactOS) Edit: my current findings are the dependencies are mainly on SMSS (which should be already running) and on whatever provides the SAM database, and probably a few others. HOWEVER: smss supports running subsystems under other subsystems, and that's precisely what psxss seems to be. It's not a native mode subsystem, it, as a subsystem, appears to be set up to run under the Windows subsystem. Unlike csrss, smss, which have UMTYPE=ntss, and autochk (which has UMTYPE=nt), psxss has UMTYPE=windows. So you are correct.
I've been livestreaming, but making video content has been difficult. I've made some headway on a new video, but its been very up and down in real life
I think this illustrates the problems with POSIX more than it illustrates the problems with Microsoft's practices (although perhaps not by much). Edit: For an amusing historical look at the UNIX community's own problems with POSIX (as well as many other things), try tracking down a copy of The UNIX-HATERS Handbook. POSIX was NOT sufficient to properly address the problem it was intended to solve.
so much wrong in this video: the windows SDK and reskit were extremely common for windows developers to have: you could download them for free off of MSDN, and the MSDN CD bundles came with them. in fact, for most win32 application developers the windows SDK was usually the primary source for windows compilers, headers & libraries since they were always up-to-date with the latest OS release whereas the MSVC headers often lagged behind. the only real reason to use the MSVC-supplied compiler is if you wanted to use the crt or MFC libraries which weren't included in the windows SDK (they're not part of windows itself), also MSVC isn't required for POSIX development, the windows SDK is sufficient. you have to remember that the posix subsystem fo NT wasn't designed to be an interactive system - windows already has a UI/shell. it was simply designed to run applications, usually background services. the utilities that are provided are simply there because they're mandatory in the posix standard. it's a little strange that you call something that literally implements 100% of what it's required to implement "half-baked". are you saying that posix is half-baked?
I feel like this could have all been avoid with a generic ‘no, don’t even think about it’ clause. MS: what if we just…. POSIX: No MS: But how about… POSIX: (slaps MS with a wet fish)
its funny you say this because there are actual email exchanges dated 1990 between the posix committee and the NT team in leaked windows source which basically go like this
lol what a complete mess. Appreciate this kind of dive into such a topic though, it was always something I was vaguely aware of but never researched the true extent of "POSIX compatibility" in Windows NT, never heard of anyone using any such functionality and never used it myself. I admit though that in modern day Windows it is rather nice to not be constantly thrown off by using Unix commands like I used to be in the old days, I still have no pressing need for things like Linux application compatibility but to not need to make that switch between Unix and old DOS commands is much appreciated, that was quite annoying in the past.
Even the Linux boot/rootdisk in the 0.11 kernel days had massively more functionality than this. When it came to SFU, I think SLS (Linux distro from ~1993) was considerably more advanced than SFU. Using SFU felt like using a 1980s version of ISC Unix.
This was part of the original architecture of Windows NT: there was a core kernel, on top of which you could have multiple “personalities”, each with their own userland API. Win32 was itself one of these “personalities”, and so was the POSIX layer. In the early days, Microsoft did let the documentation get out on how to write your own personality, and the Interix folks got hold of this. Then management changed their minds and decided to restrict this info. So they bought up Interix to prevent the details getting any further, and the Interix product became Microsoft SFU. You’d think present-day WSL would be built on the same architecture, but it is not. WSL 1.0 might have been built that way, but they could never get it working right. Which is why WSL 2.0 falls back on good old virtualization instead. And still doesn’t work completely right. So one could say that the original “personality” architecture has pretty much bit-rotted away.
@@lawrencedoliveiro9104 You meant I can't run my OS/2 1.1 and 1.0 apps anymore? And yes. That was another personality. And yes, I'm still bitter about OS/2.
CORRECTION: The resource kit was apparently a lot more available than I realized. I knew it was in the book, but I didn't imagine the book was *that* common. I never saw it at any of the shops I worked, but that's a small sample size.
I worked as admin in a small lab, the Resourse Kit seemed to be standard must have at the time. The books filled up half a shelf, and occasionally helped, but that's not why you bought it.
I remember the NT Resource Kit well. I was an NT admin back in the 3.51 / 4.0 days and that kit was indispensable. The book was also riddled with errors - documentation that was flat-out wrong. "It works like this!" No, Microsoft, it does NOT work like that.
The POSIX subsystem for Windows was superior to WSL in one very important respect: the name wasn't backwards like "Windows subsystem for Linux" is.
Yeah, WSL sounds like a Wine competitor, not the opposite 😅
Yeah, I was going to say... a Windows subsystem for Linux sounds like a way of running Win32 applications on Linux.
technically i think the grammar is correct, but regardless it does intuitively sound backwards indeed
Windows Subsystem for Linux
the Windows Subsystem (the subsystem that exists within windows) that is used for running Linux
@@shallex5744 yeah, I get that’s what they were going for, but it’s still backwards. Anywhere else, the noun adjunct to “subsystem” describes what the subsystem does, and “for” indicates what subsystem is meant to be a part of/run on. It’s not a Switch emulator for N64, it’s an N64 emulator for Switch, for example. Even if the grammar is technically ambiguous (it probably is, English is not especially precise), WSL is still strictly backwards to the standard convention used in any other similar context.
@@Thanatos2996 yes, it is a very strange name choice indeed
I guess for people wanting to use the POSIX subsystem back then, enterprise grade customer support from M$ was basically a necessity.
It's been a long time since I had to fool with it, but iirc "enterprise support" didn't get you far and in our case, lead to migrating to an entire MS shop against the wishes of everybody who didn't have their hands in the piggy bank. Which if you boil it down, that seems to be the intended result anyway.
@@randomtidbits7695 can confirm that this was exactly how it went at the law firm that I used to do IT work for
Enterprises which wanted to use POSIX software on Windows NT were very rare and those that did used one of the variety of 3rd party solutions on the market and not the POSIX subsystem. These 3rd party solutions mostly targeted software companies looking to provide their product to Windows only shops. Most POSIX software running in Windows back then was 3rd party software from these companies and the terms of the POSIX solutions allowed distribution of their software with the POSIX runtimes.
M$ does not support.
There were some major bugs reported to Microsoft in their SNA implementation (another embrace and extinguish, but then again, who really wants SNA).
Bugs were send with fixes and reproduction instructions to their enterprise support.
6 months later a reply came that they found it was a bug and that they will look for a solution.
That's how enterprise grade their support was.
@@bloepje SNA Gateway was a particularly bad product. Many of my employer's customers at the time unfortunately didn't discover this until replacing the reliable Novell SAA with it. It was so bad but also so key to my employer's customers accessing our primary product that eventually my employer was able to strike a deal with Microsoft. They loaned some of their developers to Microsoft to help improve SNA Gateway.
Besides the poor quality of that particular product my experience with Microsoft support was ok, but at that time my customers were also very large enterprises or large government agencies which had large expensive contracts with Microsoft in order to get a high level of fast support.
I've always kinda knew this subsystem existed, but wow I didn't realize it was this cursed. Crazy to think that presumably many many hours were put into designing the subsystem, only for it to be basically unused. I wonder what it was like for the Microsoft developers back then.
NT has a native subsystem capability that was leveraged for this. Microsoft simply added another subsystem, although it certainly took a long time to get it going. Maybe there were plans to improve it, maybe there wasn't
Microsoft replaced it with "Windows Services for UNIX", which was SUS compliant (probably not getting away with that half baked POSIX bs anymore), and then WSL1, and then WSL2 (which is literally a linux VM running inside hyper-v with some clever stuff to get it communicating with windows)
Btw, NTFS is also case sensitive, fwiw, windows just...kinda kludges around it
The devs probably not only knew but fully supported developing this "malicious compliance" POSIX subsystem.
Those "many hours" were spent figuring out how they could do as little as possible
This isn't just half-baked, it's god darn frozen.
This beast's still walking in the forest, that's how unbaked it is.
And it wasn't even a dessert.
Its not even there its literally just assorted piles to create a beast to walk around
@@RemnantCultThe beast's name? Windows NT.
it was just a jump for money, so why bother even unfreezing it?
I worked with the NT beta versions and release versions from 3.1 onward, porting a product which ran on various Unixes. We initially thought we'd be able to use the POSIX subsystem, but it was quickly obvious that this was nothing but a box ticking exercise. Our product which ran on NT, on MIPS, alpha and x86, was completely win32 with no use at all of the POSIX subsystem.
I worked on a project in 1995 where management had the vision that we should have a common codebase for Unix and NT. The team actually tried to use the POSIX subsystem whit the same success as showed in the video... Our solution was to write our own libraries as a version of pthread for NT and not to tell management. 😁
Embrace Enhance Extinguish
Let me guess… outlook express?
@@vkvishnu Nothing remotely like that. My work was on a PostScript interpretor for industrial printers (film recorders, A0 sized printers etc). Our product already worked very well on a range of UNIX workstations (SGI, Sun, Digital etc) and porting to other UNIX machines was easy enough because they had a common (POSIX + X11) API, but the Windows port was very much more complicated than expected because the POSIX subsystem was basically completely useless. Therefore we had to write a load of windows specific code.
The UNIX API begun in the early 70s, when the system was ported to C. POSIX provides a standard for this API, as many UNIX systems got further and further apart from the AT&T sources over time. Currently POSIX means at the very least you can run console apps on Linux, *BSDs, and Mac
tbh I'm not surprised this is such a mess, I've never had POSIX accomplish what it was supposed to. Anything beyond basic apps has always required a pile of ifdefs even for stuff like memset/memalign/malloc. They didn't even add endian swapping functions until *2014*... for a standard which is literally centered on standardized IO.
@@ShinyQuagsire it’s way better than nothing though
POSIX is C source code level. Unix API is some kernel calls wrapped by standard library.
@@ShinyQuagsire I think you are conflating Windows adherence to the POSIX standard with the actual POSIX standard.
The reason why WSL exists is because of Windows attempting POSIX compliance.
Yup, posix compat means compiles on both AIX and Linux even though AIX feels like a barren shithole compared to gnu linux
“It just existed so MS could claim POSIX compliance” yeah, pretty much how most people knew at the time as I recall
Back in the 90s I used to work for a company that implemented Windows NT and for a while had the largest NT network in Europe. I was aware of the POSIX subsystem which I understood was present to allow sale of the product to US Government entities that had strict policy on procurement where certain features had to be available regardless of use-case. Another feature was C2 security which Windows NT complied with although security capabiltiy has been enhanced since then. I recall anecodally that some software vendors had made their previously Unix targetted software available on NT using the POSIX subsystem but never saw it myself. I had access to Technet and MSDN so had the resource kit and SDK as well as development tools and there was loads of interesting stuff included. For example the source for the built in screen savers using OpenGL library. At that time Cygwin didn't exist and even doing development on Unix/Linux relied on the runes aligning to setting up Windows for this wasn't vastly more complex!
Yeah, I remember M$ bragging about that C2 compliance from the very start during the presentations of NT 3.1, and how hard work it was to get that certified. Later I found out that this approval was only valid for a certain subset of hardware/software components, and anything with a network (even just LAN) was _not_ C2 compliant. Even back then running without some form of network was pretty useless.
Considering that at that time Microsoft has already have enough experiences with developing XENIX, the half-baked POSIX implementation were deliberate moves by Microsoft.
I always wondered what the POSIX subsystem option was in Visual Studio's linker options for C++ projects, I guess it still has some support even in Windows 11 and VS 2022, how interesting.
The linker flag specifically sets the subsystem file in the PE header; I don't think it affects anything else besides that, but I'm not 100% certain.
The idea the resource kit was obscure or hard to get a hold of is just incorrect. If you wanted the book along with it every bookstore carried it in their technical book section. It was advertised and well documented in magazines of the era. It was available for free via the Microsoft BBS and the Microsoft FTP site. It was very commonly installed on NT systems.
I worked in IT in this era. I remember the 9x ones. The NT one was rarely, and it didn't note that the POSIX tools were included. The book itself doesn't even list that; I had to check the disc myself. The book just lists traverse checking has to be disabled, and nothing else.
@@NCommander I worked in IT during this era. Your recollection maybe that the NT resource kit was rare but it certainly isn't mine. It was everywhere.
My brother and I had the NT 4resource kit though he had to order it. A few years later I was able to get the 2000 resource kit but that was a lot more common..
I love the NCommander In Realtime series. Your livestreams are a bit long for my attention span, so I do like these re-narrated shorter versions, since it lets me see what curse-ed things you're doing... and I totally love the mindset. "This thing is available, I wonder if it's usable, it's probably not but they said it worked so we gonna try it". I always knew the POSIX compatability was kinda trash but you've shown it's total trash, thank you for sinking hours into this, great to watch!
My short attention span appreciates these live stream summaries, too.
POSIX on Windows is so funky, the default C library (msvcrt) fools you into thinking it might be POSIX compliant until you actually try to use it, add the 10 official implementations of POSIX, each for a different version of Windows, third party libraries like Cygwin and MKS, as well as Microsoft simply integrating a Linux kernel into the OS.
How’s POSIX on Mac?
@@MaxOakland _so_ much better. fully functional and works like you would expect it to.
@@adora_was_taken That’s awesome!
@@MaxOakland macOS is weird since it combines Old Macintosh, NeXT, Unix, and Mach operating system heritages. Macintosh gives you the GUI style and file-systems, NeXT gives you the premier programming API, Unix gives you the more standard C API and file-system structure, and Mach gives you some of the kernel and executable file format
"Microsoft simply integrating a Linux kernel into the OS"
Do you mean WSL1? That was the best one IMO.
Fun fact : Linux isn't actually POSIX-compliant. Stallman thought some of the decisions of the norm were stupid and never actually got the certification. You CAN make it essentially compliant with the variable POSIXLY_CORRECT or POSIX_ME_HARDER, but it's not official, making macOS the only fully POSIX OS outside of "real" Unix like AIX or HP-UX.
"POSIX_ME_HARDER" is the funniest variable name I've ever seen.
@@LonelySpaceDetective For some reason, I want to add DADDY at the end of it.
A couple of Linux distros (specifically, EulerOS from Huawei and Inspur K-UX, both RHEL derivatives) got UNIX 03 certification at one point, proving that Linux indeed _can_ be made "fully compliant" if needed.
There have been 2 UNIX-certified Linux distros tho.
There were few Linux distributions that passed Single UNIX Specification tests and were registered as UNIX systems.
Reminds me of the time when Java applets were a thing, Microsoft shipped their very own Java Virtual Machine. It was so bad that Sun sued them and eventually won.
This lawsuit eventually brought us C#.
Uh, assuming it did pass the NIST test suite, then the problem was with one or more of: the standard, the test suite, or people's expectations. Undocumented, 'soft' standards are a problem.
Agreed. If the government wanted to be able to build rogue on Windows NT, the requirements should have included the pieces needed for it. I can't fault Microsoft for not choosing to go above and beyond what the specified requirements were.
The literal definition of malicious compliance.
Ironically Windows NT is a fork of OS/2 there joint project with IBM. It's likely it's posix compliance was also this messy
@@JeffreyPiatt NT is not a fork of OS/2, its a nearly 100% independent codebase
POSIX itself is an example of malicious compliance though, since it was all about taking the UNIX API and whitewashing it as a standard for all OSes despite it being obviously intended only for Unixes. They even made the acronym rhyme with Unix, just to rub it in. Which is why POSIX is so limited, once they removed all the bits that were too Unix-y, there wasn't much left. Microsoft made a good call by implementing just the standard to point out the ridiculousness of the standard.
@jankrynicky no, it is literally a fresh reimplementation, the only thing taken directly from OS/2 is the command interpreter
Perhaps *that's* what MSWin's "Malicious Software Removal Tool" is for. Because I install it on a MSWindows system, yet MSWindows is still there afterwards.
The POSIX subsystem is definitely not "batteries ready" UNIX, but you should consider the amount of heavy lifting Microsoft did to make the NT kernel compatible with UNIX architecture. For instance, process creation is very different in Windows and UNIX because Windows has no notion of forking. NT therefore supports an efficient implementation of forking, using copy-on-write, even though Windows didn't need it. Furthermore, NT had to be designed to permit both a case insensitive (for Windows) and case sensitive (for POSIX) naming conventions.
I'm fairly confident that NT would have a significantly less complicated kernel if POSIX was not a requirement. They probably wouldn't even have gone with the server/client architecture of having different subsystems if not for POSIX. At the time NT was released, the only subsystems were Windows, OS/2 and POSIX. If they dropped POSIX support, I'm pretty sure they could have found an easier way to do just Windows and OS/2.
Even though CYGWIN is a "batteries ready" UNIX, it doesn't take advantage of any of the NT kernel's built-in support for POSIX. This means in principle if you have a UNIX program that does a lot of forking (like a web server), it should run much more efficiently on NT's POSIX subsystem than on CYGWIN.
In essence, Microsoft went to the effort to make the cake (everything difficult about POSIX, they implemented) but they didn't bother to frost it or decorate to make it look appealing. It was of course designed for government contractors who presumably had the ability to code up whatever bells and whistles MS didn't provide. One wonders if some government agencies were making use of this back in the day, and we just never heard about it.
Looked up the WeirdNIX contest in old Usenet posts. One of the examples in Donn Terry's 1986 announcement of the contest pointed out that the existing specification considered a single dot to be a valid return value for getcwd().
Oh awesome, which group? I went looking but couldn't find anything but references to references.
Here I am, (in Greece)helping with decorating the New Year Cake while taking breaks to watch your contribution to Computer Necromancy. Excellent, fact-packed video.
May 2023 bring more of this. You are a gem.
"Computer Necromancy", I love this term and will be stealing it 👍
The resource kit was readily available. MSDN subscriptions included it. I do agree with you 100% on the posix mess that MS shipped.
I put a pinned comment about this.
A TechNet subscription (for SysAdmins, not developers) should have also provided the necessary resource kits (but not SDK's).
This might be interesting. The subsystem I've heard of existing ever since I learned about NT4 in my teen years, yet never heard about anyone using in the real world.
Because, for all intents and purposes, nobody did.
I saw one extremely edge case once, but that's it.
Also, it was executing a file that was not .exe
POSIX subsystem was never mean to be a replacement for Unix but a runtime time environment for applications that had been designed to be posix compatible those applications didn’t make use of all the extended c libraries , Unix tools etc because by design they were meant to be posix compatible. What most commenters and the author should remember is that before nt no other major operating system had ever offered completely different runtimes running on the same kernel out of the box. This allowed specific customers to run these applications side by side with general windows applications.
Remember that every other posix compatible os offered it natively but could not run other runtimes requiring customers to have multiple systems to achieve the same results.
And the windows resource kit visual studio and related documentation was included with every enterprise level msdn subscription which back then was included in major enterprise agreements with Microsoft.
At the first company I you just walked into the it administration office to get these cds.
So a lot of the downsides mentioned in this video are really only applicable if you were a hobbyist to smal enterprise in which case this function wasn’t designed for you.
100 times this ^
POSIX isn't there to run some nerd's *nix nethack, it's there to run POSIX compliant software. That's all.
I think folk confuse _most *nixes are POSIX compliant_ with _all POSIX machines should be same as some *nix_
POSIX is designed to be _Unix-like_ , not _to be_ Unix. It's very basic, and that's the point of it.
[ and aye, as a software dev since the 80s, every company I've worked for from aerospace, thru shipping, to pure IT, has had a full MSDN if they were an enterprise business, it's the way it was - I've got folders and folders of the stuff in my spare room :) ]
Technically for a short while, Windows had: NTVDM - 16 bit, POSIX (lol), Win32 together. Technically also could run DOS apps.
3:51 It kind of reminds me of early "compatible" computers that could run MS-DOS applications, but no real software because programs didn't use DOS/BIOS calls for display etc.
Now, I also remember there was a company who made something that rounded out the POSIX subsystem and made it truly useful. MS _did_ provide the critical foundation -- stuff that must be implemented in the OS itself -- of having a 'fork' function and providing for case-sensitive file names.
Windows has the equivalent of fork(), but it is expensive to use.
For example, back in the days when Git was becoming really popular, Mozilla were looking to switch to a DVCS (I think they were using SVN or something). But they had a requirement that whatever it was would work well on Windows. And Git, at that time, made too much use of fork/exec to work nicely on Windows. So Mozilla went with Mercurial instead.
Git has been modified since then, to tone down its appetite for spawning processes, to cater better for Windows installations. On Linux, of course, there was never a problem.
Cygwin has a spawn() function that basically emulates fork/exec with native Win32 calls in the event that that's what you really want to do. But it does provide an implementation of fork that's kind of hilarious in the lengths they had to go to. Obviously it's incredibly slow, but the best part is that because of address space randomization it can sometimes just fail if Windows by chance decided to stick a DLL in the wrong place first. It will retry a finite number of times but there's always a remote possibility that it will just have to give up eventually.
@@lawrencedoliveiro9104 On early Unix systems fork() was also an expensive operation. Since every writable memory page in the parent's address space would be copied to the child.
Hence there was vfork() which would share the address space and suspend scheduling of the parent until the child called exec() (or exit()).
In more modern systems writable pages are shared and marked Copy on Write.
I remember using 4DOS at that time. I don´t know how much POSIX-compliant it was, but it extended a lot the scripting functionality of DOS and OS/2.
"While I could bore you with more dry, technical mumbo-jumbo" I think the title already weeded anyone that would be bored by that out.
You know, this is a fair point.
I was but a starting developer in 2001, but this is an eye opener! Glad M$ came a long way since then - through no choice of their own it seems.
The real punchline is that Microsoft developed and sold a very popular commercial UNIX versions of the 1980s: Xenix. Even considered pushing it as a replacement for MS-DOS... But then AT&T started selling UNIX directly, and Microsoft recognized pretty quickly it couldn't compete with the company it was already paying licensing fees to, and decided to work on a different successor to MS-DOS - OS/2.
As for Xenix? That got sold to a company that was co-developing Xenix with MS: The Santa Cruz Operation, or SCO...
@@watchm4ker I would love to get more history on the NT name, some say it actually came from a version of Windows developed for the Intel i860, code-named N10 ("N-Ten")
@@Rouxenator That, I don't know. I know it used to stand for New Technology, at launch. And that it's... Basically the original version of OS/2 3.0, after Microsoft decided to abandon THAT partnership.
@@watchm4ker There's no relationship between OS/2 Warp and Windows NT.
@@watchm4ker Ah yes, back when getting a Unix box at home was scathingly expensive. Linux really did change everything.
Microsoft: "See, look how terrible POSIX is. You should just give us a no bid contract to deploy Windows APIs."
Windows NT is still Posix compliant like it's sibling os/2
@@JeffreyPiatt OS/2 was not POSIX compliant.
"it's" is a contraction for "it is". "its" is the pronoun.
@@JeffreyPiatt NT had a os/2 subsystem, BTW
@@adg1355 Only for 1.x text mode programs. Which was actually super useful because OS/2 1.x was a better DOS than DOS, much better for software development. Many dev tools were bound to work under both DOS and OS/2 1.x.
11:27 Remember that Windows didn’t have a decent terminal emulator anyway (and as far as I know, still doesn’t). Microsoft didn’t like the traditional idea of sending inband escape sequences to trigger terminal functionality like cursor addressing, text attributes etc. So they developed their own terminal API that required special system calls to control all these things.
And, surprise surprise, nobody could be bothered writing software to use these special system calls.
They do not. And haven't had a CLI text editor since 'edit' in the 16 bit days
No wonder WSL is a thing, CLI workflow in the most basic sense is basically non-existent
Strange and incorrect characters in the stream? Sounds like logging into a non-Microsoft IRC server using MS' Comic Chat.
@@RadikAlice Yes they do have a terminal (console host, using pre-11 nomenclature), a bunch of improvements was added to conhost in 2019, and now Windows Terminal is default on W11
The Windows subsystems for OS/2 and POSIX were supported up to and including Windows 2000. I used the POSIX subsystem on Windows 2000 back in the day, and was fairly satisfied with it, except that it didn't support fork(2) (the call returned -1 with ENOTSUP / ENOTIMPL). Since I needed an implementation of fork(), I created one in native Win32 code (which was pretty ugly but worked). BTW, most of the tools and documentation for Windows POSIX were available on the Internet by the time I used it. The Windows SDK was downloadable from Microsoft as well.
No random and srandom, so no proper C Standard Library support or a really broken build environment. With no networking and very limited interop(ooh shared filesystem), They really didn't want anyone using it. Meeting the bare minimum for government contracts, just like any business would. WSL 1 pretty much ended up doing The Posix subsystem the right way, aiming for actual binary compatibility not just poor-quality source compatibility.
WSL1 isn't binary compatible, it is syscall compatible. It also wasn't the first time it had been done either. Circa 2002 out of MS Research a similar project was done which could run Linux ELF binaries on a Subsystem. To demo the ability they were running an Apache webserver and had Firefox accessing it. It was never going to see the light of day with Balmer leading the company. WSL1 is more like WINE in that the syscalls were handled by Windows and translated on the fly to an equivalent set of Windows API calls. WSL2 is more like And Linux or CoLinux where the kernel is actually running in a VM and is isolated in its own Hypervisor. Each have their own strengths and weaknesses, but WSL2 with how it visualized the file system and hardware is more resound and can keep up with Linux kernel changes in a way that WSL1 can't. The magic is how they've bridged WSL and Windows so that they are both working together with only a few seams if you know where to find them. For most of my Linux needs, Windows is the best Linux system I need and it is easy to switch between different Distros.
POSIX on the other hand. It was available for very custom work and not expected for more than supporting those very bespoke customer solutions. The subsystems were initially designed to support more than the Win32 API, but with IBM and DEC usage being significantly anemic, OS/2 and VMS compatibility wasn't a priority. NT 3.1 and 3.5 were intended to offer more, especially on the POSIX side of things seeing as Dave Cutler and many early NT engineers had built VAX for Digital before coming to Microsoft. The bones were there and have been from the beginning -- NT kernel was going to run Win16, Win32, DOS, OS/2, and POSIX applications, each in their own environments -- it was soon clear that Win32 was the Subsystem to design around.
I think it would be interesting for Microsoft to revisit supporting more subsystems. There'd be a great MSR project to make modular environments to host things like a Node/Deno container, one for .Net CLR, Android, or a modern LISP machine, with a Win32 host machine providing a way to keep those other systems running in isolation. It would look a little like Docker Containers in some ways, or a NT-based Kubernetes OS. I think Windows 10X was sort of heading that way with Win32 apps running in their own container, and I think it is still a concept worth going after. WSA and WSL demonstrate that it is possible, but it still needs refinement.
@@R.B. WSL1 is binary compatible for a subset of certain calls, actually. I've first hand tested that.
FWIW I dont think there was any malicious intention in "sandboxing" the posix subsystem. In Windows the POSIX subsystem sits above the kernel just like Win32 and OS/2. I dont think this was anything malicious just Dave Cutler drawing a natural boundary around the kernel and implementing what was needed. I worked on this operating system and I can tell you that there was a TON of stuff going on in Windows at the time and it was simply the case that OS/2 and POSIX were very quickly not worth spending developer resources on.
No, that's completely fair. I think it got left out of the actual cut, but in the original notes, I actually did compare it to the OS/2 one, which can access NetBIOS, and IPC structures, and has more integration. Like, you could actually use the OS/2 subsystem as a migration path from OS/2 server to NT, and Microsoft's own pre-NT 3.1 early NT toolchain was OS/2 based and ran through the subsystem.
@@NCommander Totally. Around this time we were building Window 95 with direct X, then merging all of that code into the NT codebase to produce Win2k, figuring out how to service the OS, doing the IE thing, I mean the origin of Windows was a rocket ship. It was a very diff env from today, tbh noone particularly cared about POSIX. Linux was nothing even close to what it is today (a trajectory which changed dramatically when Google invested in Android) and it was definitely not a high priority. The situation has changed 1000% and WSL is a much more committed attempt and IMO absolutely critical for Windows' - survival given the importance of Linux today dev machines need to be good to work with Linux and OSX has an advantage there. But IMO Windows has made great strides here e.g. passthrough GPU virtualization for WSL usage is a major deal for machine learning workloads. Anyway im rambling....
Awesome video, awesome channel, thank you - subscribed!!
On a technical level, it would've been impressive if it was completed and documented properly. The NT Kernel has support for subsystems besides Win32. Would've been fun to actually use it. Ah well.
That was Services for UNIX and Windows Subsystem for Linux v1.
Wsl is a more 21st century solution with containers and virtualization
@@timothygibney159 that's WSL2.
They really did the absolute minimum they had to do to be certified. Shockingly honest.
I'm pretty sure cc.exe and make.exe exist entirely to run the NIST test suite.
It's a miracle those aren't just one massive if-else block that spits out fixed machine code that passes the test depending on the input file name.
@@NCommander running NIST tests suite incidentally is a good candidate for “something to do with it”
Windows has another POSIX subsystem called SFU,any idea to try that?
It's on the topic list.
I bought a copy of Services For Unix back in college where I used Unix and NT workstations in the engineering computer labs and ran Windows on my dorm PC. Writing portable c++ code back then was frustrating, especially not standard Unix filesystem structures for headers/libs.
Nice! Wasn't this a big deal at the time? This is crazy stuff! As an economic history researcher, I swear this story deserves a paper, if there isn't one already. Subbing right now! ❤
4:56 tar is not a single format and I'm guessing you were trying to unpack a GNU format tar file. With GNU tar, you might have better luck creating archives with one of "--format=posix", "--format=ustar", or even "--format=v7".
Windows terminal stuff has always been atrocious and it was the first thing I thought of when you mentioned Rogue.
Even now it's quite bad, though it has improved significantly. It's still decades behind
SSH wasn’t enabled by default in PowerShell until W10 1809. It was a huge leap forward.
The problem with IE4/Active Desktop is just how resource-hungry it is compared to the original incarnation of Explorer. Plus, on 95 at least, the shipped IE 5.5SP2 runtimes make it leak memory. To each their own, but anywhere the original is an option, I take it.
Back in the day I used active desktop to have a live stream of some kittens playing in a cat shelter stream from a webcam as wallppaer. It used flash and you had to restart explorer everyone once in a while to fix the memory leaks.
6:43 in Windows 10 and newer, you can enable or disable case-sensitivity on a per-directory basis now
that is cursed
Sounds like a super convenient way to pull out your hair at some undetermined time in the future! Thanks Microsoft!
@@lucass8119 It's actually really useful for debugging when you are doing iteration on Windows but the target device uses a case-sensitive filesystem. You can set the folders as case-sensitive and catch those bugs early before trying to deploy and debug on the target hardware.
Nowadays, this could be seen as perfect example that a company changes according to their CEO and its decisions.
I dunno. I don’t trust them with Linux. I’m sure they’ll ruin it to promote their own stuff if people get too comfy with Linux Subsystem
UNIX and companies that sold UNIX were their own worse enemy. One of the reasons it was the POSIX standard and not the UNIX standard. X-Windows is another such standard that was crippled from day one. Many UNIX based companies also were not POSIX compliant. Microsoft was hardly alone in that regard.
The world wanted a graphical OS that didn't cost $2000 per year per work station. IBM, HP, Solaris and the others charged ridiculous prices for UNIX and hardware. Linux at the time was a hobbyist toy and wasn't an option. Today people get software almost free but in the 1990's it wasn't cheap and Windows NT cost 1/10th of other OS's. Not a small thing when you have 1 million employees who need computers. POSIX might have been a government specification, but the reality was it was only used by a few people in IT. 95% of computers were used for email, Word, Wordperfect, Lotus and Excel. Managers and IT departments wanted a single type of OS to support, or at least that seemed the vision. The reality was/is that different use-cases required different hardware, different OSs and different IT support.
Even at the time people were aware of the limitations of NT and POSIX. They didn't care as long as the legacy box that only mattered to the old guys running mainframes was ticked. Remember, UNIX was really only run on mainframes or very expensive workstations at the time. Google wasn't even a twinkle in the eye of Silicon valley.
It was only later in the 2000's when Linux was mature enough to save UNIX from obscurity that UNIX compatibility became important. Today things are much different. So it isn't really fair to judge MS from the 1990's. POSIX wasn't all that important and it looked like UNIX was going to die a slow death. If not for Linux, it probably would have.
Would love to see a comparison to the later Unix subsystems in NT: SUA/Interix and Windows Services for Unix (SFU). And maybe a side look at Microsoft's own standalone Unix implementation, which also influenced these efforts: Xenix.
It's on my TODO list, although I've covered xenix before I did youtube: casadevall.pro
And I thought the RS/6000 shipping with reference implementations of core features was bad (which it is)
but this makes that preferable, I bet that if Linux and open source hadn't taken off in the business world
we'd still be dealing with this kind of unsubtle yet subtle enough middle finger towards UNIX as a whole
“Unix” is a trademark that only proprietary products can afford to license. And all those products are either dead or dying right now. What matters nowadays is not “Unix”-compatibility, but “Linux”-compatibility.
@@lawrencedoliveiro9104 While I understand your point, the wording made me roll my eyes and kinda want to say _something_ out of spite
It feels insufferably pedantic, but maybe that's just a gut reaction from a bad first impression
It’s like they assigned a bunch of project managers to ship a bunch of components and when it came time to integrate it in a reasonable fashion - some higher up must have said “eh…… what’s the ROI?”
I saw mention of the posix system in Windows NT back in 1993. It never even occurred to me to go check it out, since I had access to Unix. What would have been the point?
That's what a lot of people have told me. SFU was really when it was taken more seriously.
@@NCommander I've been in software for 40+ plus years, an I have been burnt many times by emulating other peoples environments. Its always a moving target, and its really amazing how common it is for programs to skirt the rules of an interface and take advantage of undocumented quirks of the interface. This is one reason why I am not a fan of either Wine or the Linux on windows subsystem. If I want the Unix/linux environment, I'll just go use Unix/Linux. If I want a program to port to a different environment, then recompiling it for the new environment is the way to go. For programs that cross multiple platforms, that's why god made cross platform porting kits. Put it this way: a program designed for porting via a kit had portability in mind from day one. A program written for a given interface (Windows/Linux/Mac) was designed to be bedded in cement from day one.
I’m not sure if the complaints about requiring those “obscure” CDs make sense from a 1990s perspective. Pretty much nobody really needed the POSIX stuff (developers and users alike), and disk space was at a premium, so relegating those to separate CDs/products seems reasonable to me.
Thanks must go out to the FOSS Community for WINE and Bottles. Kudos to Valve for Proton!
Now we just need ReactOS to proceed past pre-Alpha stage...
Very cool video...didn't know a posix subsystem existed for windows NT...I only found posix on c functions that looked like yea that might exist on say unix...and it would say depreciated...and I was like ...yea I am using that...for it was something simple I was familiar with...just needed to refresh on the parameters, etc. I guess not knowing about this was a good thing...kept me from attempting it when I was moving from DOS to Windows NT...and didn't know much about how to get there...I found the win32 library interface and I was like ok...winner going with that...worked out ok...well less painful than anything like this.
Looks a bit like what they did to prevent the netware client from running properly...
You mentioned a book "The Unix Haters Handbook" in a video of yours, one of the solaris ones maybe, anyway, I had to tell you this book is absolutely hilarious. I haven't laughed this hard reading something in years. Thank you.
I remember popping in for a bit during the live stream. Didn't realize you had to use nmake at one point... I'm so sorry. Happy New Year!
I have a feeling that the relevant FIPS standard was written with a close guidance from someone with Microsoft's interest at heart (or in his pocket more likely).
So reaching the bare minimum for compliance was easy, even though the implementation was a complete joke and an insult.
I really like your channel, it has V I B E S. IDK how else to put it. Something about the music and the calmness of your voice and the highly interesting subject matter. Really great, thank you! :)
Wanting to say compliant on the box but doing as little as possible to make it so, and making it so damned hard to have any real practicality as a move to make such compliance a thing of the past.
would be nice to have a stream on SFU to see if it's more sane.
SFU is absolutely much better than this. Wow.
SFU is much more complete and better than this. Not only it provides much more Unix utilities and POSIX API but later versions even contains X11 libraries. SFU is much more close to something made properly than this thing ever was.
Imagine trying to admin this in the pre-Internet days
Well in the those days there was a lot of arcane stuff to get a Linux system up and running as well (i still remember modelines for the display driver or wading through long lists of kernel options "fondly"), but at least there was some documentation (read those READMEs) and it didn't have this feeling of malicious intent.
As a counterpoint, you should try out OpenSTEP YellowBox NT. It's basically all of OpenSTEP (fka NeXTStep) 4.2, both GUI apps and command-line, but running under NT 4. (I once had it installed under XP, as well.)
It's pretty complete for 1996-97, but good luck building modern stuff. Ironically OpenSTEP 4.2 (including OpenSTEP Mach, the full OS which was derived from NeXTStep and later evolved into OS X) is worse at this than NeXTStep 3.3 in some cases, as the POSIX compatibility is more broken.
We've been having a long discussion on it on Discord, and how it works (which is abusing cygwin). It's on my to cover list as a stream if nothing else.
Good 'ol malicious compliance. I think I'm preaching the choir here when I mention embrace, extend, and extinguish as MS's modus operandi back then, hell, they coined the term.
Did you try searching old newsgroup archives? That's where I would look for interesting commentary from that time period.
There wasn't a lot, most people complaining they lack parts, or being told it was useless.
Boo at Microsoft for doing the bare minimum but what'd you expect from them back then ;)
The Resource Kit, however: I don't know anyone that used NT 4 that did not have it installed. If you used NT Workstation at home you were pretty much sysadmin-level to begin with, had a work MSDN subscription, and knew what was out there.
This video gave me flashbacks when Microsoft decide not only to implement ODF in the most half backed way inimaginable but also publish it's own OOXML format as a standard in the most useless way to make difficult the implementation by the competition 😬😵
Yes, my thoughts exactly. And didn't they send their own people to the standards meeting just to vote for them, and never show up again? I remember that causing a problem because they couldn't get a quorum or something.
IAC, what they shipped was the original version without any of the feedback fixes needed by the eventual standard.
Microsoft itself does not support ISO 29500 (OOXML) properly. Microsoft -Office- 365 still only implements it in “Transitional” mode, since “Strict” mode apparently causes too many compatibility problems.
For a while back in the early days of those standards I remember discovering that Word's implementation of the ODF appeared more true to the official standard than OpenOffice Writer's was. This was because Writer had already progressed to a later version of its native ODF format by the time it became standardized, which had more extended/updated features. I cannot recall that it had any way of saving/exporting to the older but standardized version. Word's way of supporting ODF was rather cumbersome though, as one had to install and deal with it as a separate plugin. Conversely OOXML has always been a mess from its very inception, allowing for including BLOBs in proprietary formats that only Microsoft itself ever has a chance of fully supporting.
There also seems to be an OS/2 16bit character mode support in NT up to Win2000
What is the background music called, beninning at 0:04?
Amazing stuff and very informative as usual.
Also lol'd @6:50
Is this the reason Windows has system32/drivers/etc/hosts? I've always thought this implementation seemed very at odds with everything else in Windows.
The original Windows NT 3.1 TCP/IP stack was a direct port of the one from BSD 4.3 by Spyder Software. I've always assumed it was a holdover from that, although having a hosts file is de facto standard (and might even be in one of the RFCs as required).
@@NCommander The hosts file predates DNS.
On GNU systems, whether you need a hosts file or not actually depends on settings in your nsswitch.conf.
I was interested in that thing but never actually had time and motivation to try this out. Now after your video I know that it's not a bad thing that I didn't try. Good thing that it was replaced by SFU which was much more complete and useful. Also I used Cygwin and it was pretty convenient and useful as well. I was even able to run some Unix desktop environments with it and Cygwin/X.
I'm sort of tempted to get a VM setup with the POSIX environment on my end so that I can try and run neofetch in it so my guess is that I would have to get bash working in this environment
This may not be the case, but I read that the reason NTFS supports hard links (that is, more than one directory entry pointing to the same file) is that they had to implement this POSIX subsystem. Hard links are _so_ useful. For example, prior to a large scale batch rename, I can clone a directory tree without duplicating the file contents. (This uses cygwin btw.)
I think they were using HPFS at the time
Don't even need cygwin. Lookup "Link Shell Extension". NTFS supports both hard and sym links. And it has for quite some time (NT4!). That plugin has some cool functionality they built on top of the base functionality, so it's definitely worth a look.
@@ccoder4953 hell you don't even need to install anything, the standard command prompt windows ships with has a mklink command for doing this
though having it as a right-click context menu is _very_ handy if you make links a lot
Symlinks were not added to Windows until Vista. Even now, they require special privileges to create.
@@lawrencedoliveiro9104 Actually, we're both sort of wrong. According to Wikipedia:
"Symbolic links to directories or volumes, called junction points and mount points, were introduced with NTFS 3.0 that shipped with Windows 2000. From NTFS 3.1 onwards, symbolic links can be created for any kind of file system object. NTFS 3.1 was introduced together with Windows XP, but the functionality was not made available (through ntfs.sys) to user mode applications. Third-party filter drivers - such as Masatoshi Kimura's opensource senable driver - could however be installed to make the feature available in user mode as well. The ntfs.sys released with Windows Vista made the functionality available to user mode applications by default. "
As far as the special permissions thing:
"The default security settings in Windows Vista/Windows 7 disallow non-elevated administrators and all non-administrators from creating symbolic links but not junctions. This behavior can be changed running "secpol.msc" the Local Security Policy management console (under: Security Settings\Local Policies\User Rights Assignment\Create symbolic links). It can be worked around by starting cmd.exe with Run as administrator option or the runas command. Starting with Windows 10 Insiders build 14972 the requirement for elevated administrator privileges was removed in Windows "Developer Mode", allowing symlinks to be created without needing to elevate the console as administrator. On the API level, a SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE flag is supplied for this purpose"
So, the policy can be changed and it appears Microsoft will soon make the change the default.
I cut my teeth on NT4 and Visual C++ 6. I still use the VC6 key bindings in Visual Studio Code
I would imagine they did this to be able to take government and military contracts since they required POSIX compliance
Are you going to do a follow-up covering Interix? That looks like it was more of a serious attempt at providing POSIX compatibility
I will at some point. It's on the topic list.
Does this have something to do with me not being able to get Python's ctypes library to recognize files that use pthreads?
it's not half-baked, it's fully baked, exactly what they wanted to ship :P
Hi.
What I remember, but back in win2k world and appropriate MV VS, there was an option to configure a project output to be for Windows POSIX subsystem, as just one setting. I tried it literally once or twice, it worked, but there was absolutely no practical usage.
6:04 Being able to traverse a directory (e.g. CD into it or one of its subdirectories) is a separate right from reading (listing directory entries) and writing (creating files) -- I think Unix-likes use the "execute" bit for it, and Windows NT and NTFS do the same in their ACLs. However, looking up this permission for every ancestor directory in addition to the permission of interest for whatever file you're working with takes time, and since the concept didn't even exist on old-technology Windows there was little confidence software would get it right. So all users were by default granted the right to bypass these checks: if they have permission to a file, they have permission to a file, regardless of some execute bit on some parent directory. Needless to say, that's not POSIX-compliant.
While the government would have to revoke that right (compliance!), it's irrelevant to what you're doing so feel free to leave it on.
It actually doesn't work correctly with Traverse Checking left enable. sh.exe doesn't load its prompt properly and there is a bunch of weirdness.
What if you strip the NT kernel down to a DOSlike prompt, and try to set it up such that the POSIX subsystem is the only available command set? I don't know exactly how close you can get to this, but it seems like a gloriously cursed idea to me.
In theory it's easy to replace explorer.exe with anything else (for example cmd.exe or some other terminal app, maybe even written by hand to have an illusion of full screen text mode) and stick to that.
Having a "pure" text-mode out of NT would be tricky though and I'm not sure if possible on retail systems, given how system is bound to GUI and various low-level service processes that controls user session & login.
You cannot run POSIX subsystem (PSXSS) without the Win32 subsystem (CSRSS). You can, however, have a pure command line installation of Windows (e.g. the Embedded toolkit allows you to build one), where you can easily replace the command interpreter with sh. Might be an interesting idea to combine with NT Embedded.
@@dr.shuppet5452 It would be probably the most limited "UNIX" system ever 😀But could be interesting with Cygwin (for older Windows) or WSL for newer ones...
@@dr.shuppet5452 You could have text mode Windows installs, if I remember right, all the way until Server 2019, where iirc the option to not include the desktop in the install was removed. Text mode Server 2016 is a very limited experience, as the majority of modern Windows-centric server utilities, or at least the ones I've used, have major GUI components with either minimal or no terminal interface.
Maybe, to commit to the cursed idea as much as possible, the system PATH variables could all be cleared, assuming the system uses hardcoded directories instead of PATH during boot.
@@dr.shuppet5452 Is that a hard dependency, or more of the fact that you wouldn't be able to launch anything? In theory, you probably could create a native mode application (runs before csrss starts) that starts psxss and messages with it, not that it would be a sane endeavor at all, given the absolutely nonexistent documentation on writing native mode applications, and the RE work that would be required (unless you could find relevant bits in the many leaked sources over the years)
(internal details below, probably shouldn't look if you want to contribute to Wine or ReactOS)
Edit: my current findings are the dependencies are mainly on SMSS (which should be already running) and on whatever provides the SAM database, and probably a few others. HOWEVER: smss supports running subsystems under other subsystems, and that's precisely what psxss seems to be. It's not a native mode subsystem, it, as a subsystem, appears to be set up to run under the Windows subsystem. Unlike csrss, smss, which have UMTYPE=ntss, and autochk (which has UMTYPE=nt), psxss has UMTYPE=windows. So you are correct.
Dave Cutler was a VMS guy, and it shows.
that background music is the exact same as used in the Mountain Beast Mysteries channel (Bigfoor stuff). Weird to hear it on other videos...
It all came from the TH-cam Music Library
I hope you don't disappear for 4 months again!
I've been livestreaming, but making video content has been difficult. I've made some headway on a new video, but its been very up and down in real life
@@NCommander It makes sense, don't let my comments build any pressure. I really just wanted to say your videos are awesome!
*Babe wake up, new NCommander!*
I think this illustrates the problems with POSIX more than it illustrates the problems with Microsoft's practices (although perhaps not by much).
Edit: For an amusing historical look at the UNIX community's own problems with POSIX (as well as many other things), try tracking down a copy of The UNIX-HATERS Handbook. POSIX was NOT sufficient to properly address the problem it was intended to solve.
It was a Government specification. Would you expect anything else?
From what I understand, the posix subststem is same mechanism as WSL1 (not wsl2 though, thats just a linux vm)
Indeed. I understand why they went with WSL2 in the end, but WSL1 seems more ... elegant, for want of a better word ...
so much wrong in this video: the windows SDK and reskit were extremely common for windows developers to have: you could download them for free off of MSDN, and the MSDN CD bundles came with them. in fact, for most win32 application developers the windows SDK was usually the primary source for windows compilers, headers & libraries since they were always up-to-date with the latest OS release whereas the MSVC headers often lagged behind. the only real reason to use the MSVC-supplied compiler is if you wanted to use the crt or MFC libraries which weren't included in the windows SDK (they're not part of windows itself), also MSVC isn't required for POSIX development, the windows SDK is sufficient.
you have to remember that the posix subsystem fo NT wasn't designed to be an interactive system - windows already has a UI/shell. it was simply designed to run applications, usually background services. the utilities that are provided are simply there because they're mandatory in the posix standard.
it's a little strange that you call something that literally implements 100% of what it's required to implement "half-baked". are you saying that posix is half-baked?
I feel like this could have all been avoid with a generic ‘no, don’t even think about it’ clause.
MS: what if we just….
POSIX: No
MS: But how about…
POSIX: (slaps MS with a wet fish)
its funny you say this because there are actual email exchanges dated 1990 between the posix committee and the NT team in leaked windows source which basically go like this
lol what a complete mess.
Appreciate this kind of dive into such a topic though, it was always something I was vaguely aware of but never researched the true extent of "POSIX compatibility" in Windows NT, never heard of anyone using any such functionality and never used it myself.
I admit though that in modern day Windows it is rather nice to not be constantly thrown off by using Unix commands like I used to be in the old days, I still have no pressing need for things like Linux application compatibility but to not need to make that switch between Unix and old DOS commands is much appreciated, that was quite annoying in the past.
2:52 POSIX also has its own standards number: IEEE 1003.
I remember there was something called Windows Services for UNIX
Thank you for sacrificing your sanity so that we may behold this monstrosity. Compiling Star Trek for MIPS Windows sounds like a great idea!
Even the Linux boot/rootdisk in the 0.11 kernel days had massively more functionality than this. When it came to SFU, I think SLS (Linux distro from ~1993) was considerably more advanced than SFU. Using SFU felt like using a 1980s version of ISC Unix.
"when Microsoft implemented *teh* POSIX standard."
POSIX was in Windows until Windows XP or Windows 2000, but now we have WSL
"Tell me you hate standards without telling me you hate standards!"
Microsoft:
Interix and Exceed were the tools of choice for POSIX on NT 4.0 en.wikipedia.org/wiki/Interix
Remember Exceed for X11, but I mostly used Cygwin and Mingw32 for terminal stuff.
@@butsukete1806 MKS Toolkit was another early player. I think Cygwin became popular a bit later (and obviously I used that once it did).
This was part of the original architecture of Windows NT: there was a core kernel, on top of which you could have multiple “personalities”, each with their own userland API. Win32 was itself one of these “personalities”, and so was the POSIX layer.
In the early days, Microsoft did let the documentation get out on how to write your own personality, and the Interix folks got hold of this. Then management changed their minds and decided to restrict this info. So they bought up Interix to prevent the details getting any further, and the Interix product became Microsoft SFU.
You’d think present-day WSL would be built on the same architecture, but it is not. WSL 1.0 might have been built that way, but they could never get it working right. Which is why WSL 2.0 falls back on good old virtualization instead. And still doesn’t work completely right.
So one could say that the original “personality” architecture has pretty much bit-rotted away.
@@lawrencedoliveiro9104 You meant I can't run my OS/2 1.1 and 1.0 apps anymore? And yes. That was another personality. And yes, I'm still bitter about OS/2.
I wonder how later editions faired.
It wasn't half baked, it was meant to be frustrating so users switch to native Windows tools.
Hey Hey Hey
Just came back for Another ReWatch!