Linus Was Right, Never Break Userspace
ฝัง
- เผยแพร่เมื่อ 18 ก.ย. 2024
- Recently this Reddit post was floating around about deprecated functionality in grep, specifically fgrep and egrep and whether it's ever ok to actually break this old but widely used functionality
==========Support The Channel==========
► Patreon: brodierobertso...
► Paypal: brodierobertso...
► Liberapay: brodierobertso...
► Amazon USA: brodierobertso...
==========Resources==========
Reddit Post: / the_deprecation_madnes...
SCO Xenix: www.scosales.c...
Removed Utilities: pubs.opengroup...
Patch Creation: debbugs.gnu.or...
Grep 3.8: lists.gnu.org/...
Greb Docs: www.gnu.org/so...
Undeprecate: debbugs.gnu.or...
=========Video Platforms==========
🎥 Odysee: brodierobertso...
🎥 Podcast: techovertea.xy...
🎮 Gaming: brodierobertso...
==========Social Media==========
🎤 Discord: brodierobertso...
🎤 Matrix Space: brodierobertso...
🐦 Twitter: brodierobertso...
🌐 Mastodon: brodierobertso...
🖥️ GitHub: brodierobertso...
==========Credits==========
🎨 Channel Art:
Profile Picture:
/ supercozman_draws
#OpenSource #FOSS #Linux #Grep #GNU
🎵 Ending music
Track: Debris & Jonth - Game Time [NCS Release]
Music provided by NoCopyrightSounds.
Watch: • Debris & Jonth - Game ...
Free Download / Stream: ncs.io/GameTime
DISCLOSURE: Wherever possible I use referral links, which means if you click one of the links in this video or description and make a purchase I may receive a small commission or other compensation.
Tar was removed from the spec because it was deprecated decades ago. Everyone was supposed to switch to `pax` in the late 90s afaik. But from what I have seen, it was depreciated because it was broken in ways that has since been fixed. And considering that tar has had 20 years of continued development while no one has ever heard of pax, is pax even better than tar any longer?
I've never heard of pax either
Pax is Latin for peace and was the posix attempt to resolve the dispute of whether to use tar or cpio. It didn’t catch on like the committee hoped and we still have all three, like in the xkcd comic about standards.
Only heard of it because of a vulnerability report for Zimbra. One of the components, amavis, would you cpio which was flawed if pax wasn't installed
waiting linux world burn when they removing tar "one day".
@@fxddymspax do be a superset of tar, but idk the cli was really confusing to me back when I tried it (granted, only two times, but still)
"Please re-enable spacebar heating"
XKCD 1172 for those wondering
XD
So remember folks, Linux does not break user space GNU/Linux does.
And now you know why the difference matters
Not really. If you give non-nerds a Linux distro they're just gonna blame "Linux". They don't know what GNU is.
Lmao
maybe Linux doesn't make breaking changes enough, but bloody hell does GNU break things a lot.
Like, WOW. GUYS. COULD YOU _PLEASE_ STOP MAKING BREAKING UPDATES TO GLIBC FOR *_ONE SECOND_*
The GNU project exemplifies politics > code (except the politics is regarded).
@@LabiaLickerI don't believe a religiously motivated development community can ever generate as good code except by pure chance.
-Linus Torvalts about GNU Hurd
You did not consider the exemplary codebase that is TempleOS, did you@@vilian9185??
breaking changes in gcc itself is pretty annoying too, biggest headache of rolling distros and bleeding edge as a whole, keep having to temporally downgrade to do stuff
@@DDracee i think with gcc usually it's someone invoking ub and that ub changing in which case all bets are off
Since there are obviously no plans to reuse the egrep and fgrep names for anything else, I don't see why they can't stay as aliases forever.
an alias doesn't work in scripts rho
@@kash1222 Alias might have been the wrong term. But a symlink or whatever. I know there are already some programs that can detect which command was used to call them and have it imply certain options.
@@kash1222 That's exactly how they're implemented already though. On my computer, egrep is a just this shell script
#!/usr/bin/sh
cmd=${0##*/}
echo "$cmd: warning: $cmd is obsolescent; using grep -E" >&2
exec grep -E "$@"
Looks like in most cases it uses a wrapper script, but the grep executable also has the ability to detect if it's being called with "$0" set to egrep and it will assume -E and print the warning by itself without the wrapper.
So distros have the option to either use a wrapper script (with or without the warning) or a symlink (with the warning).
@enemixius yeah, keeping the aliases for deprecated usage seems a safe thing. This sort of change to finally remove egrep/fgrep costs more time to discuss and expunge than not, and saves nothing appreciable in terms of resources. Making a crontab for a script to purge the crap data one's own browser has in cache would be FAR more effective.
Removing it outright, that's a cram linux onto a storage/compute/RAM limited embedded application sort of thing. Modern system, not worth it. My ancient 15 y.o. machines... still, not worth it.
For those annoyed with the warnings.
In most rational distros, the warning actually comes from the wrapper scripts for egrep and fgrep, located in /usr/bin/egrep and /usr/bin/fgrep - respectively (they are no longer actual binaries). You can edit these shell scripts to remove the line that `echo`es out the warning.
It's also fairly simple to make fgrep and egrep wrapper shell scripts as well, if your distro doesn't provide them.
I assume these lines are on stderr otherwise you get those lines as grep results?
@@edwinkm2016 Forgive my ignorance, but I unfortunately, do not know what you mean by "on steer". What I was referencing in my comment was the warning messages like "egrep: warning: egrep is obsolescent; using grep -E" when you run egrep or fgrep. This can be disabled by removing the line `echo "$cmd: warning: $cmd is obsolescent; using grep -E" >&2` in the egrep/fgrep wrapper script, which is the line before `exec grep -E "$@"` (since egrep and fgrep are no longer binaries and just compatiblity scripts on most distros)
For reference the whole wrapper script for /usr/bin/egrep looks like this:
```
#!/bin/sh
cmd=${0##*/}
echo "$cmd: warning: $cmd is obsolescent; using grep -E" >&2
exec grep -E "$@"
```
So you get the warning "echoed" out before the you get the `grep -E` results. I'm really sorry for any confusion, English is not my first language.
Some distributions already just patch out the warning as part of their grep package.
@@Time4Technology That feels kind of disrespectful to the grep devs.
Sorry, Spellcheck changed it to “steer”. It should be “stderr”. If it is on stdout it will break the groep feature and it that case the removal of the wrapper would be the better approach. In that case o lot of scripts will at least crash (if set -e is used). That is better than incorrect groep results
I think it's generally okay to remove things that have long been deprecated, but only if there is a good reason to do so. For example removing an old library version that would take active effort to further maintain has the benefit of less work for the project. However, in this case, I don't see how an alias / symlink would have any negative impact at all. What would be the benefit of removing this considering it would probably break a bunch of stuff??
The docs say you can set up a script for an alias. That's not the issue here.
@@Megalomaniakaalyou can, but how about the project can handle it itself? Not everybody will set it, and then wonder why there are issues.
@@TVPInterpolation Yes that's what I'm saying, projects/distros or sysadmins at a company/office can set up aliases if they want. The deprecation and removal of E- and FGrep does not affect that what so ever. And distros that want to follow a standard can remove without replacing with alias or anything no problem. This whole thing is honestly a nothing-burger...
@@Megalomaniakaalthat's what's already being done, why change it?
@@MegalomaniakaalThe question remains though. Why would you remove egrep and fgrep in the first place? Why even deprecate them? It is the clear intention of the developers - but why though? I can't think of one good reason why you wouldn't just make it part of the POSIX standard and require any POSIX compliant system to have those commands.
It is clearly never going to be fully removed completely. It is way too late for that, there is absolutely no reason to do so, and there are plenty of reasons not to. So it's always going to exist. But now, some scripts that run on POSIX compliant machines are not going to run on other POSIX compliant machines because of the explicit state of limbo that it has been put in by the developers.
There's 3 options:
1) push to remove egrep completely and have people not use deprecated commands
2) have it exist in the weird limbo state that it is in now
3) just make it part of the standard, because it pretty much already is a de facto part.
It's okay if Linux has historical artefacts and vestigial parts. It's decades old, and all software is built on layers and layers of older software that came before it and still works fine. As long as those older layers remain fixed and stable, the newer layers will still work fine. But if you want to mess with those older layers, you better have a damn good reason. "Idk just didn't feel like keeping it around" is not a damn good reason.
they kept it for like 2 decades, unless they can prove a reason why it is a problem to keep it, i wonder why bother making work on removing it and also making work for all the programmers who uses it in the many places it probably is... kind of a waste of electricity and productivity for all the clean up this causes in my own opinion.
I'm not really sure what's being deprecated. According to what Brodie read out, the reason for egrep and fgrep was because it made sense to break up programs into smaller binaries back in the day but that's not necessary any more. Removing the ability to compile grep into different programs is inherently going to prevent the egrep and fgrep commands from working anymore. The actual change from the user side just requires a syntax change. For those who don't want to do that, backwards compatibility can be maintained by just making shell scripts that alias the commands.
Sure it would be trivial for the grep devs to provide those scripts but why should they do things that encourage the wider development community to keep using old commands in new code out of laziness? If they take away the guarantee that that that syntax will be used then it will push devs to use the new syntax (which has worked for 20 years) on new code.
not sure for that problem in particular, but often an shell alias is enough
It's like almost zero work. I'd rather have it break so I can find it and fix it.
@@Ormaaj well from my point of view, there is 0 reason to change it other than "an other method" exist meaning a group of 10 people will break things for thousands of programmers to have to pick up old code and change it. if they don't then it will stay broken. if it is somewhat vital, then someone will need to pick up from the available code and fork it, then tons of project will have to take from that fork...
kinda like a pyramid, the top breaks things and everyone under is stuck fixing things... but in this case, there is not really a gain to change it that i can see, it is just breaking things for the sake of change.
like i get it when you have things like the log4j thing where it has to change no matter what but here, it is really just some arbitrary change that does not change the result, only the method used for that result.
anyhow i am too much out of the loop to know the gains, maybe there is more to maintain due to how this is built so in the long run it might save time, but from this video alone, it only look as unimportant as a aesthetic change unlike a functional one.
like i said, i am too much out of the loop to know the gains of that change.
@@Ormaaj I was thinking the same thing. I was gonna delete the egrep and fgrep scripts just to see if anything really breaks
Unless there is a reason like bugs or complications due to it, i see no need to deprecate something.
And "don't break userspace" can be extended to "don't break things for users".
Removing useless or unwanted features is generally good for the package, since it makes codebase smaller and easiser to support by the programmers. Considering core system things tho, deprecating them is always a huge risk and probably not worth it.
@@elzabethtatcher9570
If you want to replace some features or program AND ACTUALLY WANT IT TO STICK you unfortunately can't half ass it. At first, just say it's depracated for the giga nerds to notice, but that phase can't last for decades, you quickly have to move on to the "stop using this you wankers" phase. It has to be agressive enough that at the point when you actually remove it, new stuff hasn't been made with the old stuff in mind for a very long time, so the side effects are managable.
You do "Not" want unused/old/barely used code in anything kind of project. That is a huge security risk and makes the project worse and make people keep using old stuff instead of moving on.
@@Shrapnel_Music Normally I'd agree but we're talking about fgrep and egrep. I just checked - on my system, these files are four lines each: the first being a shebang, the next two being the deprecation warning, and the last being `exec grep -F/-E "$@"` respectively. To call this a "huge security risk" that "makes the project worse" is a pretty big exaggeration.
I've but recently learned about egrep. I thought it was rather useful. Had no clue it'd been deprecated since 2007.
Can we start to apply this kind of rigor to windowing toolkits? If Gtk had even a quarter of this interface compatibility it would be a lot easier to commit to building apps in it.
I'm still pissed about when they broke userspace threads to replace it with fancy new pthreads stuff. It broke some eternally licensed CAD tools I was using.
Be a proper nerd and code a shim/wrapper.
@@Psychx_ It was a binary from a cad vendor, with built in protections. So shimming it wouldn't work. Also this was for my day job so I switched to a new tool.
@@davidjohnston4240so it was hard coded to only use a certain libc file hash?
I've ported and maintained applications on Linux and it is terrible trying to support it. Yeah part of the reason more apps aren't ported to Linux is the demand but a large part is it really is a moving target. You have all the different distros and everyone has their favorite and then each distro changes things between releases.
@@davidjohnston4240 You should have switched away from Linux. There is a lot I love about Linux, but application support and compatibility are terrible.
I can see why binaries like ssh or curl can get deprecated, they are actually security relevant.
But for egrep and fgrep... like bro, it's a text search. How in the hell can this be deprecated? Like yeah, old issues aren't fixed in those versions but if you work with grep to the extent that you actually encounter those issues then you sure as hell don't use those two binaries anyway.
And most distros should just link to grep by default via the shell. This should be such a non issue.
A lot of distros just make egrep & fgrep a wrapper for grep anyways.
It's wild why this is "deprecated", because as long as grep -F and grep -E act like fgrep & egrep respectively, there should NEVER be an issue with these wrappers.
They should have put the warning there 20 years ago
lol
Yeah. Like with everything, don't try to recondition your users 5 minutes before you throw it out, make sure 95% have switched beforehand by making it increasingly annoying. Sure, people are lazy and don't like change, but those aren't life changing things either.
meanwhile Gentoo maintainers:
# Delete the upstream wrapper variants which warn on egrep+fgrep use
# Install egrep, fgrep which don't warn.
This is insanely stupid. The amount of storage space and man hours that could ever be saved by removing these commands is vastly outweighed by all the wasted time energy and storage space occupied by talking about removing it. Keep it in, it hurts nobody, and it saves a few keystrokes if you choose to use it interactively.
I honestly don't understand why they would remove egrep and fgrep other than "we *technically* deprecated it so we're *technically* in the right for doing so". I don't see the practical advantage. Just define egrep to be the same as unzip, e.g. a redirection with a flag.
Like, don't get me wrong. I understand when projects deprecate certain ways of doing so, so that *eventually* they can streamline code and make things more sensible. But this isn't doing that.
Alias egrep to grep -e
@@kreuner11They already do.
grep and tar are often used in installer and uninstaller script. Removing them will break deployment of many packages. Removing packages, even if deprecated, is always a very bad idea, and specially now that storage spaces are getting larger.
Nobody is suggesting removing grep, just the egrep and fgrep commands/binaries. The actual functionality is provided by grep but uses a different syntax that is a drop in replacement.
Why deprecate them when they are literally just wrappers for grep
A question I have to ask is; where do you want linux and open source software to be in the world? Personally, I have seen machines that are tens of Thousands of Dollars and run software that is easily 10+ years old. This could be medical equipment, industrial machine tools, city infrastructure, business systems,etc. They may be small in the number of installs, but could be disastrous if they fail to work.
This is one of the things that has kept windows holding strong in the business space for so long: Backwards compatibility that goes back to the DOS days.
The reality is, if you want widespread use of linux and open source everywhere, then developers need to have an expectation that their software is going to end up in critical things that last a really long time. You will have to accept that some software will be critical but can't be maintained anymore. (granted it may be less of a problem given open source's nature but is a problem with proprietary software running on top of open source).
I do notice that reasoning for deprecating something is usually because it does not fit the current philosophy or logic. This makes since because removing feature is one way to maintain a piece of code. but the reality is.... reality is messy.
I think 20 years is long enough to wait to remove deprecated code.
@@transcendtient I agree. This is a change that's being blown way out of proportion especially considering the scripts that enable the backwards compatibility can be made very easily by those who need them. The use in medical equipment, industrial machine tools, city infrastructure, etc doesn't really change anything either because all those things don't get updated without a lot of testing first.
I swear to God I've watched a video that mentions these exact points
The issue has nothing to do with a feature being removed. It's strictly syntax. The only "feature" that would be removed from the actual binary, if it hasn't already been removed years ago, is the ability to compile grep as multiple binaries. For those that need the aliases for those commands to work, they can just create shell scripts that are one line long to restore the functionality.
For those that still use egrep and fgrep in scripts though, there's no reason not to convert them to the new syntax. It can literally be done with Find and Replace.
@@myownfriend23 You are right. this is a change of syntax to use functionally. devs can go in and change code so that it works again. that is a great thing about open source.
I realized reading from the comments something. my point is less valid because the software often running on these pieces of equipment is mostly developer abandoned proprietary software. Open source software will keep on working as long as the code is open and there are developers to work on it.
I guess is the real question is: What is reasonable amount of burden to put on others to adapt when you change how a piece of software works?
You say there is no reason not to convert. I can give one. "my project is big,old but critical and I need to tell my boss that this work is necessary so they can allow me to take the time to do the fix and validate." Not all the reasons are technical, it could be political and cost money.
We forgot how to write things that just works. Now everything have to be connected and constantly changing. Network stuff just is not secure by default and have to be constantly patched by bandaids. One minor update snowballs into costly updating all dependent systems. Everything moves out of our hands and control into subscription based external fragile mess based on misplaced trust.
Now lets apply this to a much further reaching and more complex depreciation scenario, the wayland protocol. There either needs to be a complete X11 feature set to wayland protocol interpreter before any significant adoption of wayland is implemented. (yea, theres xorg-xwayland but AFAIK it doesn't work for the vast majority of stuff) Or better yet, a complete from the ground up rewrite of the wayland protocol itself to maintain the functionality of the vast majority of the current Linux graphics stack. Might as well rename it X12 at that point and eliminate all the wayland hate at the same time. I'm in the "never, ever, under any circumstances depreciate anything from the user space" camp if it wasn't obvious.
You're obviously new to this channel, Brodie has like a million videos covering the wayland discourse and there are counterpoints to what you're saying here.
i think that for super old things that almost nobody uses, but, SOME people still might use it, (that weird, obscure intel architecture for those printer chips that you made a video about, is NOT the case) i think it would be cool if the feature was removed on the binary package, but, can be obtained on the source package by enabling it on the build system or whatever. although, i think i might be talking nonsense, and that might be pointless. not sure.
Stuff usually gets removed for maintenance reasons. If something stays in source, then it still has maintenance cost. Whether it gets enabled in a final binary is an unrelated topic.
@@09f9was literally having this issue when trying to install ns-cde
@@ayaya-ayaya yeah that's what i thought
@@09f9 Less of a reason to remove it then.
How is this even an issue thats ever shown up? I would expect every single package package maintainer that maintains some variation of grep to have simply added a global alias for these programs. I know all my systems currently do exactly that without me having had to lift a finger.
I like to use Bohemia Interactive as an example of how to handle things.
Back when they made a game called Operation Flashpoint, they misspelled guerilla in the script name upon release.
So any scripts that wanted to call something specifically related to that side, had to misspell the same way to make their scripts work.
This was patched later, but forever since then, you could still call the old misspelled version to make your scripts work, as removing or deprecating it, would cause massive breakages to a lot of user missions and addons out there.
I do recall it was there in Arma 2, however I have never done any scripting in Arma 3, so I can't verify if it is still in there, but I wouldn't be too surprised if it is.
To echo what others have said, I think it's really about maintenance and opportunity costs. If there is no maintenance for egrep and fgrep, they should be undeprecated and left alone. If they require significant maintenance, or their existence is preventing progress on another important initiative, then the deprecation warning should be put in place with a (generous) date listed for their removal.
it's all dumb, but I've learned to expect the dumb from the gnu project. If they were going to remove support for the _functionality_ I could understand it, but the functionality, and all the code that implements it is going to remain, it's just going to be accessed by the -E and -F flags, rather then running it as egrep and fgrep. The code to check what it's being ran as and behave as if the flag was passed adds less size to the binary then the string warning of it's deprecation!
In old Unix, there were good reason for grep, fgrep and egrep because size, algorithm, etc. varied. It mattered. It still matters, but now we are "wealthy", so we don't consider things like memory, algorithms, etc. GNU grep, obviously as an "all in one", doesn't matter of course. But there is a good historical reason. Does history matter? I think so. Eventually things do change though. But yes, those changes can mean that really cool older stuff will no longer function, and in some cases, will practically never ever function again.
Replying to self. I still program in good ole S.R.Bourne style bourne shell. What does that mean? My scripts will run practically everywhere, on many shells and shells over time. But of course, that would imply the use of grep, fgrep and egrep for universal portability. If all is GNU grep, then we might say "bash only" constructs as well. Is that ok? Anyway, something to think about.
Wouldn't grep just use the different algorithms when you pass those arguments?
It mattered because of program size. This commenter still thinks there are pdp 7's in production for some reason@@Spencer-wc6ew
@@Spencer-wc6ewWhat's even more important is that the algorithm does not depend on that option, it only changes the syntax of the pattern. Normal grep and egrep have the same feature set and both can handle literal patterns like fgrep (although with more escaping). I could see a use case for separate smaller fgrep, but that'd be some really weird edge case.
Bash-compliance is the new POSIX-compliance. The number of scripts in the wild that would run on a purely POSIX compliant system are vanishingly small. It's time we stopped caring about POSIX.
The commenters in that thread saying we've known about deprecation for 20 years have probably never worked with legacy systems. The concern is more for scripts that were made 35-40 years ago in the mid-80's that are _still_ in production and being used.
Because you don't replace perfectly functional legacy systems. If they work, an organization will prefer to use them until the heat death of the universe before putting money and time into the unknown variable of new code. Few people can appreciate seeing a virtualized Server 2003 in the 2010s running god knows what on an ancient version of Python. But if it stopped working, there'd be hell to pay. Sorry nuDevs--you're stuck with legacy cruft in the enterprise permanently.
If they're running legacy systems on a legacy OS they won't be affected. If they move to a new OS they will find in testing that they need to create 2 aliases or wrapper scripts.
Have used grep on various systems for 20+ years. Didn't know about egrep and fgrep. But have worked on systems that had a "native" grep and also GNU grep installed...
If you are throwing out error messages and/or ar intentionally making deprecated functionality slower, then just wait a reasonable amount if time before actually removing anything. It would also help for common tools if the maintainers would just reach out to various publications and influencers and state that they will remove x functionality in the next y years.
In actual software development there are some standards and conventions. One of the common convention is to not use deprecated functionality for new or maintained software. If you are developing a new script and know that a tool is deprecated or just happen to see the message, then don't use the old syntax!
I don't agree to keeping long deprecated stuff around just for the sake of it. One example that constantly gets on my nerves is the "comma cross-join operator" for standard SQL. This is deprecated since 1992, yet people are still writing new code in this manner because they know it won't go away any time soon and are just contributing to the problem. The major issue is that those queries will be a lot less performant than the standard calls and most importantly, newer developers don't have any idea what is happening there as no modern text-book and documentation make any reference to that style of coding. This won't disapper anytime soon as database providers still use this functionality for their internal working, even though they sometimes warn the users to not make use of the deprecated code...
I worked with companies that were using ancient software and dependencies and often it would take less time to rewrite a module than it would be to just create a long strings of hacky workarounds.
The grep problem is just a syntax issue. It can be dealt with in many ways and since the warning message, a lot if developers / projects updated their stack with the new syntax to iron out the error logs.
and it becomes REAL funny when downstreams start to patch these deprecation warning out
Python isn't afraid of breaking backward compatibility, and the result is that some Python programs simply package a program-local Python install with them to avoid dealing with that nonsense, and even more importantly, to spare the user from having to spend some quality time in the coils of anaconda or whatever the latest attempt to work around the problem is.
Things that hit me:
Removal of scrollback
Removal of ashmem (used by Anbox and some similar things)
yeah waydroid works without it @@09f9
wait anbox is dead... 💀
@@09f9 Commit 721412 staging: remove ashmem
And KH and Hellwig were fine with breaking it cause apparently Anbox is retro cause muh duh Android 7 (for now and well okay now likely forever since it was deprecated last year... unless the xdroid devs stop violating GPL and publish the code of their Anbox fork that works better than Waydroid on my X11 desktop... or unless somebody comes around and builds working up-to-date Anbox images, kek. And Anbox Cloud is still a thing iirc.) and it's easier to fix the userspace thing than to maintain ashmem or something like that...
There are many deprecated APIs in the Kernel, that are most likely never actually be removed or are rather replaced with a wrapper around newer API calls... And we're talking about aliasing 2 flags for a program to separate commands which is inherently completely transparent?
I mean, nobody actually ships a raw POSIX compliant shell at /bin/sh, most systems use bash and just symlink bash to /bin/sh because it actually just works. Everyone who doesn't do it like this with their default shell might actually be insane
AFAIK Ubuntu and Debian have dash on /bin/sh
Heh, actually witnessed this reddit thread as it unfolded. Great summary and context.
This is exactly how I feel about a lot of things on Linux. Sometimes it really just seems like change for the sake of change... like, maybe sometimes you can justify it in the name of security or maintenance burdens, but how do you justify this one? egrep and fgrep are faster to type than grep -E and grep -F... isn't that a good enough reason to keep them around for people who are used to them? And since when has GNU ever cared about complying with POSIX out of the box? I've tried writing shell scripts that work across different shells that claim to be POSIX compliant before... and let me tell you, bash has a ton of GNU extensions and doesn't follow POSIX strictly. Anyone who thinks their bash scripts are POSIX compliant is probably mistaken. Are we just being asked to live without egrep and fgrep because they weren't GNU extensions, but old POSIX extensions, and thus "not made here," meaning they can drop support for them and give the middle finger to people who complain about them not being POSIX compliant? It seems like it's probably some kind of childish GNU attitude where they feel like being able to drop support for something they only adopted because of POSIX in the first place makes them feel empowered or something. Things like this make me really think that some developers out there are just jerks who live to break people's workflows and punish people for having old habits and not keeping up with new trends, to make sure they can't continue to function on a modern system and force them out. Very much the old hippie "don't trust anyone over 30" mentality where they don't want anyone over that age to be comfortable in their spaces.
Whining about losing an alias you can setup yourself is pretty funny.
"egrep and fgrep are faster to type than grep -E and grep -F... isn't that a good enough reason to keep them around for people who are used to them?"
No not really, especially when you can alias those commands yourself if you really don't want to change a very small habit.
Here from an entirely software dev pov. When you deprecate something, you can only really count it as such, when the user gets yelled at about it by something. The compiler, the software itself (like the warning here) etc. Most programming languages let you mark your stuff for that too (even modern version of C and C++ with [[deprecated]]).
Afterwards it depends on your version numbering schema. If you use semver (or a derivative), you remove it in the next major version. If you use something else (which can still look like semver, look at the Linux kernel as an example for that), you remove when you think it was a reasonable amount of time (the longer the thing existed, the longer the wait).
In all cases you should announce the removal of it beforehand.
On the other hand, I will just cite pretty much every open source license I know of (in this specific case copied from the GPL):
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
Although if you really act in a "I don't care" kind of way, don't be surprised when more and more users stop using your software.
I saw that warning the other night and thought it was weird. Thanks for the explanation.
I would say remove from /bin or /usr/bin etc. and add "use grep -E in scripts.... " as preferred correction and "add alias egrep='grep -E' ... to ~/.bashrc" as NON PREFFERED option to the warning message.
A similar thing happened to the Type 99 - SPF type DNS record. A lot of time went in to get the record type approved and added. In the end it was deprecated and now only TXT for SPF is required. No longer needing to add both and keep them in sync.
I believe I've had them aliased since 2008.
while i dont use egrep or fgrep... i agree with you on removal of functionality,. It is kinda like ifconfig. It was simple, did the job, and ubiquious. I am in Unix and linux every day. It is nice not to have to worry what command is what depending on OS. In the case of ifconfig for ip addr., I have never found the reason for getting rid of it. This isnt like pipewire vs pulseaudio. There was real issues that some where having. Pulse audio been fine for me, however i can see the other side. I just dont get removal of stuff people actually use. Gnome file program removing compact listing for example.
I like how folks are complaining about egreg/fgrep being removed after 18 years of being deprecated, while we have OpenBSD in the corner dropping syscall in a couple months between releases lol
Linux lack of backwards compatibility it a glaring limitation compared to Windows. You are mostly fine if all you run is software from your distro. But as soon as it you look at 3rd party applications it is very random what does or doesn't work. Windows, historically has been the best on backwards compatibility but even that hasn't been good lately.
From my cold dead hands
Is it riddled with security vulnerabilities? Is it riddled with so many bugs that no one should touch it with a 3 meter pole? If not, just leave it there honestly. Not going to harm anyone by just existing
Kinda feels like how the home directory is littered with various dotfiles, due to applications not using the XDG base directory standard. For instance, bash places .bashrc et cetera directly at home dir because of 'historical reasons'... Even if we do let bash off the hook, there are tons of other applications doing the same without a proper excuse. Some things must change for the better, and yet cannot realistically be changed due to pragmatic reasons, or simply due to resistance to change.
Putting in those warnings are stupid. If GNU doesn't want to provide egrep and fgrep, distros will just make an other package to provide it as scripts. Literally nobody cares what GNU considers deprecated. User don't need to know this, because the programs will never go away, even if GNU removes them, so the warning serves no purpose other then annoying the users.
When I see something like this, I would just downgrade and freeze that package. I once had the system so broken, that I had to use "dpkg -i" directly because APT would refuse to install anything until I fix what it considered to be broken packages.
8:53: it was a clear intention in open and free BSD is same source file and same executable only sym-link to egrep and fgrep, and is was done this way probably for long time, also unix v10 (and down) nor plan 9 provides those application. So i will rather say they wasn't intended at all.
1. flag egrep/fgrep usage to system logs, not just the console, including what called it.
2. let the distros handle it with grep-legacy and grep packages. No-one wants to install something named "-legacy". New systems or major upgrades should go through validation processes so that is the time to break things, but also provide a quick fix. Having a separate package would also provides stats about usage.
While there is no reason to remove cruft "if it doesn't have much impact" the cruft problem is that it accumulates across many things.
alias egrep='grep -E'
OK, it's a little more complicated than that, but why remove the functionality even if you remove the actual program?
It's not much more complicated, that's basically what egrep does
IMO deprecating something should be saying "we'll not give you support if this is broken," not saying "we might delete this later"...
I don't know why they couldn't do like busybox does and look at what it was invoked as, so if you call grep by a `egrep` link it acts as if you used `grep -E`. Seems stupid easy, eliminates actually having egrep and fgrep codebases to maintain, and preserves compatibility.
Deprecating stuff ist part of the software lifecycle. But this case is somewhat badly organized. Usually you want to add warnings soon after deprecation and then give the community the chance to adapt. Since only now people realize the need for change it probably will take another decade before anything can be actually removed somewhat safe
Yup, I wanted to say the same thing. I am for making things cleaner and to a spec/standard. Even for very small things like fgrep and egrep which are now 2 small wrappers, they are still polluting, even if very little in practice. But I also don't want things to break so easily. So given that the warning was "just" added, supporting it for literally 10 more years, until all relevant software has got a change to update, that would not be much to ask.
Learnt about grep -F and grep -E before learning there was an egrep or fgrep. Idk, looks like either work
I don't use egrep or fgrep, but just as a test I did a grep on my /usr/bin folder to see what scripts in the system use it. There's about 13 of them using it directly, and a few that have redefined to usin $EGREP which is replaced by grep -E. There's only a few using fgrep, and one or two of them with a $FGREP define. These aren't scripts I wrote, but rather ones distributed with my OS distro. I bet I'd find even more in places like /usr/share, but that place is such a mess that I don't want to look. (oh my zsh actually defines fgrep and egrep macros though, so even if they're deprecated many people might not even notice and still use them in the scripts.)
Why was CPIO removed? I have seen it quite often in commands in android dev
To me, it's not about breaking or not breaking, it's about predictability. When you do breaking changes do them in a yearly planned releases (every 1-5)
Even if you plan the breaking changes, the users will still have to hunt down and update everything (in this case shell scripts) that use the old feature.
As long as versioning is clearly labelled and communicated I don't mind breaking changes.
That's one of the reasons why I prefer version 1.0.4 etc. over date tags like ubuntu versions.
GNU taking decades to drop support is the issue, apple deprecating features left and right in early swift was better.
it saves like 40 bytes, who cares, just leave it.
I would say to keep the egrep and fgrep as is, unless the POSIX_ME_HARDER environment variable is defined.
Your take on this is my take on Wayland
Xorg requires a lot of maintenance, a 1 line bash script doesn't
@@BrodieRobertson I'm just fucking with you I run hyprland, still using X on openBSD though.
I've seen less fuss about entire CPU architectures being removed, so why not let obsolete features just pile up in the kernel and leave it to Linux forks to remove old features?
Maintaince Burden?
20 years I ago, I would have said remove it. Now they should probably keep it.
If i wanted user space to change and break compatability I'd instead want a completely redone user space no one uses like Plan 9.
my favourite brodie filler episode. cant wait until the continuation of the current arc (wayland shilling) in the next chapter 😇
It's so ironic because x11 is like egrep
Deprecating something with no date or plan to remove is not as much a strategy as it is “wishcasting.”
If the intention is to remove it, they should make that explicit.
Imho this is not a real problem as Wayland removing X features, because there are workarounds for this (that don't require rewriting the source code of programs).
Ita an alias. No security threats, no additional dead code. An alias.
Thank you for the coreutils but we'll take it from there. The project's done, just use your foundation money to just keep looking for security vulnerabilities and thats it. Dont try to make yourself useful.
Stuff like egrep and fgrep has minimal impact on the system, but it has a bigger impact on the documentation. You said yourself that the documentation is a mess; well, having to dedicate space to describing an obsolete alias is part of what makes the documentation a mess. It's also not a great learning experience when some sites tell you to use egrep and some sites tell you to use grep -E.
I wonder if uutils will support egrep and fgrep.
I think distros should add a mandatory 20 second delay to any usage of egrep or fgrep, while they spam stderr with an error message like "error, egrep is obsolete, panicking"
I think a lot of the people who think 20 years is plenty of time are young. THe year 2000 was not a surprise, but nobody botherd reprogramming things written in 1969 until 1997.
I mean, I guess for more simple things, grep is fine... but I have largely replaced grep with sed and awk!
Yeah, 'which' on debian a couple years ago...
When I've used fgrep it's because it seemed to perform significantly better than the flag.
It is never ok to break user space.
Someone explain it to me like I'm 5, why does grep need active development, it worked perfectly fine 25 years ago, it probably doesn't need security patches, so WHY?
Because sometimes software has bugs, or dependencies change or a whole range of other reasons. This idea that you can just write software once and then never update it is completely false
In Debian Stable (currently Debian 12) both egrep and fgrep are bash scripts running grep with the corresponding option.
This is essentially just yelling 'Bad user! You should not do this! This is bad!'. And when no one listens on POSIX, start yelling from STDERR.
I don't think distros will ever remove those wrappers/aliases/functions, 'cos they, unlike GNU, are not interested in breaking things for the sake of idealism alone.
maybe adding a 10 second sleep to egrep and fgrep would do.
Note: egrep and fgrep are only "an issue" for gnu grep users.
Most other systems have removed these two long *long* time ago.
Grep can still look at $0 and figure out what it was called as. All it should take to deploy these is a couple of symlinks.
See also flash_eraseall which we're supposed to discard for a set of options to the low-level flash_erase.
Do the modern egrep and fgrep log the attempts to use them anywhere? They should so that people would be able to find any remaining uses.
Just like you can scream from the rooftops that Python 2.7 is depricated. But de facto it is as immortal as COBOL.
Have the maintainers of GNU grep ever heard of Busybox ? Why not make a VERY small piece of code which checks how grep was started ? And just make a symlink to grep itself.
Well, the only time the breaking of userspace is acceptable afaik is when a major version has been released. Now grep's 3.11, so when it goes to version 4, then they could remove it.
And then there's Kubernetes where those Google developers break userspace all the time while still in version 1.x.
I know it probably makes a headache but I feel like a lot of this could be solved by using aliases for that previous functionality....that probably sounds easy until the -E and -F add more functionality somehow.... 😅
Compromise: Add a `sleep 3` in the egrep and fgrep scripts, right after the warning.
one solution for legacy scripts would be a shim for egrep and fgrep that replaces them with a version without warnings, which could be sourced at the top of the script
egrep and fgrep will still exist. Their sole function will be to trigger grep -e and grep -f silently and pass the appropriate arguments through to grep otherwise.
One of the major successes of Windows is they NEVER break old compatibility. You can still run programs in DOS mode on windows 11. While I hate Microsoft and everything they do, this is one of the things they do right. It doesn't matter if something originated as a bug, an accident, poor planning, whatever. Once people rely on in, you can't remove it. Removing egrep and fgrep for literally no reason other than "the standards" is a terrible attitude and will not help Linux grow
While being POSIX compliant is a good thing, I can't help but think someone depreciated egrep / fgrep merely so they could claim they were accomplishing something on the POSIX standards. GNU appears to be following POSIX but Linux Distros should just ignore this and keep the sh scripts that reference grep -E or grep -F as they have been doing for many years. The sh scripts are 90 bytes or less (90 on NixOS because of the long /nix/store hash. I don't understand why any distro would intentionally break things by being a POSIX purist. One could also create system-wide shell aliases for egrep / fgrep.
What is the problem here? As you say, most people don't read the change logs so it's perfectly sensible to give a warning that they're deprecated. The point is that they haven't been a part of the standard for a long time so do not build on them. It doesn't matter if they're not actually planning to remove them, this is about what is considered "bad practice" now.
More generally, how long should projects (that aim to exist for a long time) drag some legacy features with them after those features have been deprecated? 20 years? 50 years? 100 years? Should they just accumulate obsolete features "for legacy support" and to "not break userspace" indefinitely? Nah, they should consider carefully what the future is going to be built on, mark everything else as deprecated so people know not to build on them, and eventually just remove those features.
The kernel breaks stuff all the time, so time to just break things!
A lot of these things don't seem to be 'problematic' at current, but over time they turn into opportunity costs for other things into the future. A lot of tiny crystals that set thinking into stone, rather than keeping it malleable and able to adopt new ways of doing things.
Removing tar may be so that certain embedded or minimal systems can say they're posix compliant without having to provide tar. In many embedded systems, it would be just unused, so it doesn't make sense to provide it just for POSIX compliance. Think router or fridge microcontroller.
Linus is talking about the kernel API tho. So old binaries continue working on modern kernels, as long as all the libs are in place.
I learned them, but on very mismanaged systems, so I first I thought those were "sketchy aliases to make some unsmart peoples life easy" and used "grep -E" all the time.
For me the removal of egrep has no impact.
@Brodie Robertson - is you wallpaper HexWorld Autumn from Plasma 6 wallpaper competition?
Yes I liked it so I yoinked it
Before you mentioned it, I had a passing thought of, "Don't tell me they're trying to mess with grep or something now..." And then you mentioned grep and I was like, "What is wrong with these people?! Leave grep alone!" 🤣
The only reason a change should be made is for an improvement/optimization, because of a security bug, or because it's an unused feature that nobody wants to maintain. Besides that, they should just leave this stuff alone.
I don't really use egrep or fgrep in any of my scripts, I just use grep -E and grep -F -- That said, what are non-GNU systems doing, because there may be cross *nix compatibility issues. I really only use Linux systems, and the non-GNU variants ( Alpine ) that I use haven't given me issues -but there's more than just MUSL and GNU out there in Linux userspace - I haven't used them all... and let's not even get started on the *BSD or Solaris userspaces.