There I said it !
Yeah, it’s a pain. Leads to bad one liners:
for i in $(ls); do zcat $i || cat $i; doneBtw, don’t parse ls. Use
find |while read -rinstead.find -maxdepth 1 -name "term" -print |while read -r file do zcat "$file" 2>/dev/null || cat "$file" doneWon’t this cause cat to iterate through all files in the cwd once zcat encounters an issue, instead of just the specific file?
You are correct. This probably produces something more similar to what you’d want the original command to do, but with better safely:
find -- . -type f -regex '^\./[^/]*$' -exec sh -c -- 'for file in "${@}"; do zcat "${file}" || cat "${file}" || exit; done' sh '{}' '+'That assumes you want to interact with files with names like
.hidden.txt.gzthough. If you don’t, and only intend to have a directory with regular files (as opposed to directories or symbolic links or other types of file), using this is much simpler and even safer, and avoids using files in a surprising order:for i in *; do zcat -- "$i" || cat -- "$i" || exit; doneOf course, the real solution is to avoid using the Shell Command Language at all, and to carefully adapt any program to your particular problem as needed: https://sipb.mit.edu/doc/safe-shell/
Yeah, i was tired and had $file there first, then saw that you wanted to cat all in directory. Still tired, but i think this works now.
You can just do
for f in *(or other shell glob), unless you needfind’s fancy search/filtering features.The shell glob isn’t just simpler, but also more robust, because it works also when the filename contains a newline;
find .. | while read -rwill crap out on that. Also apparently you wantwhile IFS= read -rbecause otherwise read might trim whitespace.If you want to avoid that problem with the newline and still use find, you can use
find -execorfind -print0 .. | xargs -0, orfind -print0 .. | while IFS= read -r -d ''. I think-print0is not standard POSIX though.because it works also when the filename contains a newline
Doesn’t that depend on the shell?
I don’t think so and have never heard that, but I could be wrong.
Thanks !
But still we shouldn’t have to resort to this !
Also, can’t get the output through pipefor i in $(ls); do zcat $i || cat $i; done | grep mysearchtermthis appears to workfind . -type f -print0 | xargs -0 -I{} sh -c 'zcat "{}" 2>/dev/null || cat "{}"' | grep "mysearchterm"Still, that was a speed bump that I guess everyone dealing with mass compressed log files has to figure out on the fly because zcat can’t read uncompressed files ! argg !!!for i in $(ls); do zcat $i 2>/dev/null || cat $i; done | grep mysearchterm
I think that providing an exit status that is not 0 when
zcatis used with an uncompressed file is useful. Though my opinion is less strong regarding whether it should write more text after an error occurred, it’s probably more useful for a process to terminate quickly when an error occurred rather than risk a second error occurring and making troubleshooting harder.I think that trying to change any existing documented features of widely used utilities will lead to us having less useful software in the future (our time is probably better spent making new programs and new documentation): https://www.jwz.org/doc/worse-is-better.html https://en.wikipedia.org/wiki/Worse_is_better
Not improving existing software leads to stagnation.
It’s certainly a good part of why so much of linux is an awkward kludgy idiosyncratic mess to use.
Whatever the first implementation does ends up being a suicide pact by default.
Another option is to change cat to auto decompress compressed files, instead of printing gibberish.
What operating system should I use with my laptop that isn’t an awkward kludgy idiosyncratic mess? I would say that Windows has plenty of kludges, like having problems with certain file names. Many versions of macOS are UNIX® Certified Products (for example, macOS version 15.0 Sequoia on Intel-based Mac computers and on Apple silicon-based Mac computers), so it’s surely not any less kludgy than Linux.
I suppose that it’s not bad to change documentation to be more specific, and change a program such that it matches the new documentation and wouldn’t cause any harm if it replaced all the existing versions of the program, but makes it possible to use the program to solve more problems. That would be to “add functionality in a backward compatible manner”.
You are also free to create new programs that are not an exact replacement for existing programs, but can enable some people to stop using one or more other programs. That would not be what I describe as stagnation.
“The cat utility shall read files in sequence and shall write their contents to the standard output in the same sequence.”, so I would be very annoyed if it did something different with a certain file but not others. I wouldn’t say that the contents of a file and the contents after the file is expanded are the same. In fact, I expect that some people use
catto process compressed files, and changing howcatacts with compressed files would probably cause them a large amount of annoyance, and would needlessly make a lot of existing documentation incorrect.Whatever the first implementation does ends up being a suicide pact by default.
I agree. The behavior of
rmandcatandcpandmvandddand many other utilities don’t necessarily have the interface I would prefer, but they are too widely used for it to be helpful to radically change them. It’s somewhat unfortunate that these names are already reserved, but I don’t think it’s necessary to change them.In the same way, I don’t have a problem with packages having generic names but not actually being useful: I’ve read that the
requestsandurllib3packages for Python aren’t being maintained very well, but I don’t mind that as long as I can accomplish things while following best practices.Because of this, I’m not afraid to use names like “getRequest” or “result”, especially if they were generated with an automatic refactoring, and I’m not upset when I see similarly generic names being used with source code I’m changing, since I know that the second name for something that’s similar to an existing thing will have to actually be descriptive, but the first name is likely to not be.
I have another example of how I’d apply these thoughts: the process for developing v2+ modules for the Go programming language strikes me as inelegant, so I would probably prefer to just create an entirely new repository rather than try to attempt that.
Well in this particular case, zcat failing with error on uncompressed text isn’t a behaviour worth preserving.
It should do the expected zcat behaviour, which is just print the text.
I have a hard time imagining a scenario where you call zcat and would prefer an error rather than a useable output
I can imagine that someone would find a program like this to be useful, and depends on the presently common behavior of
zcat, so I expect this is an important part of a system used by a corporation I interact with (and probably many more than I’d expect):if zcat ./file.txt.gz >/dev/null then process_file ./file.txt.gz else printf '%s\n' "There was a decimal exit status of ${?}" fiA failure to understand whether something is useful is not a good reason to change it.
Celeste. Are you here? In a future search maybe?
Well, the source code is available. Fix it if you need it that bad.
Where is it? I can’t seen to find it https://github.com/zCat?tab=repositories
