this post was submitted on 20 Jun 2024
25 points (100.0% liked)
Operating Systems
3768 readers
10 users here now
All things operating system related, from Windows to Mac to Linux distros and the more obscure.
Subcommunity of Technology.
This community's icon was made by Aaron Schneider, under the CC-BY-NC-SA 4.0 license.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I don't think that's what's happening. There's no hard requirement for
cat
to read everything straight into memory. It can send data once it's available, and the receiving process can read it as fast as it wants. There are cases where this might be more clear: Let's say you have a big video file that you want to convert to something that only supports like y4m input and is not in ffmpeg. A common way is something likeffmpeg -i infile -f yuv4mpegpipe - | encoder --y4m outfile
- I'm pretty sure ffmpeg won't read the whole infile into memory, nor will it store the whole y4m representation in memory. Instead, it will decode infile as necessary and push into the pipe at the speed the encoder can handle.But yeah, I remember something about tar using libraries for compression being more efficient that piping its output to a compressor. So it's still the better route, but probably not as much better as you think.
Good points. Yeah, rethinking it it doesn’t make sense at all that it would read the whole thing into memory.