this post was submitted on 03 Oct 2023
215 points (95.4% liked)

Linux

48137 readers
500 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
 

if you could pick a standard format for a purpose what would it be and why?

e.g. flac for lossless audio because...

(yes you can add new categories)

summary:

  1. photos .jxl
  2. open domain image data .exr
  3. videos .av1
  4. lossless audio .flac
  5. lossy audio .opus
  6. subtitles srt/ass
  7. fonts .otf
  8. container mkv (doesnt contain .jxl)
  9. plain text utf-8 (many also say markup but disagree on the implementation)
  10. documents .odt
  11. archive files (this one is causing a bloodbath so i picked randomly) .tar.zst
  12. configuration files toml
  13. typesetting typst
  14. interchange format .ora
  15. models .gltf / .glb
  16. daw session files .dawproject
  17. otdr measurement results .xml
you are viewing a single comment's thread
view the rest of the comments
[–] Supermariofan67 48 points 1 year ago (8 children)

Ogg Opus for all lossy audio compression (mp3 needs to die)

7z or tar.zst for general purpose compression (zip and rar need to die)

[–] [email protected] 23 points 1 year ago (1 children)

The existence of zip, and especially rar files, actually hurts me. It's slow, it's insecure, and the compression is from the jurassic era. We can do better

[–] [email protected] -5 points 1 year ago (1 children)

@dinckelman @Supermariofan67 I think you mean unsecure. It doesn't feel unsure of itself. 😁

[–] [email protected] 14 points 1 year ago (2 children)

in·se·cure (ĭn′sĭ-kyo͝or′) adj.

  1. Inadequately guarded or protected; unsafe: A shortage of military police made the air base insecure.

https://www.thefreedictionary.com/insecure

Unsecure

a. 1. Insecure.

https://www.thefreedictionary.com/Unsecure

[–] [email protected] 4 points 1 year ago (1 children)
[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

One thing I didn't appreciate about English until reading a Europe forum for a while is that it has a lot of different prefixes that mean something like "not", and this is not very intuitive to people learning the language. Their use is not regular.

Consider:

  • "a-" as in "atypical"

  • "non-" as in "nonconsentual"

  • "un-" as in "uncooperative"

  • "im-" as in "immortal"

  • "in-" as in "inconsiderate"

  • "il-" as in "illegitimate"

  • "mal-" as in "maladjusted"

  • "anti-" as in "anti-establishment"

  • "de-" as in "deconstruct"

And sometimes, some of the prefixes are associated with base words to form real words with similar meanings, but meanings that are not the same. For example, "immoral" and "amoral" do not mean the same thing, though they have related meanings.

[–] [email protected] 3 points 1 year ago

@hungprocess Also this. https://english.stackexchange.com/questions/19653/insecure-or-unsecure-when-dealing-with-security

It seems that I was quite wrong, but that a lot of other people are wrong as well. lol

[–] [email protected] 6 points 1 year ago (2 children)

why does zip and rar need to die

[–] Supermariofan67 13 points 1 year ago

Zip has terrible compression ratio compared to modern formats, it's also a mess of different partially incompatible implementations by different software, and also doesn't enforce utf8 or any standard for that matter for filenames, leading to garbled names when extracting old files. Its encryption is vulnerable to a known-plaintext attack and its key-derivation function is very easy to brute force.

Rar is proprietary. That alone is reason enough not to use it. It's also very slow.

[–] [email protected] 8 points 1 year ago (1 children)

Again, I'm not the original poster. But zip isn't as dense as 7zip, and I honestly haven't seen rar are used much.

Also, if I remember correctly, the audio codecs and compression types. The other poster listed are open source. But I could be mistaken. I know at least 7zip is and I believe opus or something like that is too

[–] [email protected] 3 points 1 year ago (2 children)

Most mods on Nexus are in rar or zip. Also most game cracks; or as iso, which is even worse.

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago)

I have seen RAR on Nexus, but I wouldn't say that it's common, at least for Bethesda's games, which is where I've seen it.

Things may have changed, but I recall that yenc (for ASCII encoding), RAR (for compression and segmenting) and PAR2 (for redundancy) were something of a standard for binary distribution on Usenet, and that's probably the main place I've seen RAR. I think that the main selling point there was that it was just a format that was widely-available that supported segmented files.

[–] [email protected] 1 points 1 year ago (1 children)

That would explain why I don't see them often. I haven't been very active in gaming as of late, let alone modding. And I generally don't pirate games. I'm cool with people that do, I just don't personally. (Virus fears, being out of the loop long enough that I don't know any good sites, etc)

[–] [email protected] 2 points 1 year ago

virus fears

Honestly, if desktop operating systems supported better sandboxing of malware, I bet that piracy would increase.

[–] [email protected] 4 points 1 year ago (2 children)
[–] Supermariofan67 15 points 1 year ago

It's a 30 year old format, and large amounts of research and innovation in lossy audio compression have occurred since then. Opus can achieve better quality in like 40% the bitrate. Also, the format is, much like zip, a mess of partially broken implementations in the early days (although now everyone uses LAME so not as big of a deal). Its container/stream format is very messy too. Also no native tag format so it needs ID3 tags which don't enforce any standardized text encoding.

[–] [email protected] 6 points 1 year ago

Not the original poster, but there are newer audio codecs that are more efficient at storing data than mp3, I believe. And there's also lossless standards, compared to mp3's lossy compression.

[–] [email protected] 3 points 1 year ago (1 children)

How about tar.gz? How does gzip compare to zstd?

[–] Supermariofan67 5 points 1 year ago

Both slower and worse at compression at all its levels.

[–] [email protected] 3 points 1 year ago (1 children)
[–] [email protected] 23 points 1 year ago (1 children)

Big file size for rather bad audio quality.

[–] [email protected] 4 points 1 year ago (5 children)

I've yet to meet someone who can genuinely pass the 320kbps vs. lossless blind-test on anything but very high-end equipment.

[–] Supermariofan67 11 points 1 year ago (2 children)

People are able to on some songs because mp3 is poorly optimized for certain sounds, especially cymbals. However, opus can achieve better quality than that at 128k with fewer outliers than mp3 at 320k, which saves a lot of space.

[–] [email protected] 0 points 1 year ago

False. OPUS achieves transparency at 192kbps compared to 320kbps for LAME MP3.

[–] [email protected] 0 points 1 year ago

Oh, yeah, not arguing that Opus is the superior format. It 100% is. Not questioning that.

Indeed, the first place that gets hit by lower bitrates with MP3 is high frequencies. MP3 does have a pretty harsh cutoff at very high frequencies... that the vast majority of equipment can't reproduce and most ears can't hear. It's relatively debated, some claim to be able to "feel" the overtones or something like that. I'm extremely sceptical, if I'm being honest. Last time I did the test - must have been a decade ago - I couldn't distinguish lossless and high bitrate MP3 any more accurately than a coin toss.

[–] [email protected] 8 points 1 year ago (1 children)

We're not talking lossless. The comment above specified Opus-encoded OGG, which is lossy.

For example, I converted my music library from MP3 to OGG Opus and the size shrank from 16 GB to just 3 GB.

And if converting from lossless to both MP3 and OGG Opus, then OGG does sound quite a bit better at smaller file sizes.

So, the argument here is that musicians are underselling their art by primarily offering MP3 downloads. If the whole industry would just magically switch to OGG Opus, that would be quite an improvement for everyone involved.

[–] [email protected] 0 points 1 year ago (1 children)

Yeah, I'm aware. I probably wasn't clear. I think MP3 is just the default cause of immobilism. People still using "physical" medium/digital libraries rather than streaming are becoming a rare breed, and MP3 is just... good enough. Also familiarity - I remember googling "some song - some artist mp3" being the easy way to find single titles in my teenage years lol, if I wasn't aware of the new codecs, I'd probably default to MP3 without asking myself the question.

[–] [email protected] 1 points 1 year ago (1 children)

Well, I understood this post to mean, if you had a wish, what would you wish for? Not necessarily that it's realistic...

I do agree with your points. Although, I can't help but feel like more people would prefer local files, if those actually sounded better than the bandwidth-limited streaming services.

[–] [email protected] 1 points 1 year ago

I was expanding on the subject, generating side discussion stuff. Maybe I came across as standoffish - if that's the case, I apologize.

I'm not sure how much people would care... Even back then, convincing people around me that their 128kbps MP3 sounded like it played on a tiny dollar store external speaker playing in a shower was almost impossible. Tons just download MP3s off of YouTube and call it a day. So many people don't seem to care, unfortunately.

Convenience is the best motivator, IMHO. Downloading MP3s and loading them on your MP3 player used to be easy. You had sites literally letting you download songs directly. Torrents were big. Hell, going back, eMule/Kazaa, even Limewire, all was much easier than buying CDs and ripping them, or even when buying from online stores became a thing, with the DRM early on, etc, downloading was much less hassle.

Now people pay one price and get to listen to all the music they want to listen to.

[–] [email protected] 4 points 1 year ago

yeah, but guess what opus is transparent at? depending on the music, anywhere from 96kbps to 160kbps.

[–] [email protected] 4 points 1 year ago

320 kbps is approaching lossless audio compression bitrates.

Opus does better in about half the space. And goes down to comically low bitrates. And his obscenely small latency. It's not simple, but hot dang, is it good.

The Quite Okay Imaging guy did a Quite Okay Audio follow-up, aiming for aggressive simplicity and sufficient performance, but it's fixed at a bitrate of 278 kbps for stereo. It's really competing with ADPCM for sound effects in video games.

Personally, I think an aggressively simple frequency-domain format could displace MP3 as a no-brainer music library format, circa 128 kbps. All you have to do is get forty samples out of sixty-four bits. Bad answers are easy and plentiful. The trick is, when each frame barely lasts a millisecond, bad answers might work anyway.

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago) (1 children)

I think that people overstate MP3's losses, and I agree at 320k that it's inaudible, but I can or at least have been able to tell at 128k, mostly with cymbals. Granted, cymbals aren't that common, but it's nice to not have them sound muddy. And, honestly, there just isn't a lot of reason to use MP3 for anything compressed today, other than maybe hardware decoding on very small devices and widespread support. There are open standards that are better.

[–] [email protected] 0 points 1 year ago

Oh, for sure. See my other comments. Opus is probably my favorite as well.

[–] [email protected] 2 points 1 year ago (1 children)

its worth noting that aac is actually pretty good in a lot of cases too

[–] Supermariofan67 7 points 1 year ago (1 children)

However, it is very patent encumbered and therefore wouldn't make for a good standard.

[–] [email protected] 2 points 1 year ago

aac lc and he-aac are both free now hev2 and xhe aren't, but those have more limited use

[–] [email protected] 1 points 1 year ago (1 children)

How about xz compared to zstd?

[–] Supermariofan67 2 points 1 year ago

At both algorithms' highest levels, xz seems to be on average a few percent better at compression ratio, but zstd is a bit faster at compression and much much faster at decompression. So if your goal is to compress as much as possible without regard to speed at all, xz -9 is better, but if you want compression that is almost as good but faster, zstd --long -19 is the way to go

At the lower compression presets, zstd is both faster and compresses better

[–] [email protected] -2 points 1 year ago* (last edited 1 year ago) (1 children)

(mp3 needs to die)

How are you going to recreate the MP3 audio artifacts that give a lot of music its originality, when encoding to OPUS? Past audio recordings cannot be fiddled with too much.

Also, fuck Zstandard, its a problematic format due to single file compression ability, hard to repair, not fully stable and lacking too many features compared to 7Z/RAR. Zst is also 15-20% worse at compression ratio. Its only a good format for temporary fast data transit applications (webpage/CDN serving, quick temporary database backups).

[–] [email protected] 1 points 1 year ago (1 children)

How are you going to recreate the MP3 audio artifacts that give a lot of music its originality, when encoding to OPUS?

Oh, a gramophone user.

Joke aside, i find ogg Opus often sounding better than the original. Probably something with it's psychoacoustic optimizations.

[–] [email protected] -1 points 1 year ago

The artifacts can determine the quirky sounds. They are like film grain in MPEG/WMV/AVI or the old VHS rips, you cannot recreate them in OPUS, because they get recorded that way. The recording gear and the mastering determines how the streaming audio should be encoded. OPUS probably is better sounding now with Sox Resampler equipped audio players.