Open Source

31380 readers

201 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Posts must be relevant to the open source ideology
No NSFW content
No hate speech, bigotry, etc

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 5 years ago

MODERATORS

[email protected]

453

XZ Hack - "If this timeline is correct, it’s not the modus operandi of a hobbyist. [...] It wouldn’t be surprising if it was paid for by a state actor." (lcamtuf.substack.com)

submitted 8 months ago by [email protected] to c/[email protected]

72 comments fedilink hide all child comments

Thought this was a good read exploring some how the "how and why" including several apparent sock puppet accounts that convinced the original dev (Lasse Collin) to hand over the baton.

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 9 points 7 months ago (7 children)

My guesses wildly range on this topic.

Facebook probably wanted Zstd adoption over XZ/LZMA
There was probably an analysis of who uses LZMA compression a lot, and it so happens that archivists, pirates, people and countries with low bandwidth speeds, people in Russia, game repackers et al use it a lot compared to "good law abiding" money blinded consumers of rich countries
Somebody wanted to screw over LZMA/XZ/7Z users
(most favourite right now) implanting a network backdoor into Linux servers and ecosystems
Someone thought it would be a good idea to troll open source community and make it look worse than closed source, so that closed source security can be popularised ("security" trolls in FOSS community I harp about love such ideas, beware of any Graphene/Chrome/Apple and Big Tech lovers just as example)
Tying into the idea of making FOSS ecosystem look bad, it might be a concerted effort by closed source company/companies to propel themselves above, as FOSS development is shitting on closed source corporate model
A different approach, it could be the first step in a series of steps to dismantle FOSS ecosystem, considering how much trust and transparency it has that attracts everyone enlightened enough

I could think of many other scenarios and outcomes if I put enough time, but I think this should be enough food for thought. The beneficiaries are limited, the actors few, and the methods cannot vary too much.

[–] Supermariofan67 7 points 7 months ago* (last edited 7 months ago) (3 children)

The first 3 seem incredibly far-fetched.

What exactly does Facebook gain from more people using zstd, other than more contributions and improvement to zstd and the ecosystem (i.e. the reason corporations are willing to open source stuff).
Why do you consider zlma to be loved among pirates and hackers and zstd not to be, when zstd is incredibly popular and well-loved in the FOSS community and compresses about as well as lzma?
Every person in the world uses both lzma and zstd extensively, even if indirectly without them realizing it.

I think it's likey that, of all the mainstream compression formats, lzma was the least audited (after all, it was being maintained by one overworked person). Zstd has lots of eyes on it from Google and Facebook, all of the most talented experts in the world on data compression contributing to it, and lots of contributors. Zlib has lots of forks and overall probably more attention than lzma. Bz2 is rarely used anymore. So that leaves lzma

[–] [email protected] 0 points 7 months ago (2 children)

Cloudflare deploys Zstd, and many web servers and CDNs use it. Endless possibilities for Facebook and US gov. They can put Yann Collet out of the way or gag order him.

LZMA is the highest compression algorithm outside of PAQ and SuperRep+LOLZ, while being magnitudes faster than both. Zstd compression ratio is a joke and is only good for webpage asset loading times.

[–] Supermariofan67 4 points 7 months ago (1 children)

Facebook may be evil but I don't think they're anywhere near "inject malware into global supply chains to push adoption of a public engineering side project that they don't directly profit from and most executives don't care about" level of evil. Is it possible? Sure anything is possible, but that is wildly beyond many many more plausible explanations and there's zero evidence leading us down this path. And why would they go through the trouble of backdooring zstd, which has a highly observed codebase, when they just successfully backdoored lzma because it didn't have a lot of maintainers?

While it's true that zstd is commonly favored for having "good" compression at blazingly fast speeds, which is useful on the web and on servers, Zstd 's max compression setting (zstd --long -19) is actually within about 5% of LZMA's but faster, so it replaces most use cases of LZMA except when that extra 5% (and that's not even constant; some inputs are even better on zstd) really does matter at all speed cost

[–] [email protected] 1 points 7 months ago

I have extensively benchmarked Zstd and it is a joke compared to LZMA2 when it comes to compression ratio. And not even that, the lack of features Zstd has, that 7Z does have, makes it a far bigger joke. 7Z is a feature complete archival solution unlike Zstd, with possible options for archive repair. RAR is far superior for that bitrot resistance.

The amount of possibilities Facebook and US gov get with backdooring XZ are endless, since it could destroy trust in it if uncaught, and Zstd adoption meant web malware deployment could become a matter of when, because Facebook already does it right now with actual malware JS scripts through fbcdn domain.

load more comments (3 replies)