this post was submitted on 10 Oct 2024
104 points (100.0% liked)

Open Source

30750 readers
441 users here now

All about open source! Feel free to ask questions, and share news, and interesting stuff!

Useful Links

Rules

Related Communities

Community icon from opensource.org, but we are not affiliated with them.

founded 5 years ago
MODERATORS
 

Are there any alternatives to the Internet Archive that are built around P2P, so that everyone can contribute to hosting/sharing web archives? Seems like having all these important archives hosted by a single organization isn't the best idea for longevity/redundancy

@opensource

top 16 comments
sorted by: hot top controversial new old
[–] CameronDev 31 points 1 week ago* (last edited 1 week ago) (1 children)

There was a ActivityPub wiki clone, no idea where it got to.

The major upside of IA being built and owned by one central company is trust. We can (so far at least, if I'm wrong please correct me) trust IA to not censor/rewrite history. As soon as every man and his dog can contribute, that gets a lot harder to guarantee.

Edit: https://github.com/Nutomic/ibis

Don't take me linking it as endorsement, I think federated wiki's for anything other than fandom stuff to be madness.

[–] [email protected] 20 points 1 week ago (2 children)

Sounds like a potential application for a blockchain techs, that allows to do verifications, voting and consensus.

[–] [email protected] 5 points 1 week ago

Ive had similar ideas on verified unaltered media to combat fake news. Such an archive but for journalism would be a great start.

[–] CameronDev 4 points 1 week ago* (last edited 1 week ago) (1 children)

Yeah, quite possibly. Could still be very hard to get right. Region blocking might make consensus difficult.

Edit: just occurred to me, any method of consensus could be used to ddos sites as well. Might be best left for people smarter than me

[–] [email protected] 2 points 1 week ago (1 children)

I think that proof-of-work approach to blockchain can make ddos attacks much harder, but I'm not an expert too :)

[–] CameronDev 1 points 1 week ago (2 children)

I figured that every node would need to scrap the site, in order to validate the content. If there are thousands of nodes, that would ddos the site.

I don't really understand how PoW would solve that, can you explain?

[–] [email protected] 1 points 1 week ago* (last edited 1 week ago)

I figured that every node would need to scrap the site, in order to validate the content. If there are thousands of nodes, that would ddos the site.

with cryptocurrencies the blockchain is distributed, its not stored centrally. your idea could work similarly

[–] [email protected] 0 points 1 week ago (1 children)

I think it can be done like a NFTs on top of Bitcoin. In this case evey archived page is NFT and all the blockchain is available, so there is no centralized cite. If each action will require some computations (PoW) then ddos attack or spam attack will be very hard to implement.

[–] CameronDev 2 points 1 week ago* (last edited 1 week ago)

Thats for proving its untampered with right? I'm more thinking of validating the archive copy is a "true" copy when adding it initially, which requires each node to check against the live site?

Its definitely an intriguing idea though, but I don't know enough to know how feasable it can be

[–] [email protected] 21 points 1 week ago

I think there was an officially-lead attempt to distribute internet archive over IPFS but I'm not sure what it became

[–] [email protected] 15 points 1 week ago (1 children)

We have already the technology for sharing with P2P, lot of Torrent websites. The problem is, sharing with P2P is illegal, if you don't have the rights to. Internet Archive is different, because they have special rights we users don't have, such as being a library in the US with rights of a library.

In example Internet Archive is allowed to share books, but only lend one at a time for each real local copy they have.

Downloading from Internet Archive is harmless and only Internet Archive and your ISP knows your IP when doing so. But with P2P (if we are talking about Torrents) everyone who connects to it knows the IP from everyone else. There is a higher risk for uploading stuff, or sharing with others. Basically the government can join the P2P network and suddenly see everyone.

[–] [email protected] 0 points 6 days ago

You are vastly misinformed Libraries do not have special rights The Internet Archive was recently and previously sued for exactly what libraries do They also share files via direct download and torrents P2P is not illegal Sharing files you don't have the rights to is illegal I share many files from the Internet Archive via torrent from home

[–] [email protected] 10 points 1 week ago* (last edited 1 week ago)

That's what private trackers (and torrents in general) are for, but private trackers commonly share copyrighted materials, which is of course not so legal depending on where you live.

[–] [email protected] 5 points 1 week ago (1 children)

You can store stuff in the IPFS. But probably, someone has to build some archiving tool around it. But you would be vulnerable to copyright claims.

[–] [email protected] 2 points 1 week ago* (last edited 1 week ago) (1 children)

This is what i never understand about attempts at P2P hosting of copyright content. It only works BECAUSE its obscure and irrelevant. As soon as it gets popular, all of these people and their VPN providers would get crushed by lawsuits.

Dont get me wrong i love P2P tech, but it will never solve these issues unless the laws are fixed.

[–] [email protected] 2 points 6 days ago

If VPN's actually won't be able to protect its users from copyright claims anymore, there'll still be anonymisation networks like I2P (at least so long as encryption isn't banned).

Yes, it's slow atm, but if it was included in more torrent clients and enabled by default, speeds would likely get better.