this post was submitted on 15 Oct 2023
29 points (100.0% liked)

Programming.dev Meta

2442 readers
1 users here now

Welcome to the Programming.Dev meta community!

This is a community for discussing things about programming.dev itself. Things like announcements, site help posts, site questions, etc. are all welcome here.

Links

Credits

founded 1 year ago
MODERATORS
snowe
Ategon
29
Test object storage (programming.dev)
submitted 10 months ago by snowe to c/meta
11 comments fedilink hide all child comments
 
top 11 comments
sorted by: hot top controversial new old
[–] snowe 20 points 10 months ago (1 children)

IT LIVES

[–] gamma 5 points 10 months ago

Glorious

[–] kogasa 15 points 10 months ago

Glad to see you're back up and running. 👏

[–] Lodra 5 points 10 months ago (2 children)

So if you don't mind sharing, what happened?

[–] snowe 19 points 10 months ago

yeah in addition to that I'll do a full writeup, but had several problems

  1. lemmy-ansible had pict-rs set to 0.4.0, which is quite an old version, meaning migration couldn't occur concurrently (it was copying files to object storage sequentially)
  2. upon stopping the migration at one point and attempting to restart (on a newer pict-rs version) the server had completely filled up, so the migration could no longer continue on either version
  3. therefore I had to mount more block storage, copy the entire drive over to the block storage, in order to perform the migration.

there were lots more issues, but copying hundreds of gigs many times over is a cause to lot of the downtime.

[–] Ategon 15 points 10 months ago* (last edited 10 months ago) (3 children)

Site ran out of storage due to pict-rs storage growth (280GB) (pict-rs is used for images on lemmy) so was being migrated over to object storage today. Theres some posts about it over on the mastodon account https://mastodon.social/@programming_dev

[–] burtek 5 points 10 months ago

Wish I knew about this mastodon account earlier tbh

[–] Lodra 3 points 10 months ago* (last edited 10 months ago)

Thanks for the info! Thanks also for the link! I wasn't aware of the mastodon account 🙂

[–] cmeerw 1 points 10 months ago (1 children)

So that's 280 GB in roughly 5 months? At what rate is it growing now?

[–] Ategon 1 points 10 months ago

that would have to be something for snowe to answer since I dont know the numbers on that

[–] intelati 2 points 10 months ago

NICE!