this post was submitted on 31 Jul 2023
100 points (100.0% liked)

Meta (lemmy.one)

1004 readers
1 users here now

A place to discuss or ask anything about lemmy.one's instance or moderation.

For discussion about Lemmy (the software) itself, visit [email protected]

founded 1 year ago
MODERATORS
 

As you reading are undoubtedly aware now, the Lemmy.one instance experienced a massive failure this weekend. Unfortunately the data was not in a recoverable state, so the server was restored from a July 26th backup, and data after that time is likely lost.

Since this occurred while I was out of town, I haven't had the time to collect all the details of what happened, so I will have to post more information at a later time.

all 44 comments
sorted by: hot top controversial new old
[–] [email protected] 24 points 1 year ago (3 children)

My biggest concern was that there was no way to find information about what was going on. Your Mastodon had no information.

There was a thread on Lemmy support about it and some people thought the instance was gone forever.

[–] [email protected] 6 points 1 year ago (1 children)

I’m in the same boat as you. Although I was told to check http://lemmy-status.org/ to see the status of the instance, truth is I didn’t know if Lemmy.one was ever going to be up again.

[–] [email protected] 3 points 1 year ago

I had no idea this existed!

[–] [email protected] 5 points 1 year ago (1 children)

I got confused, and ended up making an alternative account in another instance. Glad Ido not have to worry about it, but redundancy is a virtue.

[–] [email protected] 2 points 1 year ago (1 children)

With each day with no news I started considering making another lemmy account in case this one didn't come back. I didn't know which one to make my new home though, since I liked privacyguides run one the best.

[–] [email protected] 2 points 1 year ago

I think this is a wonderful instance and I am happy to be in it.

I hope it stays stable.

[–] [email protected] 4 points 1 year ago (1 children)

Yeah really would be good to have a separatly hosted status page or something, but I understand that is a luxury. Considering Jonah has freely volunteered to host it's hard to complain.

[–] [email protected] 16 points 1 year ago

Thank you for your work! No one is expecting you to be always available, considering this is voluntary work you are doing.

[–] [email protected] 10 points 1 year ago

@jonah can this post be unpinned? Don’t think I need it at the top of my feed anymore :)

[–] [email protected] 8 points 1 year ago

Why is this still pinned?

[–] [email protected] 7 points 1 year ago

Just another appreciation comment here. Thank you! Glad it was just temporary, and hope it doesn't cause you too much stress.

[–] [email protected] 7 points 1 year ago

Thanks for getting it back up.

[–] [email protected] 7 points 1 year ago

Can we please remove this as a pin?

[–] [email protected] 6 points 1 year ago

ty 4 ur work 🙏 look after yourself.

[–] [email protected] 5 points 1 year ago (1 children)

Thank you for your hard work none-the-less.

I had a feeling it was just that. I heard rumbles about some type of issue in the lemmy software itself; although I don't know the exact details. It appears similar problems happened to other instances as well; so don't take it too hard...several massive instances did in fact crash.

[–] [email protected] 2 points 1 year ago (1 children)

I noticed lemmy.world started displaying the same server error as lemmy.one a day or so after (I switched over to that instance after this one had issues).

[–] [email protected] 2 points 1 year ago

...And lemmy.world just went down again.

[–] [email protected] 5 points 1 year ago

Thank you for your work! I'm glad everything is back up and running.

[–] [email protected] 4 points 1 year ago

Unpin this. It's 3 months old.

[–] [email protected] 3 points 1 year ago* (last edited 1 year ago) (1 children)

Glad to be back; Thanks, man!

What's up with browsing lemmy.world? Did it de-federate from us here or is it a syncing issue?

[Edit]Fine now

[–] [email protected] 5 points 1 year ago

It seems that lemmy.world is expirencing server issues so thats likely it

[–] [email protected] 3 points 1 year ago

Thanks for the hard work and expense of hosting. Appreciate it

[–] [email protected] 3 points 1 year ago (1 children)

Does anyone know why this is still showing up at the top of my "active" feed?

[–] [email protected] 1 points 1 year ago

I think that's because it's pinned by the instance admin - if you unsub from this meta community it should disappear from your feed

[–] [email protected] 2 points 1 year ago (1 children)

I never used mastodon so much in my life.

[–] [email protected] 2 points 1 year ago

Same! 😅 huge thanks to @[email protected] for bringing lemmy.one back online. I wasn't ready to mourn the loss of my fediverse homepage.

[–] [email protected] 2 points 1 year ago

Welcome back! We were all terribly worried about the outage and the radio silence. Thanks for posting up an explanation!

[–] [email protected] 2 points 1 year ago

Thank you for your hard work! I love this instance the best.

[–] [email protected] 2 points 1 year ago* (last edited 1 year ago)

Uptime robot can provide both monitoring and status page for the user. Could be an option?

https://uptimerobot.com/status-page/

If something where to happen I think a message could be added to the status page as well.

[–] [email protected] 2 points 1 year ago

Thanks for your work on this instance!

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

So I have a question. Since the last backup using which the server was on the 26th, there are several posts and comments missing. But since most of these comments and posts originate from other instances and are still there, why can't lemmy.one pull these comments/posts from the instances¿?

Also Jonah thanks for all the hard work.

[–] [email protected] 1 points 1 year ago (1 children)

Are new signups not being accepted? It's been nearly a week since I applied. I thought I answered the questions appropriately. Other instances accepted relatively quickly, within a couple of days.

[–] [email protected] 1 points 1 year ago

Life's probably happened I think, he hasn't uploaded to his YT or been on the Techlore Talks podcast in a while

[–] [email protected] 1 points 1 year ago (1 children)

And there is lies the point of failure. If we generate posts and so on...will they also disappear if a particular lemmy server disappears?

[–] [email protected] 2 points 1 year ago

No - the posts and comments you make are copied to other federated servers

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago) (1 children)

Thanks for all you do! Can someone (either you or someone that knows) post a link to your funding page? I've been meaning to set up a donation since this is my preferred instance but haven't got around to it. Missing it all. Weekend is a good reminder!

[–] [email protected] 2 points 1 year ago
[–] [email protected] 1 points 1 year ago* (last edited 1 year ago) (1 children)

Is it me, or are some of our comments missing? I feel like I made some comments on another server, but they are not there. I guess I don't understand how this works.
By the way, thanks for your help.

[–] [email protected] 3 points 1 year ago* (last edited 1 year ago) (1 children)

restored from a July 26th backup

[–] [email protected] 2 points 1 year ago (1 children)

I don't underhand why the comments disappeared from a different server.

[–] [email protected] 1 points 1 year ago

Maybe because that server synced with this server, assuming this is your home instance. If the comments aren't on any record here, they might get deleted from the other server.

Option 2: you attempted to make those comments but they never went through because this server was already having problems. So they never truly existed on the other server?

I'm just guessing, I don't know how the hell this fediverse stuff works.

[–] [email protected] 0 points 1 year ago* (last edited 1 year ago)

I’m just glad it’s back up. Thanks for the hard work.