this post was submitted on 14 Jan 2024
11 points (100.0% liked)

important instance shit

150 readers
1 users here now

founded 2 years ago
MODERATORS
 

today's (later than planned) upgrade to lemmy 0.19.2 provisionally appears to have gone alright. if you see excessive amounts of jank (and your page footer can't decide what version of lemmy it's running on, IE it shows separate FE and BE versions), clear your browser cache and cookies since lemmy doesn't seem to do that cleanly on its own

next up I'm planning to deploy the Proton frontend as an alternative to the default and I'm also going to start pushing code to codeberg (most likely) so stay tuned for that

all 41 comments
sorted by: hot top controversial new old
[–] [email protected] 6 points 11 months ago (3 children)

In today's episode of awful systems I found outt my phone's browser can't manage individual site cookies. Deleting them is an all-or-nothing deal.

Tfw the mono no aware hits you when you have to log out of everything just to make fun of techbros.

[–] [email protected] 5 points 11 months ago (4 children)

christ I’m sorry. my phone probably needs the same — the mother fucking cache-control immutable header some lemmy dev decided to hardcode into lemmy-ui’s app server seems to have triggered a bug in my phone where I’m both logged in and not simultaneously, and switching tab groups logs me out until I open a new tab

since all this unpleasantness is due to a bad cache-control header, hopefully it should all clear up when caches expire and pull in the new version of the site and corrected cache settings… in like 20 hours. fuck

[–] [email protected] 6 points 11 months ago (1 children)

They were using immutable cache control headers on resources that were, in fact, mutable? Uh oh.

[–] [email protected] 5 points 11 months ago (2 children)

oh yeah! it’s been causing a ridiculous number of problems for our instance and others (I thought beehaw was down for a long time but I’m realizing it was this cache issue) and basically makes upgrading to a backend that isn’t completely compatible with the previous backend impossible to do cleanly. and yet, I think I might have been the only one who’s discovered this issue, judging by the lack of relevant GitHub issues or other chatter about it.

as for why some lemmy dev chose immutable, all I can figure is they saw that rust is immutable (and I like rust a lot, but these are not good rust devs) and figured that meant immutability is a good thing everywhere regardless of the actual implementation. or maybe Google’s SEO tools insisted they needed it, and they didn’t read on or understand the versioned path requirement for cache busting? you’d think the devs of a federated platform would focus on literally anything other than SEO, but none of these folks had a plan.

anyway… only 7 hours remain until everyone’s browser hopefully marks the old content stale

[–] [email protected] 3 points 11 months ago (1 children)

I saw a few closed issues relating to cache control stuff. They've fixed some bugs related to it and also saw at least one where more aggressive cache control was implemented to improve performance.

[–] [email protected] 3 points 11 months ago

oh hey speak of the devil https://github.com/LemmyNet/lemmy-ui/issues/2325 so this isn’t even purely a cache issue, somehow they also broke upgrading credentials between lemmy versions and nobody noticed

fucking brilliant. I might see if I can push a new version of the error page that doesn’t lie about what’s wrong now that caching hopefully isn’t busted

[–] [email protected] -2 points 11 months ago (1 children)

Lemmy-ui immutable static files have a cache busting hash in their path:

On awful.systems, I see you have replaced this hash with an undefined:

By the way, clean upgrades are completely possible if you know what you're doing. I frequently roll out new upgrades on lemm.ee without any downtime (so some servers running 0.19.1, some running 0.19.2 for example).

[–] [email protected] 3 points 11 months ago (1 children)

By the way, clean upgrades are completely possible if you know what you’re doing.

oh do tell

[–] [email protected] -1 points 11 months ago (1 children)
[–] [email protected] 3 points 11 months ago (1 children)

no thx but while we’re here

the reason why the cache busting broke is because I’m building for an immutable environment (NixOS sans docker) and my builds don’t have .git cause it’s a source of mutability. lemmy’s cache busting utterly breaks without git but it still incorrectly marks the files as immutable, breaking a ton of users. utterly broken behavior.

but the very funny part is the cache behavior isn’t even the bug that’s breaking everything right now. that’s this bug: https://github.com/LemmyNet/lemmy-ui/issues/2325 which you didn’t mention as a factor while you were trying your best to sound smart

also, have you ever gotten banned from an instance for being a shithead reply guy before? I’m betting this ain’t the first time

[–] [email protected] 4 points 11 months ago

I frequently roll out new upgrades on lemm.ee without any downtime (so some servers running 0.19.1, some running 0.19.2 for example).

also motherfucker, those versions are compatible! no shit there’s no downtime for a hotfix upgrade. we got into this mess upgrading from 0.18.4 to 0.19.2 cause nobody seemed to think the upgrade process through in any detail

for anyone interested, the hack that NixOS uses to fix the asset path is here. the config we’re running predates that, and we may not be able to use that hack with a flakeified lemmy-ui (which is what makes it a hack — using src.rev like that is brittle and feels like it breaks Nix’s abstractions. maybe using git hashes like this is actually a bad fucking idea, huh? and the real fucked up part is, the UI knows its version so they could use that, but they woke up and chose jank)

[–] [email protected] 4 points 11 months ago* (last edited 11 months ago) (1 children)

Haha wow, now it's hitting me with the same thing I think. Keeps logging me out randomly and putting me in a state of Schrödinger's session cookie.

All hail the jank!

Edit: seems to happen if I refresh the page.

[–] [email protected] 3 points 11 months ago

yep, exactly what I’m seeing except it stops when I open awful.systems in a new tab, but happens again if I switch tab groups. I don’t know how they managed this

[–] [email protected] 3 points 11 months ago* (last edited 11 months ago)

Only two hard problems in CS: cache invalidation, naming things and off-by-one errors. But especially cache invalidation and especially especially when you make it even harder lol.

[–] [email protected] 2 points 11 months ago

no such luck lol

[–] [email protected] 3 points 11 months ago

this is also in Chromium for desktop

[–] [email protected] 3 points 11 months ago

This happened to me with mobile Firefox too, RIP my sessions

[–] [email protected] 6 points 11 months ago (1 children)

this is what happens when you piss off elite hackers like musk and yudkowsky

[–] [email protected] 4 points 11 months ago (1 children)
[–] [email protected] 5 points 11 months ago

And we will finally stop showing version numbers below every page here on awful.systems (waste of screen space & compute). Literally everyone knows lemmy never breaks during upgrades …

[–] [email protected] 5 points 11 months ago* (last edited 11 months ago) (1 children)

Not sure if this is related, but any time I try to upvote or downvote a comment, I get this error:

Error: invalid_bot_action
[–] [email protected] 3 points 11 months ago (1 children)

huh, I’ve never seen that before. let me look into this new jank (maybe your account somehow got marked as a bot account?)

[–] [email protected] 4 points 11 months ago* (last edited 11 months ago) (1 children)

fucking waaaaait is that why there’s a B next to your name? check your profile page and see if the “user is a bot” checkbox is enabled

[–] [email protected] 3 points 11 months ago

[b]lessed [b]e the jank

[–] [email protected] 5 points 11 months ago

I have used my immense technical skills nurtured over the course of decades to bypass the error page.

I am pleased to report that I have successfully cleared my cookies.

[–] [email protected] 5 points 11 months ago (1 children)

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

[–] [email protected] 3 points 11 months ago
[–] [email protected] 4 points 11 months ago (1 children)

Looks like it reset 2fa on accounts, as well

[–] [email protected] 3 points 11 months ago

checked the patch notes and that part is intentional. there was apparently bad 2fa UI before, which got people locked out, the reset is intended to rescue their accounts

[–] [email protected] 4 points 11 months ago* (last edited 11 months ago)

Sharing just in case anyone else needs to see it - Safari on Mac gave me all of those login / logout things, and would not show any posts at all, just an error screen. I couldn't even see this note to know what the problem was, even when refreshing. I went to a clean Firefox browser, saw this, cleared Safari history and all is well.

Mac / OSX Sonoma.

[–] [email protected] 4 points 11 months ago (1 children)

I’m impressed by the way some threads now have "-2" new comments. How does that work?

[–] [email protected] 6 points 11 months ago

acausal trade

[–] [email protected] 4 points 11 months ago

I've deployed a fix for the JWT bug that's been breaking all our users, as well as the broken cache behavior. special thanks to the lemm.ee infrastructure admin/professional software developer/lemmy contributor below who bravely sacrificed their posting privileges by pointing out the source of the cache problem in the most jackass way possible and then insisting on a solution that doesn't work for our instance

there's one remaining lemmy-ui bug on my end I haven't been able to track down, and that's the weird half-logged-in session thing where depending on how you interact with the page, your session might disappear. that one stumps me to be honest; I'm going to try and debug with dev tools later, since currently it's only happening in Safari on my phone.

[–] [email protected] 3 points 11 months ago

Just in case anyone else runs into it: looks like this issue also completely wedged my install of Mlem - had to nuke the app from phone and reinstall, its sign-in procedure didn’t cope with this failure mode and it had no way to forget about account without also wanting to remotely delete an account (which I figure probably would’ve failed because no auth token, but I didn’t chance it)

I’ll log a bug their side when I have some headspace to write it up

[–] [email protected] 3 points 11 months ago* (last edited 11 months ago) (1 children)

I humbly ask that you do not update the UI from 0.18.4, because pagination of comments is disabled? broken? with this update.

Alternatively, if you did update to 0.19.2 FE, and I still see the old functionality because of my cache+cookies, then the jank is even more powerful than I imagined.

[–] [email protected] 4 points 11 months ago (1 children)

the jank is extremely powerful, and it sounds like you’re running a mix of 0.18.4’s libraries and 0.19.2’s html and css right now. see elsewhere in this thread for the extremely drunk programming mistake some lemmy dev made that causes this

clearing your browser cache and cookies should get you a fully working lemmy UI, I hope (maybe try lemmy.ml first to see if pagination is working as expected on your end; they should be on the same version as us)

[–] [email protected] 3 points 11 months ago (1 children)

Nope, cleared cache and cookies for lemm.ee and the pagination for comments is still not working. Must be a bug with lemmy, pagination for posts works fine.

[–] [email protected] 3 points 11 months ago (1 children)

hmmm crap, I’ll keep an eye out for a hotfix. in the meantime, Proton might be a good option once it’s deployed if you don’t mind how it looks

[–] [email protected] 3 points 11 months ago

Proton

Thanks for suggesting alternative frontends, proton doesn’t have what I wanted, but alexandrite does, and does it better than lemmy-ui.

[–] [email protected] 3 points 11 months ago* (last edited 11 months ago)

this is wild, i can successfully log in and everything is fine in a chromium incognito window but not in a normal window even after clearing cookies and cache

EDIT: chrome://settings/content/all shows the cookie and I can delete it

firefox works after i just clear the cache

MOBILE SUGGESTION: if you have android, why aren't you using firefox with ublock origin already

lemmy on urbit on windows