I've noticed that [email protected] is abandoned and is already a child community of [email protected] which basically serves the same purpose and on average receives only a couple of posts per week.

This got me thinking: we have a process in place to create new communities in [email protected], but is there a process in place to remove abandoned communities?

Over the weekend we had a large intermittent outage, followed up by unplanned maintenance that I had put off for way too long.

Lemmy runs with several different services.

• lemmy-ui (the reactesque frontend)
• lemmy (the rust backend)
• postgres (the data store for operations, comments, posts, etc)
• pictrs (the image data store)

The outage concerns itself with the last one. We always knew we'd eventually need to migrate to an object based store, but Lemmy defaults to file based picture storage and that's what we stuck with up until now. This eventually caused the VPS that programming.dev is running on to seize up, and resulted in the outage over the weekend.

Saturday night I spent several hours testing out the object migration on the beta.programming.dev site in order to validate that it worked. During this time I struggled with some very obtuse ansible errors that I hadn't encountered before and so I was not able to start the migration that night. I delayed until the next morning (thank goodness).

I began work Sunday morning at 10:00 America/Denver time. Initially the migration started off quite well, but was moving incredibly slowly. Looking back on it now, the migration would have taken over 144 hours if I left it to do its thing. I let this run for about an hour before messaging the pictrs dev to understand why logs weren't showing up for the migration (even though objects were showing up in the store). Apparently lemmy-ansible is set to use 0.4.0 of pictrs, which not only is quite old, but doesn't have the ability to run migrations concurrently. There was the issue. I asked the dev is it was possible to stop a migration in the middle of the running, upgrade, and continue. They told me what changes I'd need to make, I made them, did the upgrade, and restarted the migration. It immediately failed. This was the start of my issues.

The server was now too full of data to do anything, including running apt update or apt install to install tools to assist me. I was able to attach more block storage, but I'm not enough of a linux guru to figure out how to mount it where the current pictrs filesystem would be able to take advantage of it. I had to result to copying the entire pictrs filesystem to a fresh ~500gb mount, fixing permissions, and then rerunning the migration from there. By the time I got to this point, it was about 12:30PM. The migration from then on took several hours.

After the migration completed, I needed to deploy the new stack with the correct settings. The ansible script needed to run apt though, and, well, that wouldn't work when the server was still full. At this point I was not confident in the migration and I also hadn't realized that you could do the migration while the site was running (big oversight from me). I therefore wanted to maintain the entire pictrs file store until I proved the object store was working. I created another block storage, copied the entire pictrs directory over to it again (another 20 minutes or so) and then deleted the original directory. I was now able to run the ansible script and deploy the new settings for pictrs, confident that I had a backup available in case something went wrong (this is not the main backup method, the server is backed up externally as well, but I didn't want to have to resort to those during the migration).

That completed the migration, some 5 hours after it originally started.

There were several things that exacerbated the issue that made it take several hours longer than I wanted.

1. I let it go so long before doing the migration to object storage that the server was too full to even perform an apt update. This resulted in me not being able to install tools I needed, along with a host of other issues as mentioned
2. pict-rs was at a very suboptimal version. If it had just been two minor versions newer it would have migrated perfectly fine, in a few hours.
3. my limited knowledge around ansible led me on wild goose chases several times

Things I would change if I had to do it again:

1. Dig in a bit deeper on the concurrency flag in the pictrs docs. It was not present in the original guide I followed (from a lemmy post on another instance), and thus I didn't realize that it wouldn't run with concurrency at all.
2. Don't wait so long so that the server is full
3. Migrate while the server is running. That would have been dumb in this case, since the server wouldn't stay up anyway, and could have caused other issues. But there was no reason to take the server down if it had been stable, and other instances have done so with no problems.

Hey everyone, update on https://programming.dev/post/4613085

We did some changes in the database and it seems like outbound federation is working again. If you run into any more problems with it let us know.

Im pinning this post for around a day to let everyone know since we had the last post pinned for a bit

Any posts or comments made while it was broken still wont be federated but new ones should be. The instance essentially marked all other instances as dead which meant it thought it didn't have to send things to them

Hey everyone. Currently the instance is being affected by a bug that prevents outbound federation. This seems to have started when the instance upgraded to 0.18.5

Im pinning this post in the instance while it is affected by the bug so everyone is aware as this affects everyone in the instance

Theres an open issue on the lemmy github repository and we are attempting to track down why this is happening and fix it

https://github.com/LemmyNet/lemmy/issues/4039

What the bug means is essentially

• We still get all posts and comments around the fediverse like normal (so youll see posts made in other communities populate the feeds like normal)
• Posts and comments made by users in programming.dev are not visible outside programming.dev

Hopefully itll be resolved soon. If anyones able to get some info that would be helpful towards resolving the bug throw it here or in the github issue

For example, I can't upvote or comment on a single thing in this thread https://programming.dev/post/4577853?scrollToComments=true which is from [email protected]

Same goes for !linux https://programming.dev/post/4449644?scrollToComments=true

I just made this post https://programming.dev/post/4307457

but it isn't showing up here https://sh.itjust.works/c/speedrun

I had a similar issue on a previous post but I didn't think much of a single failure and fixed that one myself by doing the search

https://programming.dev/post/4295735

but now I see it happening again and with a different instance I think there's something wrong

Getting Fingerprint Detector reports about high number of fingerprinting API calls on Programming.dev instance.

Is there a reason for that? Not getting it on others instances. Report

Last year I was working on this website that could be described as a cross between OkCupid and Stackoverflow Jobs, called CareerCupid.

The idea is to provide a quiz with sorts of questions related to people's preferences and values around their ideal job environment, tech stack, company structure, etc. By comparing what people's answers with what they would expect from the other people to answer, we could build an affinity score between team members, or job seekers and positions and even between the person and company itself.

To help with the discussion around the quiz, I wanted to let people share their answers on social networks like Twitter/Reddit/Hacker News/LinkedIn. Right about the time I was finishing the "share answer" functionality, I started a new job and didn't have the time to keep this side project.

Anyway, an year and a half has passed and I'm now looking to resume working on this. I however do not want to continue contributing to reddit, so I'm looking for an alternative place in the fediverse that could be used for this type of conversation.

I could create the equivalent "CareerCupid" community on my own instance, but given that programming.dev is somewhat established as the instance for developers, I'm wondering if the admins here would be interested in creating a community for the types of questions/conversations that would go around this.

I can volunteer to be a mod if needed.

Hey yall hope your weekend has been good if not relaxing at least~

After some internal discussion we've decided to temporarily remove Kbin.social/m/{gaming,opensource,science} as just like Lemmy.world we in Programming.dev are encountering several spam posts from Kbin users that remain on the site even after they've already been purged on Kbin.social.

Just like Lemmy.world we hope to see Kbin.social addressing this issue/bug(federation of purged users and posts) soon!

Thank you for reading this instance update :D

TL;DR: Temp removal of Kbin.social/m/{gaming,opensource,science}.

Here are some alternative communities around the fediverse for the same kinds of content:

Lemmy.zip - Gaming
Mander.xyz - Science
Programming.dev - Opensource

Hey everyone, just wanted to do a post looking for a couple more admins for the instance

Currently most tasks have been getting done by snowe or I with snowe handling most things in the backend and me for things people see such as community management

So that tasks are spread out a bit more and so that I can focus more on things such as building up our new frontend just wanted a couple more people to help out with various tasks

Mostly looking for people to help with community management in the instance. This includes

• Handling the creation of communities from [email protected]
• Handling community adoption from [email protected]
• Making sure communities in the instance are running smoothly
• Handling meta communities such as [email protected] [email protected]

If you dont have much history in the instance send me a link where I can see past chatting (e.g. mastodon, reddit, etc.). We also are using discord as a platform for the admins to talk to each other so you have to be able to use that

If you want to apply feel free to dm me or say so below.

Hey everyone, quick update on the instance

Ive synchronized our instance to fediseer which means we will start pulling blocklists from certain instances and then adding that to our own.

I have it set up where it will pull blocks for certain tags such as csam, pedophilia, poor moderation, etc. from instances like lemmy.world, lemmy.dbzer0.com, etc. so that that kind of content is able to be handled more quickly if one of those admins gets to it before us.

Should mean that you have to worry about content that breaks the rules even less and be able to enjoy the site more. If a site im pulling stuff from starts blocking instances for the wrong reasons ill remove it and revert the blocks but shouldnt happen for the ones ive added (and soon there will be a feature where I can set stuff like only block an instance if 50% of chosen instances have blocked it)

Continuation of my last federation update post with some updated info

Hexbear

After we got removed from the hexbear allowlist I talked with one of their admins about it. Theyre running a poll currently for their users to determine if they want to refederate or not with opinion in that poll seeming to be mostly in favor

It is very likely we will be federating with Hexbear again. However, things such as political communities will be hidden from our feeds by default meaning you wont see those posts unless you explicitly subscribe to those communities (so if youre just browsing the all feed you wont see any of them)

In terms of comments made in other instances that aren't hexbear or programming.dev (e.g. lemm.ee) that is up to the admins of those communities to handle (and if an instance has defederated from hexbear such as lemmy.world threads in their instance will have no hexbear members on them)

If you participate in the hexbear instance make sure to follow their code of conduct and dont participate in communities that are labelled as being for hexbear members only (I should be hiding any that exist)

If theres a community I missed that should be hidden reach out to me with it and ill hide it. Hidden communities will show up with no posts when you look at them without subscribing

And to leave off this update ill give you guys a couple hexbear communities that might be relevant to you once federation is up again

https://hexbear.net/c/gamedev https://hexbear.net/c/technology https://hexbear.net/c/games

Hey everyone, ive got pangora-ui (our new frontend being built ([email protected]) to a decent enought state where I can start doing mod actions from it

Due to that ive managed to implement a community hiding system to make certain communities' posts not show up by default so that we can now fully follow our rules in the sidebar

Communities that are hidden

• Will show up as having no posts
• Will not have posts show up in the feeds or from searching

However if you subscribe to a hidden community you will then be able to see all of their posts like normal in every location

Hidden communities will still be findable themselves in the community list and from searching

If you are currently subscribed to hidden communities you should see absolutely no difference in your ability to view them

You should however see the all feed getting more manageable to navigate and sort by new now that it isnt being drowned out by bot posts

Communities that are currently being hidden include: communities that are 50% bot posts or more and politics communities

For bots this includes communities in instances like

• lemmit.online
• derp.foo
• (the bot art community in) lemmy.dbzer0.com
• zerobytes.monster
• radiation.party
• lemmy.smeargle.fans

Quick comparison between our all feed sorted by new vs lemm.ee's so you can see what hiding bot communities did

You may still see a couple pop up since I havent dealt with all of them yet but the amount of bot communities showing up will get lower and lower over time

Hey everyone! Wanted to give a quick update on some federation status with other communities

burggit.moe

burggit.moe has been defederated from for two main reasons

• They ignore DMCA requests and dont follow it. We get a copy of all of their posts due to how federation works and since this rule is instance wide it can easily apply to a new community in the future if we purge current ones. We dont have the manpower to handle other instances continually breaking the law and then us needing to deal with that with takedowns

• They post pornographic content that is illegal / in a gray area in the US that I would rather err on the side of caution in terms of allowing due to things like the PROTECT Act and certain states explicitly banning it themselves

hexbear

Hexbear defederated from us so to prevent one way conversations I have added them to our blocklist as well. If the hexbear admins decide to unblock us I can do the same and go back to the previous stance of hiding their politics related communities from the all feed once that feature is implemented into the site

Misc

There are a couple misc pleroma and mastoton instances that ive added to our blocklist (and there are a couple more I might add soon) that post content that is extremely against our rules. As they arent lemmy instances purging communities isnt an option and its allowed instance wide

if you have other requests of instances that clearly break our rules (you can find them in the site sidebar) feel free to send me them and ill take a look

Do we have an established practice for requesting defederation?

I would very much like to request that we defederate from burggit.moe (because I would prefer to keep my lemmy experience free of loli porn) and I went looking for the best way to do this, but the best I could come up with was posting to meta and that seems like the wrong place.

I feel like I missed something obvious ¯_(ツ)_/¯

Hey there,

I was using https://mlmym.org/programming.dev/ to browse programming.dev because I don't really like the default Lemmy UI. However, as of today https://mlmym.org just redirects to this gist: https://gist.github.com/rystaf/4d591ffdcbaab1c49efa406885efd814.

When checking both https://old.lemmy.world and https://lemmy.world - they both resolve to the same IPs - So it seems like the intended use for this UI is not use it though https://mlmym.org anymore, but for the instances to host it themselves under the ".old." subdomain. In a similar way reddit is doing.

As for how it would look, have a look at https://old.lemmy.world - and probably enable dark mode in the settings.

Was hoping programming.dev would consider supporting this UI as well, under old.programming.dev - It makes the transition from Reddit to Lemmy a lot easier

You can find the repo of it over here: https://github.com/rystaf/mlmym

https://programming.dev/post/2010603

I don't see myself as a moderator when I visit the community from programming.dev, but when I visit it from a different instance, say lemmy.film, it's showing my name as a moderator.

https://programming.dev/c/[email protected]

https://lemmy.film/c/[email protected]

Please help.

Whenever I try to post an image this morning, the upload fails no matter what browser/client/frontend I use with:

Request error: error sending request for url (http://pictrs:8080/image): operation timed out

Also this shows a popup with SyntaxError: Unexpected token 'R', "Request er"... is not valid JSON but I assume that's a bug with the lemmy frontend since voyager shows the correct error.

Question: I sometimes see posts in PD communities that are not in English (most recently in Flutter community). Is there a reason for allowing languages other than English? It definitely doesn't make moderation easier. Or maybe is there a technical reason for that (like Mastodon posts doesn't necessarily specify language and will fall under undetermined)?

We now have some possibility for custom emotes available in the instance (now that the bug that was happening with them was fixed). If anyone has some suggestions for what to add feel free to reply with them

Ive added the p.d logo and a couple game engines for now as an example. You can send the p.d logo by typing :pro (or more of the name) and then it should show an autofill you can select if youre using the website (theres an emoji picker as well but it doesnt work for custom emotes)

Ill likely be adding some for the logos of various things but mainly want to know if theres anything people want other than that

Hey everyone! I'll be going through and making features for the site soon (as I've almost got my laptop back) and wanted to know if there was any features that are wanted that I dont already have noted down

I'm currently planning to do:

• Customizable icon shapes (circle, hexagon, etc.)
• Flairs
• Ability to block an instance
• Ability for admins or community mods to hide communities from the all feed
• Community short descriptions (to show in the community list + top of sidebar)

And polls are also being worked on by snowe

Will try to get the changes merged into lemmy itself as well but they'll be released here first

Some communities there are literally calling for the annihilation of Ukraine

https://programming.dev/comment/1999097

edit: Since people are either willfully or otherwose are missing my point, this is a pro-russia, pro-fascist hate group, therefore we shouldn't give them the platform to spread their shit. this is not about me seeing their community or then causing issues accross other communities, it's about de-platforming fascists.

I've been experimenting with putting random quotes from the [email protected] community as taglines in the instance.

It will automatically put a random quote every time you go to a page in the site. I put the ones that are shorter so they don't flood the page, and that were upvoted.

Theres also a link back to the quotes community if you click on the credit where you can discuss about that quote. As more quotes get added to the community the pool of tagine quotes will grow

• note they may not be visible on certain lemmy clients depending on what features the clients support but they work on web

Hey everyone! Just wanted to create a post on different things going on in the instance

Matrix Space

Programming.dev now has a matrix space at https://matrix.to/#/#p.d:matrix.org . We had a matrix room before but a space is essentially a collection of rooms so we have sections for different communities, a support room, etc. If you manage a community and want a matrix room for it listed feel free to reach out to me

Instance Rules

I've collected together our instance rules and extended it to be a bit more clear. Here are the rules (and they'll be going on the sidebar as well)

• Dangerous content on the instance and federated instances including phishing, spamming, brigading, unmarked nsfw, doxing, etc. is not allowed and subject to be handled by the mods or admins
• Hate speech is not allowed on the instance or in instances federated with it. This includes remarks directed at sex, gender, orientation, disabilities, etc.
• Other content is up to the discretion of the mods of the community its posted in
• If a community has no dedicated mods it will be managed by instance admins until a mod team is found
• If a community mod team has gone inactive and no longer maintains a community new mods may be added
• Any federated communities that have shown they will not enforce these rules while their content shows up on our feeds will have their community removed from our instance
• Instances that continually encourage this type of behavior or are built to only do this type of behavior are subject to be defederated
• Certain content may be hidden from our all feed including politics and bot spam. (Not currently implemented due to lemmy limitations but will be eventually). This means it won't show up in all but you can still view it and subscribe to the communities to have it show in your subscribed feed
• Bots interacting in our communities must follow our Programming.dev bot guidelines (see below)
• If an instance is not being managed (due to owners being gone, etc.) it is subject to be defederated

Things that do not follow these rules will be handled. This currently includes

• Communities in exploding-heads will be removed from our instance
• Rammy.site will be defederated from due to the site owner being MIA

Bot Guidelines

1. Mark Bots as Bots Bot accounts should be marked that they are a bot using the checkbox in the user settings. This includes any automation in an account (if you automate part of your personal account, put what the automation is in your bio so people know)
2. Put contact info The owner of the bot and some way to contact them must be in the bot's bio (unless its an automated personal account, in that case the account itself it the way to contact)
3. Use mentions as prefix If a bot has commands it should use its mention as a prefix. (This will allow different bots to use the same commands without interfering with each other)
4. Dont spam Bots should not spam posts or comments. (If a bot command has been used its fine to do a response but dont drown out non bot posts in the local new sort.) Certain cases are unavoidable such as if you want to get a bot posting weekly threads in different communities at the same time but in those cases please contact me first to get it approved
5. Allow mods to opt in Bots participating in a community should be opt in so communities can choose what bots they want. If you want a bot to be allowed in a community please contact the moderators of that community first. EXCEPTION: If a bot is deemed to be a well behaving bot that brings net good to the instance it can override this rule as long as it has been approved by an admin. This can include things such as a tldr bot, remindme bot, link fixer bot, piped link converter bot, etc. If a bot has overriden rule 5 with this exception community moderators should still be able to opt out of the bot

These guidelines should bring us up to parity with lemmy.world along with allowing a bit more leeway within the instance since there has been some nice bots getting made that I dont want unfairly punished

If you have a bot you want whitelisted for the entire instance for the rule 5 exception feel free to dm me here or on matrix

And that's everything for this month. Progress on some features for the site is still ongoing. My laptop is currently being repaired but once I get it back I'll do a bunch of progress and share some updates here

If you have any suggestions for the rules, bot guidelines, new site features, etc. feel free to throw them down below