this post was submitted on 28 Aug 2023
56 points (95.2% liked)

Arch Linux

7777 readers
2 users here now

The beloved lightweight distro

founded 4 years ago
MODERATORS
 

Hello. Please critique how I'm updating / maintaining my new Arch installation so I can fix anything I'm doing wrong. This is mostly what I could gather from the Arch wiki tailored to my system. I think I know what I'm doing - but as I've often learned, it's easy to misunderstand or overlook some things.

Step 1: perform an incremental full system backup so I have something to restore if the update borks anything. I've chosen to use the rsync command as laid out on the wiki:

sudo rsync -aAXHv --delete --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} / /media/linuxhdd/archrsyncbackup

I have a large hdd mounted as a secondary drive under /media/linuxhdd. It is configured to automatically mount from fstab using uuid. Both my root drive and that hdd are formatted ext4. I'm not using the -S option because I don't think I'll be using virtual machines (I have other hard drives I can make bootable). --delete is used so I maintain one current set of files for restore purposes. This keeps the copying and transfer time to a minimum. (I maintain disk images offline with a different tool - this is simply one local copy for easy restoration purposes)

Step 2: Check the Arch wiki - follow instructions for any manual steps

Step 3: once every 1-2 months, update the mirror list using reflector

sudo reflector --protocol https --verbose --latest 25 --sort rate --save /etc/pacman.d/mirrorlist

This should sort the fastest 25 mirrors into mirrorlist. Remember to use the -Syyu option in step 6 if this step was done

Step 4: Clean the journal

sudo journalctl --vacuum-time=4weeks

This should keep 4 weeks of files.

Step 5: Clean the cache

sudo paccache -r

This should keep no more than 3 versions laying around. Once and a while, I can clean out all uninstalled packages with -ruk0 options instead.

Step 6: Upgrade Arch packages with pacman

sudo pacman -Syu

I need to watch for pacnew and pacsave files and deal with them (although I haven't seen any yet)

Step 7: Review the pacman log

nano /var/log/pacman.log

This should tell me about any warnings, errors, instructions, or other things I need to deal with.

Step 8: Remove Orphans

pacman -Qtdq | sudo pacman -Rns -

This could be recursive and needs to be run more than once. Instead, I'll just run it once every time I update. This should keep things cleaned out.

Step 9: Update AUR packages

Check the build scripts to make sure the package hasn't been taken over and that it won't run anything funny.

yay -Sua

This should update just the AUR packages

Step 10: Remove AUR orphans

yay -Yc

The wiki says this "removes unnecessary dependencies" which I believe means AUR-only orphan packages.

Step 11: Reboot

reboot

Step 12: Update flatpaks from the GUI (Gnome-->Software-->Updates)

Any mistakes? Suggestions?

Thanks!

top 31 comments
sorted by: hot top controversial new old
[–] [email protected] 54 points 1 year ago (3 children)

Well you're being far more in-depth than me. I just run pacman -Syu periodically.

[–] [email protected] 18 points 1 year ago* (last edited 1 year ago)

Yes, and very rarely, I have to pacman -S archlinux-keyring after it complains about keys and run it again.

[–] [email protected] 8 points 1 year ago (1 children)
[–] [email protected] 14 points 1 year ago (1 children)

Isn't that the same as yay?

[–] [email protected] 7 points 1 year ago
[–] [email protected] 5 points 1 year ago

Same here. And I never had any issues w/ it.

[–] [email protected] 16 points 1 year ago* (last edited 1 year ago) (1 children)

you can automate some parts of that away or can be combined
most AUR helper will basically invoke pacman -Syu internally before actually upgrading any AUR package (as otherwise this can lead to issues)
yay in particular can just be called via yay (no arguments) which is an alias of yay -Syu

Point 3: reflector.timer comes with reflector now and runs weekly by default, you need to configure and enable it though
https://wiki.archlinux.org/title/Reflector#systemd_timer

Point 4: you can restrict the journal size to much smaller than it is by default (10% of the partition size OR maximum 4 GB): https://wiki.archlinux.org/title/Systemd/Journal#Journal_size_limit
or alternativly set MaxRetentionSec to 2419200 seconds (4 weeks)

Point 5: can be done via pacman hook automatically after every upgrade
example: https://aur.archlinux.org/packages/pacman-cleanup-hook

there are also many other useful pacman hooks which tell you if there are any orphans or if AUR packages need to be rebuild or there is a hook which reload kernel modules after a kernel upgrade (which otherwise would warrant a reboot unless you are ready for the issues of many unloaded kernel modules)

[–] [email protected] 5 points 1 year ago

Awesome - thank you! I knew about yay, but wanted to understand it in case yay ever disappears like yaourt did. I did not know about the others!

[–] [email protected] 11 points 1 year ago

I don't even have backups except git repos for my code and config. :)

[–] Unquote0270 10 points 1 year ago (1 children)

I just run 'paru'.

Looks interesting and comprehensive though, a few things I should probably implement.

[–] [email protected] 1 points 1 year ago

paru is the way, combined with btrfs, "nothing" can go wrong.

[–] [email protected] 9 points 1 year ago

I just backup my stuff - dot files, docs, etc regularly. My whole system is disposable - meaning I can reformat anytime (I do it regularly).

[–] [email protected] 8 points 1 year ago* (last edited 1 year ago)

You are way more involved than I am. I usually just do pacman -Syu and yay -Syua every couple of days and then pacman -Sc about once a month to clean out the package cache.

[–] [email protected] 7 points 1 year ago (1 children)

Just so that you have an additional data point, here's how I do it.

I run a backup first, using borg-backup. I used rsync in the past, then rsnapshot and now borg since it allows for compressed incremental backups, diffing on the "chunk" level, meaning I won't backup the entirety of a modified file again and safe a lot of space.

I used yay before, but like you I didn't want to go into it blindly and do some modicum of sanity-checking the PKGBUILD for changes beforehand. Since it wasn't obvious on what would be the best way of using yay for doing this, I asked around on the ArchLinux Forum, and ultimately decided to try one of the simpler tools suggested in the Arch Wiki, aurutils.

After setting it up (the author helped me migrate), I now use it as follows:

  • aur repo --upgrades: Searches for new versions of aur packages and displays them
  • aur sync --upgrades --no-build: Performs a git-pull under ~/.cache/aurutils/sync and opens vifm so that I can look at a diff of the PKGBUILD and all the other changes in the affected directory.
  • aur sync --upgrades --no-view: Builds the package. It is now available as part of the custom (local) repository used only for aur packages, but hasn't been upgraded yet. That is, a package.tar.gz or whatever has been created and put into ~/.cache/aurutils/sync/, where the PKGBUILD resides as well
  • sudo pacman -Syu: Upgrades all packages from all repositories, including the ones from the custom repository

I won't argue pro or against one aur helper or the other, but I feel like I have a little more insight about what happens under the hood since I made the switch. That being said, in the very beginning, I managed aur packages manually. This works also, but at some point became too tedious for my taste. I am happy with the semi-automatic approach I am using now.

[–] [email protected] 2 points 1 year ago

Great suggestion-I really appreciate this! I’ll look into it.

[–] [email protected] 5 points 1 year ago* (last edited 1 year ago) (1 children)

I have one suggestion and that is to save yourself some time.

Auto-Backups, snapper + snap-pac, a hook that runs reflector (or alternatives) on its own, another one for flatpaks. I'm also not sure whether you actually need to run yay independent of pacman. I never did with yay or paru.

You could eliminate like half of that process I think, but please correct me if I'm missing something.

[–] [email protected] 2 points 1 year ago

I agree. I think step 1 was understanding what should ideally be done. Now I can make my life easier through automation (which seems to be the common feedback - “hey why don’t you automate some of this stuff, try these out”). Which is exactly what I intend to do! ;)

[–] [email protected] 5 points 1 year ago
[–] [email protected] 4 points 1 year ago* (last edited 1 year ago)

TL;DR
I'd advise you to automate backups as incremental daily backups, automate reflector, clearing paccache and rebuilding initramfs/grub config.

  1. My system automatically backups at 4 am or boot time anyway, with basically the same command as you (also excluding games, caches 'n stuff).
  2. yay -Pw should show you unread news.
  3. reflector can be run as a systemd service
  4. For me, the journal never grew to a large size, despite not being cleared for any of my systems. Also useful for checking logs on systems where some errors may stay undetected for weeks, like servers (which I do run, successfully, without problems at all, on Arch testing repos. Even arch for arm works without any problems at all).
  5. There's a pacman hook in the AUR that automatically clears the cache to the last three versions, or what you want. My yay cache is ~60 GB tho, which is more than the rest of my system combined. Still not an issue, as it is excluded for backups and I can just clear it or change the limit.
  6. As yay basically just invokes pacman -Syu, that step is actually pretty redundant.
  7. Errors should be outputted at runtime of pacman/yay, so as long as you don't have 200 pkgs to update you could monitor normal and AUR packages at once.
  8. If packages are removed from dependencies they should be uninstalled automatically I believe and using pacman -Rs should remove packages that were only installed for the package you are removing. (Btw, -n removes config files too, if you don't want them anymore). Despite that, I do check for orphans etc. when I feel like I want to clean my system(s).
  9. Just yay upgrades AUR and pacman pkgs, as said in 6.
  10. Just as 8, orphans shouldn't happen anyway. Though I did not know about the AUR specific command, thank you!
  11. As I'm, not even joking, updating and "newest software" addicted, rebooting every time is very annoying, especially for servers. Luckily, as I update extremely often, it's mostly very few packages so I can quickly decide if to reboot or not. In general, packages like screen or python can be ignored. Packages like sshd can be reloaded by just restarting the service (systemctl restart sshd). Only kernel, CPU-ucodes and drivers are a real update trigger for me.
  12. I don't have flatpaks, and eg. pip is redundant as my python packages are managed by pacman/yay. So it's just yay to worry about.

Also, you missed a little but important step: Recompiling initramfs and the grub config (If you use grub, that is). If so, there's also a hook for that.

Of course, just automating everything screams for errors and system crashes - if not checked. I'd also advise you to check systemctl status regularly, as it can inform you of failed (critical) services, like backups, DynDNS or just shadow.service informing you there's a user missing/extra in /etc/shadow (which is the password file, btw).

In conclusion, many things could be done as service automatically, are redundant or could be done less often. Though I must say, it is better to do everything explicitly and with care, like you do, than to skip over things.

[–] [email protected] 3 points 1 year ago (1 children)

Maybe it is too late because of your filesystem choice but btrfs snapshots delievers enough security if something goes wrong. Rsync seems like a little bit overhead for updating only There is even a pacman hook that makes pre and post snapshots of your filesystem with snapper. Tldr: most of your steps can be automated with pacman hooks. But if you like it this way its fine

[–] [email protected] 3 points 1 year ago (1 children)

Yup you are 100% correct. I didn’t understand what btrfs brought to the table until i started playing around with backups. What I really need is a snapshot since my backups really are handled differently. I may reformat simply for this reason. (Or I’ll switch things up the next time I break something)

[–] [email protected] 4 points 1 year ago* (last edited 1 year ago)

i successfully switched an arch install from ext4 to btrfs.

in my case, i had 2 ssd, so i moved from one to the other, but it should work if you use one of the rsync backups.

what i did was:

  • boot from a live usb arch
  • create a btrfs partition on the new drive with all the subvolumes you need
  • mount the btrfs partition and all the subvolumes
  • rsync my system from the old drive to the new mount (https://wiki.archlinux.org/title/migrate_installation_to_new_hardware#File_copying)
  • than continue to follow the arch wiki guide, fstab will be wrong, the bootloader needs to be reinstalled. while you are chrooted into it, you will need to reinstall linux to rebuild initramfs.
[–] [email protected] 3 points 1 year ago (1 children)

You can update flatpaks by running flatpak update.

You can automate updating the mirrorlist by enabling reflector.service or reflector.timer.

How often do you do all this?
How long does it take?

[–] [email protected] 2 points 1 year ago

yup the automating the mirror thing is new to me - gonna try that! Thanks!

I do this like once every week or two. On a sunday when I'm drinking my first morning coffee and have time. Doesn't take long. Maybe 10 minutes. Most time consuming bits are reading the wiki, checking the log file for errors, and if something in the AUR needs updating - checking the scripts (but let's be honest - I don't do that every single time. What I do is sometimes different than what I should do).

[–] [email protected] 2 points 1 year ago

So yeah. If I needed a 12 step process to update my system it would still run kernel 4 :D (I'm lying of course) but i really don't do as much as you and I don't do more than necessary manually. Basically I run 2 commands to update:

  1. update which runs the update process more below.
  2. clean which removes orphans and cleans the cache

So update is the alias where the magic happens and it does the following:

  1. It runs sudo pacman -Syu checking with the Informant hook if there are any unread news, if so it stops the update. Otherwise it just runs through it.
  2. Then it runs pacdiff to check if any config files changed (I ran into an issue with that before where I didn't check properly and suddenly couldn't login anymore.
  3. Then, if flatpak is installed it will also update all flatpaks.

So my only manual intervention is "accepting" the news if there are any. Accepting the updates. And (of necessary) merging the pacnews. This way I have kept a fresh and stable system for a couple of years now.

[–] [email protected] 2 points 1 year ago (1 children)

I have an alias in bashrc called upgrade. It runs reflector looking for the fastest of the newest 10 near me. Then it upgrades the keyring, then yay -syu and then paccache -r.

I have the journal limited to 100mb so I don't ever bother cleaning house on that.

I probably don't review pacman logs as often as I should, but stuff rarely breaks and is normally pretty easy for me to figure out what when it does.

[–] [email protected] 0 points 1 year ago (1 children)

So I have a note written down that says "keyring" It's unclear to me if that's something I should do regularly, or just if I have a problem. If I'm not mistaken, it is possible for a package to need the updated keyring even if you do pacman -Syu regularly. But I also think it just pitches an error on that package, which will fail to update. Then you can just update the keyring, then rerun pacman -Syu and everything will be fine?

[–] [email protected] 2 points 1 year ago (1 children)

The archlinux-keyring package holds they gpg signatures used for signing the packages and if you go too long between upgrades it's possible you won't have a signature or an outdated one and a normal -syu will fail because of it. So I just upgrade keyring first and it gets ahead of that issue.

[–] [email protected] 0 points 1 year ago (1 children)

@Shortbus @Kongar

On hardware that I don't update often, the problem happens to me and this is the route I normally use. What I'm wondering is why the Arch guys don't implement this "automatically"... for example if the keyring is in the packages to be updated, we update it first and then the rest of the packages.

[–] [email protected] 1 points 1 year ago

Probably boils down to the arch ethos of K.I.S.S. not making too many decisions for you.

[–] [email protected] 1 points 1 year ago* (last edited 1 year ago)

Some suggestions:

Step 1:

rsync has a -x flag that stops it from crossing filesystem boundaries. I find it quite useful so that I don't have to manually exclude all these directories with mounted pseudo filesystems and whatnot. Alternatively, you can also do a second mount of the filesystem you want to backup in a more convenient location than /, and run your rsync from there.

If you want to be fancy, you can also use rsync's --link-dest option to maintain different versions of your backup, so that you can go a bit further back in time. Using this option rsync creates hardlinks for files that are unchanged between the current backup and the previous one, and you can have multiple backups without much storage overhead and while still keeping the backup incremental and speedy.

Step 3:

Reflector comes with a weekly timer preinstalled. You can enable it with systemctl enable reflector.timer and then just edit the config file to your preferred settings.

Alternatively, it also has a service that you can enable so that it runs at boot time, but that doesn't work for one of my systems that is using wifi. For some reason it runs before the network is truly available and returns an error. On my ethernet connected system, I don't have that issue.

Step 4:

Instead of vacuuming your journal manually, you can also set limits in /etc/systemd/journald.conf. I just set SystemMaxUse to something sensible like 512M, so I don't ever have to worry about my journal overflowing my disk.

Step 5:

I just automate the paccache stuff with a simple cron.weekly entry. You may also want to look into where yay caches its packages.