this post was submitted on 22 May 2024
22 points (100.0% liked)

Selfhosted

39435 readers
6 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

I have set up a refurbished PC as a media PC with storage. The OS, Debian, is on an m.2 nvme disk of 256 GB. I have connected 2x4TB risks in zfs mirror mode to store my media.

Off late, while booting, I've noticed some messages that suggest that the health of nvme disk is not good. Searching the error, i realised that I should not rely on it. I've done a number of tweaks to set up my system the way I like that I want to save by creating an image of the OS drive on a fresh nvme disk of same size that I have.

How do I go about doing it? I could boot using a live USB and create the image on the HDDs but the live USB OS won't recognise my zfs, right? Is using another external disk or another PC my only option here?

Thanks and cheers!

PS: The machine is a HP Elitedesk 800 G3 that has a wifi port that I've heard can be used as additional port for m.2 drives. Is it true?

top 19 comments
sorted by: hot top controversial new old
[–] [email protected] 5 points 5 months ago* (last edited 5 months ago) (1 children)

It sounds like nobody actually understood what you want.

You have a non-ZFS boot drive, and a big ZFS pool, and you want to save an image of the boot drive to the pool, as a backup for the boot drive.

I guess you don't want to image the drive while booted off it, because that could produce an image that isn't fully self-consistent. So then the problem is getting at the pool from something other than the system you have.

I think what you need to do is find something else you can boot that supports ZFS. I think the Ubuntu live images will do it. If not, you can try something like re-installing the setup you have, but onto a USB drive.

Then you have to boot to that and zfs import your pool. ZFS is pretty smart so it should just auto-detect the pool structure and where it wants to be mounted, and you can mount it. Don't do a ZFS feature upgrade on the pool though, or the other system might not understand it. It's also possible your live kernel might not have a new enough ZFS to understand the features your pool uses, and you might need to find a newer one.

Then once the pool is mounted you should be able to dd your boot drive block device to a file on the pool.

If you can't get this to work, you can try using a non-ZFS-speaking live Linux and dding your image to somewhere on the network big enough to hold it, which you may or may not have, and then booting the system and copying back from there to the pool.

[–] [email protected] 3 points 5 months ago

Phew! I almost believed I was asking for something beyond the scope of linux-fu. English not being my first language may be part of the reason but still I think I covered everything that was relevant.

Yes, that's exactly what I want and your post has given me the clarity I needed. M.2 wifi slots don't support disks so that option is definitely out. I'm going to boot with the latest Ubuntu live OS on a USB and attempt what you've outlined.

I don't have anything really critical on the zfs that is not backed up separately so I'm definitely going to attempt this and learn in the process.

Thank you for taking the time to respond!

[–] [email protected] 5 points 5 months ago (1 children)

If you want an image, it doesn’t matter what the underlying file system is. You should be able to use a tool like Clonezilla and get a 1:1 copy. Depending how you’ve set up partitioning, you could also use sgdisk to set up the proper partitions and zfs send/recv for the new data portion of the drive and install a boot loader. That’s probably the way I’d go in this instance.

[–] [email protected] 2 points 5 months ago

I get the part that the cloning software does not care for the underlying OS. My worry is the fact that I'll run the cloning software/command from a live USB which will not be able to detect the zfs mirror on my backup drive on its own and thus break the zfs mirror with bad consequences for the existing data. I could not find any commands to make the live USB OS discover and respect the existing zfs configuration.

[–] [email protected] 5 points 5 months ago* (last edited 5 months ago) (1 children)

don't mess around with imaging to a file on the zfs, then restoring it. simply clone nvme -> nvme using a usb nvme adapter then replace the internal with the clone.

[–] [email protected] 4 points 5 months ago (2 children)

I would like to avoid buying additional hardware if possible.

[–] [email protected] 5 points 5 months ago (1 children)

usb nvme adapters are not expensive and it likely won't be the only time you need it. they are a handy accessory to have on hand if you have nvme storage.

[–] [email protected] 3 points 5 months ago

I'll definitely take this route if the wifi slot will not support the m.2 drive. Thanks for the suggestion.

[–] [email protected] 1 points 5 months ago

You can old nvme -> SATA -> new nvme with any old SATA drive you have lying around.

[–] [email protected] 4 points 5 months ago (1 children)

Use an Ubuntu live USB, all recent versions of Ubuntu have ZFS drivers baked into the live environment. Then you should add your new SSD to the ZFS pool, and remove the old one from the ZFS pool. Your m.2 WiFi slot should be able to host the 2nd drive while you do this, but if not you can use an external USB housing for it, you'll just have to make sure that the ZFS pool knows its UUID so that it knows it's the same drive.

[–] [email protected] 2 points 5 months ago (1 children)

Why do I need to add my nvme to the zfs pool? That doesn't really make sense.

If the wifi slot does support m.2 drivers (I was just looking for some confirmation/document), it'll solve my problem. Thanks nonetheless.

[–] [email protected] 1 points 5 months ago (1 children)

You need to add the new drive to your existing pool because ZFS stores data across all drives by default, similar to a RAID0. Then you remove the old drive and ZFS will automatically copy the data off the failing drive onto the healthy one and allow you to remove the failing drive with no data loss.

[–] [email protected] 2 points 5 months ago (2 children)

The SSD is 256GB while the two HDDs are 4TB each. What kind of zfs config/array do you suggest I create from them?

[–] [email protected] 2 points 5 months ago* (last edited 5 months ago)

That depends on your use case, I just did a simple zpool with no redundancy because I wanted maximum speed/capacity and all my data is backed up on an external HDD. If you need redundancy, I would look online for how to configure that and what the optimal setup is.

[–] [email protected] 1 points 5 months ago* (last edited 5 months ago)

Actually, I assumed you just had the SSD, if you have more than 256gb of free space between those HDDs, you can go ahead and remove the SSD from your zpool right now (unless your bootloader is there, then you'll have to make an EFI system partition on one of the HDDs and install a bootloader first)

[–] [email protected] 1 points 5 months ago

You could use something like Veeam Endpoint (free), it will image while the OS is running so the underlying ZFS system shouldn't matter.

[–] anzo 1 points 5 months ago (1 children)

I think you want to see "zfs import" command, to get your pool back con the new OS.

[–] [email protected] 1 points 5 months ago

Yes, the other answer also suggests this and I think this will do the trick. Thank you for your response.

[–] [email protected] 1 points 5 months ago* (last edited 5 months ago)

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters More Letters
SATA Serial AT Attachment interface for mass storage
SSD Solid State Drive mass storage
ZFS Solaris/Linux filesystem focusing on data integrity

[Thread #758 for this sub, first seen 22nd May 2024, 23:15] [FAQ] [Full list] [Contact] [Source code]