this post was submitted on 28 Mar 2024
41 points (97.7% liked)

Linux

48234 readers
533 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
[email protected]
41
Right way to check free/used storage of btrfs filesystem [SOLVED] (self.linux)
submitted 7 months ago* (last edited 7 months ago) by unhinge to c/[email protected]
11 comments fedilink hide all child comments
 

While checking for used and free space in a btrfs subvolume, I'm not getting a consistent value. It's confusing and doesn't help.

  1. What is the correct way to find used/free space?
  2. Why are these values inconsistent (except normal du)?

According to btrfs fi usage /home, 83.21 GiB is used.

Overall:
    Device size:		 149.98GiB
    Device allocated:		 100.07GiB
    Device unallocated:		  49.91GiB
    Device missing:		     0.00B
    Device slack:		     0.00B
    Used:			  83.21GiB
    Free (estimated):		  63.06GiB	(min: 38.10GiB)
    Free (statfs, df):		  63.06GiB
    Data ratio:			      1.00
    Metadata ratio:		      2.00
...

As per btrfs fi df /home, used space is 82.86 GiB, not 83.21 GiB.

Data, single: total=96.01GiB, used=82.86GiB
System, DUP: total=32.00MiB, used=16.00KiB
Metadata, DUP: total=2.00GiB, used=178.61MiB
GlobalReserve, single: total=99.50MiB, used=0.00B

As per btrfs fi du -s /home , used space is 63.11 GiB.

     Total   Exclusive  Set shared  Filename
  63.11GiB    13.64GiB    49.01GiB  /home

While according to du -hs /home, 64GiB is used.

Also, maximum space used should be close to 72 GiB as per btrfs fi du -s / and 73 GiB as per du -hs /, if btrfs fi usage includes all subvolumes . '/home' and '/' are on separate subvolumes.

top 11 comments
sorted by: hot top controversial new old
[–] [email protected] 8 points 7 months ago (1 children)

As per btrfs fi df /home, used space is 82.86 GiB, not 83.21 GiB.

That's just used data. The global used metric likely incorporates metadata etc. too. System aswell as the GlobalReserve are probably accounted as fully used as they're, well, reserved.

As per btrfs fi du -s /home , used space is 63.11 GiB.

     Total   Exclusive  Set shared  Filename
  63.11GiB    13.64GiB    49.01GiB  /home

While according to du -hs /home, 64GiB is used.

Likely compression or inline extents. btrfs only reports apparent size to du and friends unfortunately.

Also, maximum space used should be close to 72 GiB as per btrfs fi du -s / and 73 GiB as per du -hs /, if btrfs fi usage includes all subvolumes . '/home' and '/' are on separate subvolumes.

Your home has a lot of shared extents which indicates to me that you have at least one snapshot of it.

You also wrote 13.6GiB of new data to your home since the snapshot. Assuming a similar amount of data was deleted/overwritten since, that would add up to 76GiB. If there's perhaps one or two more snapshots, that would explain the rest.

Snapshots are "free" only so long as you don't write or delete any data in the origin.

[–] unhinge 1 points 7 months ago* (last edited 7 months ago) (1 children)

Thanks that helped. I have one snapshot of home. ~~Size of diff between btrfs subvolume and~~ Additional space used by snapshot is 11GiB (probably) and btrfs fi du -s / is 72GiB, making 83GiB (closer to btrfs fi df /).

[–] [email protected] 2 points 7 months ago (1 children)

Size of diff between btrfs subvolume and snapshot is 11GiB

WDYM by "diff"?

Also forgot to mention but if you want to know what's taking how much space on your btrfs, try btdu. It uses a sampling-based approach and will therefore never be 100% accurate but it should be quite accurate enough after a little bit.

[–] unhinge 1 points 7 months ago (1 children)

just comparing the subvolume with its snapshot with btrfs send -p /example/subvol /snapshot/of/example/subvol >examplediff.btrfs.

From man btrfs-send

generate a stream of instructions that describe changes between two subvolume snapshots

Thanks for btdu, it seems useful as 'btrfs fi du' probably doesn't account for compression, will check it out sometime.

[–] [email protected] 2 points 7 months ago (1 children)

Note that the diff does not necessarily correlate with the amount of data that changed, not how much additional space the snapshot takes.

[–] unhinge 1 points 7 months ago

You are right. I shouldn't have used diff. I'll fix that

Also, incremental changes from subvolume to its snapshot might be incorrect as that will be new data added to subvolume, rather that old data deleted from subvolume while still present in snapshot. I'll have to check carefully.

[–] [email protected] 6 points 7 months ago (1 children)

I'm not an expert on btrfs, but I assume the inconsistencies come from deduplication, metadata, and maybe compression. I think some of them just count raw block storage, and some include the cost of metadata.

Traditional du assumes that each file takes up it's full space on disk which isn't always the case on btrfs. When using btrfs backed oci images, storage can easily appear multiple times higher.

I use btrfs filesystem usage /. I'm not sure that it is the "correct" way, but it works fairly well.

[–] unhinge 2 points 7 months ago* (last edited 7 months ago)

Between btrfs filesystem usage / and btrfs filesystem du -s / there's nearly 11GiB difference for used space. ~~I have checked btrfs du -hs <path/to/subvolume> for all subvolume in the filesystem, and total seems to be 72 GiB, hence the confusion. Still I don't know if I'm using the tools properly or something else is at fault here.~~

To correct myself, 11GiB is additional space used by snapshot probably ~~used space difference between btrfs fi usage and btrfs fi du -s / is because of diff between snapshot and parent volume (didn't consider that while adding all used GiB of subvolumes)~~. So btrfs filesystem usage works well to check used/free space.

edit: fix incorrect args; additional space is not diff

[–] [email protected] 5 points 7 months ago* (last edited 7 months ago)

It'd be really nice if df -h actually worked for btrfs...

Also, the discrepancies can be way worse than that.

Which is not to say don't use it, it's a way superior filesystem, just needs some polishing with these things. Also, remember to occasionally sudo btrfs balance, or better, set up a systemd to do so...

[–] [email protected] 4 points 7 months ago

I needed to find large directories on disk the other week, and found the tool btdu to be quite useful.

https://github.com/CyberShadow/btdu

[–] [email protected] 3 points 7 months ago* (last edited 7 months ago)

Masbe try compsize:
https://github.com/kilobyte/compsize