this post was submitted on 13 Sep 2023
40 points (90.0% liked)

Linux

48317 readers
632 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
 

Started with 50 MBps, went down to 20 MBps shortly after and is declining slowly since. Running for 7+ hours.
HDD is 5 years old, rare use but very well kept.

Edit: external 1.5 TB HDD connected over USB 3. Overwriting with zeroes while formatting using gnome-disks.

Update:

Stopped gnome-disks ~78% and continued writing zeros using dd for the remaining sectors.
command used: #sudo dd if=/dev/zero of=/dev/sda1 bs=1M seek=1001250 status=progress (don't copy without understanding), used seek here to skip already zeroed sectors.
write speed went up from ~14 MB/s to ~100 MB/s.

slow speed could be caused by multiple passes of overwrites by gnome-disks (not sure if it does that), or by "initializing the filesystem at the same time as zeroing" as mentioned by @ares35.

gradual speed decrement was observed in both methods, as mentioned by @Synthead.

Thanks to everyone for being so helpful.

top 21 comments
sorted by: hot top controversial new old
[–] [email protected] 85 points 1 year ago* (last edited 1 year ago) (1 children)

Looks like you're zeroing the disk. Hard drives are faster on the outside of the platters compared to the inside because there's more rotational velocity on the outside. Since the data is moving directly across the disk as its writing, yes, you'll see a speed difference during this process.

See https://superuser.com/questions/643013/are-partitions-to-the-inner-outer-edge-significantly-faster

Also related :)

[–] [email protected] 13 points 1 year ago

Yup I am zeroing it.

That explains the declining speed. Thanks a lot for the insight!

[–] [email protected] 26 points 1 year ago* (last edited 1 year ago) (1 children)

Formatting is almost instantaneous, this seems to be overwriting the files for a safe deletion (a.k.a. deleting white space). In any case, it's weird to measure a safe deletion speed as MB/s, as the operation involves multiple overwrites. I'd just format it using gparted if you don't care about deleting the white space.

[–] [email protected] 4 points 1 year ago (2 children)

Yes I am overwriting it with zeroes while formatting (using gnome-disks). I didn't know it did multiple overwrites, thought it did 1 pass overwrite with zeros.

[–] [email protected] 8 points 1 year ago (1 children)

It depends, it might be doing a single pass, but doing several is quite common.

[–] [email protected] 3 points 1 year ago

Probably selected secure erase or something which would write a bunch of random and then zero it out.

Super common in the enterprise world, probably overkill for OP.

[–] [email protected] 1 points 1 year ago

Why though?

[–] [email protected] 12 points 1 year ago* (last edited 1 year ago) (1 children)

That is very slow, unless the drive is connected over USB or failing or something, a drive of that capacity should easily be able to handle sequential writes much faster than that. How is the drive connected, and is it SMR?

[–] [email protected] 6 points 1 year ago (1 children)

It's connected over USB 3, it's SMR

[–] [email protected] 16 points 1 year ago (1 children)

If it's SMR garbage, this looks normal.

[–] [email protected] 4 points 1 year ago (1 children)

correct me if I'm wrong but just overwriting the whole disk it wouldn't make a difference if it is SMR. SMR is only slower if you want to overwrite something kn the middle of it because it then needs to change and re write adjacent tracks.

[–] [email protected] 3 points 1 year ago

I don't know what the firmware is doing here (it's device-managed SMR) but the write speed penalty also occurs for sequential writes in all DM-SMR drives I've seen measured.

[–] [email protected] 9 points 1 year ago (1 children)

initializing the filesystem at the same time as zeroing the drive is probably freaking the smr out and causing it to constantly rewrite tracks.

i would have separated the two operations. a sector-by-sector wipe via software which would run at the full sequential write speed the drive internals and interface supports (connecting internally would be faster), then partition and format the filesystem. being smr shouldn't matter here when you're writing sequentially, start-to-end.

alternatively for the wipe, use secure_erase (some usb don't support it though), handled internally by the drive so it runs as fast as it can. still takes awhile with hdd, but can be faster than doing it via software and being limited by the interface speed.

[–] [email protected] 4 points 1 year ago

Stopped gnome-disks format at 78% (~6 hours remaining).
Used #sudo dd if=/dev/zero of=/dev/sda1 bs=1M seek=1001250 status=progress, write speed is ~100 MB/s.
Thanks a ton!

[–] gudu 9 points 1 year ago (2 children)
[–] [email protected] 7 points 1 year ago (1 children)

Here is an alternative Piped link(s):

https://piped.video/tDacjrSCeq4?si=BLnrZ3nLaf5UiMhy

Piped is a privacy-respecting open-source alternative frontend to YouTube.

I'm open-source, check me out at GitHub.

[–] [email protected] 4 points 1 year ago
[–] [email protected] 2 points 1 year ago
[–] [email protected] 4 points 1 year ago (2 children)
[–] [email protected] 4 points 1 year ago

I'm not sure why this page has an obsolete warning and at this point I'm too afraid to ask 😅

[–] [email protected] 2 points 1 year ago

This is not recommended for USB connected drives