this post was submitted on 27 May 2024
38 points (95.2% liked)

Linux

46794 readers
1250 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by AlpΓ‘r-Etele MΓ©der, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
 

Solved: decided to avoid the funkyness this would invoke and just bought another drive. all good nowπŸ‘

About a year back, I moved my internal 8tb and 4tb HDDs from my main Windows machine to my old PC-turned-Linux-server. They hold a bunch of bulk data like Youtube channel archives and torrents that are open to download.

I would like to do an in-place ext4 conversion, if possible. Currently I've just started shuffling data off to an SSD and the plan was to slowly shrink the NTFS partitions and turn the new space into ext4, 500gb at a time (size of the intermediary SSD), but it is taking an unbearably long time. Shrinking the 4tb partition in gparted has been running for 13 hours, with an estimated 22 hours remaining! And I'll have to do it 7 more times for the 4tb, and 16 times for the 8tb!!

Is there a better way to do this?

all 20 comments
sorted by: hot top controversial new old
[–] [email protected] 22 points 2 months ago* (last edited 2 months ago) (1 children)

hard drives are going to be slow af copying data to itself, or moving data to a different partition on it.

then you're also adding partition size manipulation to the mix, which will also be slow af when data has to be moved off the 'end' of partitions to 'make room' to enlarge or create another with a different fs.

your best option is to get another drive, even if it's also a hard drive instead of ssd. use that to move (copy, really, to preserve the original as a backup for the time being) all the data to that you want to preserve.

[–] [email protected] 7 points 2 months ago

Found that also myself trying to do the same thing haha. I did the same process as OP, gparted took 2.5 hours in my 1TB HDD to create a new partition, then copying the data from old to new partition was painfully slow, so I went to copy it to another dive and into the new partition.
Afterwards I deleted the old partition and grew the new one, which took a bit more than 1.5 hours.

If I had the space I would have copied all the data out of the drive, formatted it and then copied back into. It would have been quicker.

[–] [email protected] 19 points 2 months ago

Since you really should be creating a backup of the data before doing such a conversion in the first place, the best (not necessarily the fastest, but definitely the safest) way would be to copy the data to another medium, and copy it back when the space has been formatted.

[–] [email protected] 11 points 2 months ago

Files are files and filesystems are filesystems. You keep your files on filesystems.

NTFS and ext4 are non convertible - you cannot turn one into the other directly, in place. However you can take files from one and put them on another.

Yes, moving TBs does take time, sorry it is unbearable.

[–] [email protected] 6 points 2 months ago

Backup and restore after formatting? I would assume you have backups in place already

[–] [email protected] 6 points 2 months ago

It'll be much faster the next time. It has to make sure all the data is out of the space to be freed. Assuming it moves it as close to the start of the partition as it can (and you're shrinking it from the end) then it'll be faster.

If you're shrinking it from the start, yeah, it's going to take forever because it will always have to move a lot of data.

[–] [email protected] 6 points 2 months ago (2 children)

you can't get there from here.

you can convert to btrfs, but no matter what anyone says, there lie dragons.

if you end up doing big bulk reads and writes to and from your disks, keep them cool. if that means limiting bitrates with the rsync command or just plugging them up to a 802.11g laptop and copying from there to your target then that's fine. if it means making a funnel to direct a box fan's worth of cfm onto your drives then that's fine.

[–] [email protected] 3 points 2 months ago

limiting bitrates with the rsync command

/cries in USB2 HDD caddy

[–] [email protected] 1 points 2 months ago* (last edited 2 months ago)

Decided to buy another drive instead of doing any more harm than I needed to, no worries

[–] [email protected] 4 points 2 months ago* (last edited 2 months ago) (1 children)

I'm not sure what the chances actually are, but that much rewriting and shuffling data, if it doesn't immediately result in data loss it is going to put a lot of wear on your drives. If your largest drive is 8 TB, I'd look for another 8+ TB drive so that you can copy the data and then reformat. Even a slow external drive is likely to be faster than what you're doing.

[–] [email protected] 1 points 2 months ago

Yep, I've just ordered another 8tb to copy to and avoid the headache that could be a drive failure. And it'll certainly be faster, gparted is still giving a 13 hour ETA for the first resize! Thanks for the help!

[–] [email protected] 3 points 2 months ago (1 children)

I don't know anything that can do an in-place ext4 conversation, but there's ntfs2btrfs which is already in the Debian repos if you're okay with BTRFS.

Of course, backup anything important, ntfs2btrfs should create a backup snapshot if you need to revert back to NTFS, but I wouldn't count on it.

[–] [email protected] 1 points 2 months ago (2 children)

What's the advantage in btrfs over ext4? I've kept hearing about it since I started with Linux but the only advantage I can see with it is the snapshot rollback feature, which while useful looking, I don't think would be something I would use

[–] [email protected] 4 points 2 months ago

If you are still dual booting, btrfs has a very good windows driver. Btrfs is newer and only recently started becoming the default on a couple mainline OSes. Ext4 has been around forever and is assumed to be much more stable.

Fwiw, I've been using btrfs for the last 18 months or so without any issue. I don't use any of the tools. There's no obvious or immediate performance difference like there was in the old days. We're all on SSDs now and they're fast no matter what.

[–] [email protected] 4 points 2 months ago

It actually checks for corruption, can do raid to fix corruption, is CoW so less likely to corrupt. And it can do transparent compression.

[–] [email protected] 2 points 2 months ago

Use rsync -avs to move files from NTFS to your new filesystem. XFS is pretty good for lots of files.

[–] [email protected] 1 points 2 months ago

If it is not sensitive data, and you're okay being morally objectionable, you can buy a large hard drive from some place with a good return policy, transfer your data to it, format/repartition your drive, transfer everything back and return the hdd for a 15% stocking fee.

[–] [email protected] 1 points 2 months ago

Why not just leave them as NTFS for now? The new in-kernel NTFS3 driver is actually pretty decent (since kernel 6.2), and shouldn't pose any issues if you're just using it as a bulk data store.

Eventually when you replace the disks, you can can format your new disks as ext4 (or even better, use btrfs or bcachefs).