this post was submitted on 30 Jan 2024
43 points (95.7% liked)

Linux

48375 readers
1001 users here now

From Wikipedia, the free encyclopedia

Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).

Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.

Rules

Related Communities

Community icon by Alpár-Etele Méder, licensed under CC BY 3.0

founded 5 years ago
MODERATORS
 

I know about Clonezilla and copy pasting partitions with gparted, but can I just use dd to copy a partition with batocera to a USB stick and will it then boot from the stick? Do I have to set the boot flag or take any other steps?

Thank you for any tips.

all 50 comments
sorted by: hot top controversial new old
[–] ricecake 18 points 10 months ago (3 children)

It might work, but probably not without a little tweaking.

A lot of things will reference drive identifiers or drive path to know where to mount them. These things don't get copied by dd.

[–] [email protected] 10 points 10 months ago* (last edited 10 months ago) (1 children)

huh?

Isn't it the other way around?

I once cloned an nvme with dd and had to physically remove one of the two, because they had the same id and the bios couldn't differentiate between them and would randomly boot into either the first or the second one, inconsistently.

And removing either one would boot into an identical system with everything mounted and working. Which caused some confusion until I realized that the id was copied over.

So unless you didn't use the id in fstab, you should be fine. Sure the device path may differ, but that can happen anyway to usually devices should be referenced by id.

[–] ricecake 6 points 10 months ago

There are different schemes that different distros use. Some user partition id, some use fire system id, and some use device serial number and partition index.

[–] [email protected] 3 points 10 months ago (1 children)

Thanks, so just update the UUID in fstab for the stick? Or is there more?

[–] ricecake 4 points 10 months ago

Probably all you need to do is check to make sure things look right, and actually test it.

I can't think of anything else that would be common that you'd want to check. If you're running weird virtualization setups on your laptop you might have to do more. :P

[–] stewi1914 1 points 9 months ago* (last edited 9 months ago)

dd'ing /dev/sdx will copy all IDs

dd'ing /dev/sdx1 will keep UUID but PARTUUID will remain the same on the destination

[–] [email protected] 10 points 10 months ago (3 children)

If you dd your entire laptop drive to the entire flash drive it should be able to boot. It might not work the same with just one partition though. Also, how big is your USB stick!?!?

[–] [email protected] 18 points 10 months ago

( ͡° ͜ʖ ͡°)

[–] [email protected] 6 points 10 months ago (1 children)

You can easily get 1TB USB sticks.

[–] [email protected] 1 points 10 months ago

The biggest flash drive I've ever used is 32gb lol

[–] [email protected] 4 points 10 months ago (2 children)

Lmao they are huge these days! But I'm talking about a 40gb partition, cloning it to a 64gb stick.

You can't even buy 2gb sticks anymore :/

[–] [email protected] 4 points 10 months ago

Small USB flash drives are still available, you just won't find them in the local stores. There are expensive, industrial grade drives from places like digikey and dirt cheap, no name drives from Chinese sellers where they will print your logo on them for free if you buy a hundred of them.

[–] [email protected] 3 points 10 months ago* (last edited 10 months ago) (1 children)

One partition is not enough. You also need to copy a EFI boot partition and create a correct partition table manually. And note that you cannot get a correct result when partitions you are copying are mounted. You need to boot some live system to do this.

[–] [email protected] 1 points 10 months ago (1 children)

I've heard that systemd-boot can do it with just one partition but I don't know howww also somebody please eli5 BTRFS to me, my brain's doing the full-sponge thing.

[–] [email protected] 2 points 10 months ago

It is possible to boot from a single partition, USB live images work this way. But regular installation that OP wants to clone likely has a separate partition.

[–] [email protected] 7 points 10 months ago* (last edited 10 months ago) (2 children)

You can just use `# cat /dev/your-disk > /dev/your-stick, no need for a (dd) scalpel there.

If your system uses UUID's in /etc/fstab, you have to change them to match the current partitions to have it bootable. lsblk -o+UUID is nice for that.

[–] Secret300 2 points 9 months ago (1 children)

Wait what? You can use cat like that?! That's dope

[–] [email protected] 3 points 9 months ago (1 children)
[–] [email protected] 1 points 9 months ago (1 children)

.. not if you took a blue pill overdose ..

@MonkderZweite @Secret300

[–] [email protected] 2 points 10 months ago (2 children)

Ok, thanks. That sounds pretty good.

If i want to compress it and save it as a backup can I do

cat /dev/sda3 | gzip -9 > drive.img.gz

?

[–] [email protected] 4 points 10 months ago* (last edited 10 months ago)

fill up the remaining space on the drive completely with 0s with a dummy file you delete then, before gzipping

[–] [email protected] 2 points 10 months ago* (last edited 10 months ago) (1 children)

Yes, but like @kuneho said, since "deleted" stuff only is marked as deleted (not wiped), there's always a bunch of random on the "empty" space part of a disk, which compresses badly.

Do cat /dev/zero > /path/to/mounted/partition/zeroes and delete it after cat errored out because no space, to fill the "empty" space with zeroes.

[–] [email protected] 2 points 9 months ago (1 children)

That worked really well! I got a 50gb partition with about 30gb free space into a 10gb zipped image. Is there any way to show progress during the operation like with dd's status=progress?

[–] [email protected] 1 points 9 months ago* (last edited 9 months ago)

I often use pv instead of cat for this. And there's some 'hack' where some tool looks at some kernel feature to guess progress of cp & co. But i forgot it's name.

[–] [email protected] 5 points 10 months ago* (last edited 10 months ago) (1 children)

#Big tip: use chatGPT to help you with commands.

You can say something like "I want to backup my entire file system using [this program]. I want to create a tar file, and then copy that to an external drive. I want the program to display progress while it's copying, then verify the files have been copied correctly once it has finished. I am using [whatever distro]. Please give me the commands I need to achieve this."

Then it will spit out the commands you need to input with the correct syntax.

The best part is that you can then ask it to clarify any parts of its answer so you can learn how it all works and make sure the commands are doing what you asked. And if your computer spits out an error, just paste that back into chatgpt and it will help you resolve it.

Seriously, chatgpt is like a 24 hour on call Linux guru friend. It's saved me so much time and effort over the last few months doing things like this.

[–] Secret300 1 points 9 months ago (1 children)

Swear to God. I hope someone develops something so I can just say what I want to do in the terminal and an AI will create the command then prompt "run this command? Y/N/E" E to explain it

[–] [email protected] 2 points 9 months ago

Someone’s probably working on that as we speak

[–] [email protected] 4 points 10 months ago* (last edited 10 months ago) (1 children)

you can copy your system live, but that would involve other tools than dd too.

with dd when copying the whole device (instead of just partitions) everything gets cloned. This includes uuids, labes, lvm devices with the names of their lv and vg names and raid devices in case you have any. all of these (c|w)ould collide unless the original disk was taken out or either the new or old disks labrls uuids etc are previously to the boot changed to prevent collusions or accidently mounting/booting the original partitions. also if (!) you use device names i.e. in fstab, crypttab, scripts or such, like with the uuids things could break. also you might have to take action for your bios to actually boot from the stick. most people disable usb boot on notebooks for security reasons.

using dd, cloning the full disk to the full stick, then removing the original disk + set bios boot setting might work out of the box, i'ld try that first as it takes only the effort to boot from another os to do the dd-copy offline (preventing filesystem damage while copying).

a live copy could be done by cloning only the partition layout and bootloader, then setting up new filesystems (with new uuids) and new lvm group/volumes etc if any, copying original disk using rsync then (maybe "bind" mounting to separate partitions if needed), then adjusting boot config to match new uuids/labels. This could be done while running the system to be copied, but of course even running rsync twice might lack some updates of currently open files by sth like desktop programs or logfiles.

Without knowing the exact setup, only limited answers can be given, but you have to make sure the boot process will work, so at least the boot loader (grub?) and its files will be needed, which -at least in the past and for old lilo/grub- could not reside at some position on the disk after some "high value" like some number GBs. if that limitation is still there, your new exact partition layout on the usb stick might be relevant for success, but try/error should give you the hints you need.

you might use "language models" for getting hints, but they are language models, not friends, their "solution" might break your system and delete your data, and they are trained to say they are sorry afterwards, but the are'nt sorry, its just a sequenze of probabilities and words to them not more.

So always only work on data that has been backed up and prooven to be suitable for you to recover everything you need from scratch, no matter if friends, language models or lemmy users assist you ;-)

UPDATE: just learned that batocera is "designed" to be just copied to usb stick and run from there, so it will most likely already include everything you need. best is to follow their instructions how to create the usb stick to boot from. if you already have it running from partition, you most likely can copy your current data using rsync. but beware, if you have two copies with the same uuids (partition +usb) that might not work as expected.

[–] [email protected] 1 points 10 months ago (1 children)

Thanks, I'm gonna have to read that a few times ':D

[–] [email protected] 1 points 9 months ago

and i hope that has nothing to do with my chaotic style of writing =D

[–] [email protected] 4 points 10 months ago (2 children)

First tar up your filesystem. You can use gzip to save space. Once that's done copy it over. Make sure you use a good filesystem on the USB device that isn't going to cause data loss. (ie not fat)

Keep in mind that you can't do this live and that copying the archive over will take time and the drive may lie to you. Its always best practice to run sync or eject before unplugging.

[–] [email protected] 2 points 10 months ago (4 children)

I just moved to Linux two days ago and omg there is so much to learn but something I never heard of until 20.minutes ago and again right now is tarring...what is it?

[–] [email protected] 5 points 10 months ago (1 children)

Its a compression program for making file archives.

[–] [email protected] 2 points 10 months ago (1 children)

So basically Zip but better?

[–] [email protected] 4 points 10 months ago

Its not necessary better as tar by itself doesn't have compression. You need gzip, bzip2 or something else. On modern systems with gnu tar can handle tar with compression.

If your still curious I would look up how to use it.

load more comments (2 replies)
[–] [email protected] 1 points 10 months ago (1 children)

Wait why can't I do it live? Thanks for the tips.

[–] [email protected] 3 points 10 months ago

Because it is in use. You technically can hot swap a file system by switching run levels and temporarily switching to a tmpfs but I've never succeeded.