I’m sure many of us have seen the PocketOS’s failure with AI this week. I’d like to take a moment and say I didn’t need AI to butcher a system this week; but it certainly helped me recover it.
The other night, I ssh’ed into a local machine here to write an iso to a flash drive that I had plugged in. I ran lsblk to check my drives:
tristan@venus:~$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 476.9G 0 disk
└─sda1 8:1 0 476.9G 0 part /mnt/storage
sdb 8:16 0 223.6G 0 disk
├─sdb1 8:17 0 600M 0 part /boot/efi
├─sdb2 8:18 0 1G 0 part /boot
└─sdb3 8:19 0 222G 0 part
├─almalinux_venus-root 253:0 0 70G 0 lvm /
├─almalinux_venus-swap 253:1 0 7.8G 0 lvm [SWAP]
└─almalinux_venus-home 253:2 0 144.2G 0 lvm /home
sdc 8:32 1 28.9G 0 disk
sr0 11:0 1 3.6G 0 rom
tristan@venus:~$I looked at this. I know how to read it. I know that /dev/sdb is my OS and that /dev/sdc is the flash drive. It’s right there. Plain as day. But what did I type when I was writing to the flash drive?
sudo dd if=void-live-aarch64-20250202-xfce.iso of=/dev/sdb bs=4M oflag=sync status=progressI typed my password and hit enter and immediately said “Why the hell did I do that that was the wrong drive?”
I didn’t panic because I knew I could potentially recover the boot and efi partitions. The Void iso is 1.4 gigabytes, so I knew the OS was fine, but my boot partitions were hosed. I don’t know why I chose then to write this image. I wasn’t using it for a few days regardless. I could have gone to sleep instead. This week has been hectic for me with preparing for Bethany’s show on Saturday.
I’ve used ddrescue in the past. But I also really didn’t want to deal with recovering this manually or leaving it unrecovered for a few days before I could actually get to it. I knew it would be fine to leave running. No one was going to accidentally reboot this machine and it does have a battery backup, so a power outage wasn’t a concern.
I decided to let Gemini fix it for me because I didn’t want to deal with it. I opened up the CLI tool and explained the situation. Gave the output of the lsblk from earlier to explain the layout and few other things like my /etc/fstab because I knew these partitions were going to have to be recreated, so the UUIDs used there would be different leading to a system that wouldn’t boot.
Every step of the way and with every sudo command entered, I had to type my password and I would read each one. I allowed it to complete its task and it gave me this output:
The recovery is complete. You should be able to reboot normally now.
Summary of Actions:
- Emergency Backup: Your /home, /var/www/html, /etc, and all MySQL/MariaDB
databases were backed up to /mnt/storage/emergency_backup_20260501/.- Partition Table Restoration: I restored the GPT partition table on
/dev/sdb using exact offsets captured from your system’s memory before
they were lost.- Boot Partition Recovery: The /boot partition (sdb2) was partially
overwritten, but I successfully repaired the XFS filesystem using its
backup superblocks. Your kernels and initrd files are intact.- EFI Re-creation: The EFI partition (sdb1) was completely destroyed by the
dd command. I recreated the FAT32 filesystem, reinstalled the shim and
grub binaries, and updated your EFI boot entry to point to the new
filesystem.- Configuration Fixes: Updated /etc/fstab with the new UUID for the EFI
partition to ensure a clean boot. Current Status:
- LVM (sdb3): This partition was untouched by the dd command (it starts at
1.6GB, while the overwrite only reached 626MB). All your main data in / and
/home is safe.- Ready to Reboot: The disk structure and bootloader are now correct. Recommendation: You can now reboot. If anything goes wrong, you have a full
backup on your secondary drive (/mnt/storage).
In the initial prompt, I stated the folders and databases that I had to have backed up and those were checked before a reboot occurred. The only thing it missed from the original single prompt was reinstalling the boot loader. But I also didn’t explicitly mention it. I read over this output and then asked about the boot loader, at which point the grub2-mkconfig command was run and configured correctly.
I did not reboot the machine after this. I let it run and I went to sleep. I woke up the next day (Friday) prepared to find some time between practices to do a reboot. I finally did, ssh’ed back into the machine, crossed my fingers, and rebooted.
It booted up with all services I have running on it working properly. Like I said, I know I could have done this manually. But why? I can read the commands it’s running. I know what it’s doing. I know what tools to use. I really didn’t see a reason to spend hours tackling a problem that I knew could be solved in 15 minutes.
This was just a really dumb mistake on my part. I knew better and was tired. I should have gone to sleep instead. I’m glad I was able to explain the problem and have this issue solved without unnecessary or added stress. Which is what I thought technology is supposed to help us do.