source ~/.bash_history
Dear god
That's the scariest horror story in 2 words I've seen so far
I'm genuinely having a chuckle at how shocked people are at my submission, made my day xD
Perhaps not the same definition of "broken" that you're looking for, but when I first started using Linux, I was using Kubuntu as my first distro have some brief experimenting with Manjaro.
Anyway, back then, I for some reason had the Skype snap installed. Can't recall why I had it to begin with, but I decided later on that ofc I didn't need Skype, and of course uninstalled the snap.
A few days later, I was met with some storage issues, where I had a limited amount of storage left on my SSD. I'm sitting there a little confused since I swore I was using less storage, but I did a thorough cleaning of my computer by deleting files I didn't necessarily need, and uninstalling any programs that I hardly ever used. That seemed to do the job, even if it was less storage space...
Until the next day, when the storage was full again. After getting some help from someone, I found that Skype, despite being uninstalled, was still running in the background, and found that there were residual files. The residual stuff running in the background was trying to communicate with what I had uninstalled, and logged multiple errors per second in a plaintext file that ended up being 176GB.
Whether I did something wrong or if there was something up with the snap, I still don't know as this was over a year ago and I was still learning the ropes of Linux at the time.
Least broken snap
Many many years ago I wanted to clean up my freshly installed Slackware system by removing old files.
find / -mtime +30 -exec rm -f {};
Bad idea.
Not me, but one I saw... dude used chmod to lock down permissions across the board... including root... including the chmod command.
"What do I do?"
🤔
"Re-install?"
You could boot on an USB, mount the filesystem and change the permissions. But if the dude changed a whole lot of permissions, reinstalling might be the smart thing to do...
Changed it all to 000. ಠ_ಠ
Tried to convert Ubuntu to Debian by replacing the repos in sources.list and apt dist-upgrading. 💣 Teenagers...
One that I can remember many years ago, classic trying to do something on a flash drive and dd
my main hdd instead.
Funny thing, since this was a 5400rpm and noticed relatively quick (say 1-2 minutes), I could ctrl-c the dd, make a backup of most of my personal files (being very careful not to reboot) and after that I could safely reformat and reinstall.
To this day it amazes me how linux managed to not crash with a half broken root file system (I mean, sure, things were crashing right and left, but given the situation, having enough to back up most things was like magic)
sudo rm -f /lib /usr/share/backup/blah blah.tar.gz
Note the space.
Oh man, you really owned those libs
You need to use chown if you want to own the libs
First, the classical typo in a bash script:
set FOLDER=/some/folder
rm -rf ${FODLER}/
which is why I like to add a set -u at the begining of a script.
The second one is not with a Linux box but a mainframe running AIX:
If on Linux killall java kills all java processes, on AIX it just ignore the arguments and kill all processes that the user can kill. Adios the CICS region 😬 (on the test env. thankfully)
If on Linux killall java kills all java processes, on AIX it just ignore the arguments and kill all processes that the user can kill.
jfc, is ignoring arguments the intended behavior?
On a real UNIX (not only AiX) killall is part of the shutdown process - it gets called by init at that stage when you want to kill everything left before reboot/shutdown.
Linux is pretty unique in using that for something else.
Wow, the last one is quite unexpected. What a useful command
I can't even remember how I did this, but overwriting the partition table on the main production server at our small startup (back when "the server" would usually live on the premises of the startup). I remember my boss starting to hyperventilate from panic while I reconstructed it from memory / notes, and all the filesystems came back and he calmed down.
Same job, they gave me a little embedded-systems unit for me to use to build a prototype on. I hooked it up, nothing worked. I brought it back to them.
Hey, this one doesn't work.
Huh... that's weird, it was working before. Did you break it?
I don't think so. Can I have one that works?
They literally told me, as they were handing me the second one: Okay, here's another one. Don't break it.
I figured it out literally seconds after breaking the second one... I was hooking it up to 12 volts of power when it needed 5. Second dead computer. Explaining that and that I needed a third one now was fun.
I thoroughly backup up my slow nvme before installing a new faster one. I actually didn't even want to reuse the installation, just the files at /home.
So I mounted it at /mnt/backupnvme0n1, 2, etc and rsynced
The first few dry runs showed a lot of data was redundant, so I geniously thought "wow I should delete some of these". And that's when I did a classic sudo rm -rf in the /mnt root folder instead of /mnt/dirthathadthoseredundantfiles
CTRL-C-ing apt because it looked stuck for more than 10 minutes. I don't recommend doing it.
Not quite catastrophic but:
I'm in the process of switching my main server over from windows to Linux
I went with Deb 12 and it all works smoothly but I don't have enough room to back up data to change the drive formats so they're still NTFS. I was looking at my main media HDD and thought "oh, I'll at least delete those windows partitions and leave the main partition intact."
I found out the hard way that NTFS partitions can't just reclaim space like that. It shuffles all the data when you change the partition. It's currently 23 hours into the job and it's 33% done.
I did this to reclaim 30 MB of space on a 14 TB drive.
Tinkering is all fun and games, until it’s 4 am, your vision is blurry, and thinking straight becomes a non-option, or perhaps you just get overly confident, type something and press enter before considering the consequences of the command you’re about to execute… And then all you have is a kernel panic and one thought bouncing in your head: “damn, what did I expect to happen?”.
Nah, that's when the fun really starts! ;)
The package refused to either work or install complaining that the version of glibc was incorrect… So, I installed glibc from Debian’s repos.
:D That one is a classic. Most distributions don't include packagers from other distros because 99% of the time it's a bad idea. But with Arch you can do whatever you want, of course
My two things:
- I've heard about some new coreutils (rm, cp, cat... this time the name really fits the contents :D) and I decided to test it out. Of course it was conflicting with my current coreutils package and I couldn't just replace it because deleting the old package would break requirements. So without thinking I forced the package manager to delete it "I'll install a new one in just a second". Turns out it's hard to install a package without cp, etc :D
- I don't remember what I was doing but I overwrote the first bytes of hdd. Meaning my partition table disappeared. Nothing could be mounted, no partitions found. Seemingly a brick.
Turns out, if you run a rescue iso, ask it to try and recognize partitions and recreate the table without formatting, Linux will come back to life as if nothing happened
Before installing Arch on a USB flash drive, I disabled ext4 journaling in order to reduce disk reads and writes, being fully aware of the implications (file corruption after unexpected power loss). I was confident that I would never have to pull the plug or the drive without issuing a normal shutdown first. Unfortunately, there was one possibility I hadn't considered: sometimes, there's that one service preventing your PC from turning off, and at that stage there's no way to kill it (besides waiting for systemd to time out, but I was impatient).
So I pulled the plug. The system booted fine, but was missing some binaries. Unfortunately, I couldn't use pacman to restore them because some of the files it relied on were also destroyed.
This was not the last time I went through this. Luckily I've learned my lesson by now
First time trying Linux I went with an arch install because I Googled "best version of Linux" and went with arch. Followed a guide to the point of drive formatting and I decided to go with a setup with drive encryption. I didn't understand what I was doing, ended up locking myself out of my hard drives and couldn't get windows to reinstall on them. I used a MacBook for a week until I installed Ubuntu and managed to wipe and reset my drives and reinstalled. Needless to say I am going to read up a little more before I try that again.
Found out the hard way that if you edit /etc/sudoers with anything other than visudo you best be absolutely sure the syntax is correct, otherwise sudo will refuse to read it and you'll be locked out.
Also learned to add -rf to the rm command at the end, after I re-read it to make sure it does what it should do. Something like rm /path -rf instead of rm -fr /path. That protects you from your fat fingers hitting the enter key half way through.
fstab bind mount for /home that I misspelled, so I couldn't login as myself.
fstab external hdd mount that didn't have ignore flag so PC would pop if I booted while unplugged
Accidentally booting windows after a year and it overwrite my EFI boot entry.
The best I've see however was an acquaintance who accidentally set perms to own user on /usr/bin
So everything went from root:root to user:user which removed all the SUID/SGID bits as well so a bunch of bins broke lol.
Believe it or not, it was actually fairly easy to fix with chmod and chown
I have an embedded device that runs as root and has busy box. I accidentally ran chmod -x on one of the busy box sym links (strings I think) and it made all of the core utils un-executable. Unfortunately chmod is a core util. Every bash command was throwing wild errors and outputing gobbledygook. You really start to sweat when ls and cat stop working.
I had full disk image of the device and started deducing the issue. Luckily I could still execute non core utils and ssh/scp was working so I wrote a little program to restore permissions, uploaded/ran the binary, and learned nothing from the experience.
stupid was when I wanted to test Linux Mint on an external SSD, and didn't check that the bootloader wasn't going to overwrite my internal drive's.
So anyway I'm running Linux Mint now.
I uninstalled Python.
I was playing around with Pygame of all things, and it wasn't behaving as the (apparently out of date) documentation was saying it should, so I figured I'd just uninstall and reinstall Python.
EVERYTHING borked. APT wouldn't even work.
Deleted my entire efi partition while trying to install some grub themes.
And then my backup didn't work when I tried to restore it.
I have pretty colours now though, so it was all worth it :)
Years ago I was dual-booting with Ubuntu just to try out whatever this Linux thing was that all the nerds were talking about. Liked it and played around with it, but for whatever reason I wanted to go back to just Windows, I needed the space I had partitioned off or something, can’t remember why. So I just uninstalled or deleted the bootloader somehow (maybe I just deleted the Linux partition and expected the space to clear up like normal).
Go to restart the computer… oh shit. Ohshotohshitohshitohshit.
I've done my plenty of stupid stuff, from dd disks I was using to forcefully uninstall dependencies of the package manager. But the one that takes the cake for me happened back in 2012, I was working at a research lab in the university and was sharing a computer with another intern. That other intern used Gentoo and so we agreed that the machine should be Gentoo, I've installed it at my house on my PC and got comfortable with it before we shared that computer. One thing that I learnt when installing Gentoo is that the /dev
folder is created on boot, you don't populate it when installing, instead you mount the one from the host system you're using to install.
The computer had an issue with a device, can't even remember what it was, so I thought I'll run rm -rf /dev
that should take care of the issue and after a reboot it will be repopulated... It might have worked, but what I actually ran was rm -rf /etc
.
Just straight up overwriting boot sector and superblock of my hard drive thinking it's the USB drive.
Udev tried to warn me, saying there's no permission, and I just typed sudo without thinking.
Then after a second I remembered USB block devices are usually writable by users, but it was too late.
I don't know if that counts, but on a fresh default Debian Stable system, my cat walked across the keyboard and the DE crashed.
I could still switch to another TTY and reboot via command line.
After the reboot, I was greeted by a blinking cursor and nothing else. Had to reinstall.
keyboard cat strikes again
I wanted to use fio to benchmark my root drive. I had seen a tutorial saying that the file=
parameter should point to the device file, so I pointed it at /dev/sda. As you might expect, the write test didn't go so well.
I have a stupid one, but far from funny.. I've been using and building computers for a very long time so I'm far from a noob, but I'm still quite cautious, bordering on paranoid, so I like to unplug all other drives when re/installing an OS just to avoid stupid mistakes. I go through the installer on the livecd, there's only one drive to choose from so I don't think much about it, select that it should erase everything, I set up the new partition structure, and start the process. After about a minute I begin wondering "why is it taking so long?", and "what is that ticking noise? SSDs shouldn't be making any sounds when written to", when I realize that I had unplugged the wrong drives and that I was currently overwriting my main storage drive. Of course I had backups of the most important things like photos and code, though not really synced for a couple of months so I lost some stuff permanently.
I can totally feel that sudden clot in the gut the moment you realize on which drive the action is happening, just by reading this.
Learned about the importance of trailing slashes in rsync by using the -delete flag.
My own classic was fiddling with the nvidia PRIME config to try and get rid of some very mildly irritating screen tearing. No graphics output at all. Now this is fixable of course, but it's a pig.
And I'd decided to do this 2 hours before an incredibly important progress review meeting for my PhD.
Got it back with about 10 mins to spare and decided just to leave the driver config alone after that.
Bonus round
Also a friend managed to bork his ubuntu 16 laptop by trying to switch from unity to gnome and ending up with sort of neither. That was reinstall territory right there.
Backed up the whole disk image to an external drive because I didn't have time for a proper backup but knew I would need some of those files later.
Installed a fresh new OS on the same disk, used it for a couple of months.
Needed to make some space on the external drive I had the backup on so I'll just delete the backed up system files from it.
cd /mnt/external_drive
rm - r /usr /boot ...
As you can probably see, a fresh new install was happening again
Somehow convinced a person to run sudo chmod -x /usr/bin/*
I don't remember the exact command so it could be a bit different but it did the job. It was a fun evening.
An intern nuked their workstation by sudo chmod -R 777 /
. Turns out adding exec to everything isn't good either.
The first time I read this, i thought "shouldn't all that be executable anyway?"
And then I read it again and realized, minus x
I've broken systems far too many times in the last 24 years, since Mandrake 6.x, to count:
- I've dd a disk or more
- I've rm *
- I've chmod
- I've brought down the network, with every intention tar it would come back - on a remote box
- I've failed to RTFM far too many times
This was pre-linux for me but something you can still do in most distros so I think it's a valid story.
In 1999 I was using Napster on computer running MS-DOS. I was 12 years old and an aspiring open media enthusiast/stupid script kiddie. I was using the file explorer interface in Napster and accidentally gave access to my entire C drive. I also had opened ports to share certain media and to fuck with my friends using daemon tools (back then you could do stupid stuff like control a friend's desktop with certain versions of daemon tools). Immediately I started receiving packages called things like "sleep.tight.tiny.mite" and I knew I was fucked so I clicked in the Napster interface and clicked "delete" and deleted my entire active drive.
I panicked and installed the only operating system we had which was a random copy of Red Hat. When my dad came home I pretended like it had always had Linux on it. I do think he was more impressed than mad.
Linux
From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Rules
- Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
- No misinformation
- No NSFW content
- No hate speech, bigotry, etc
Related Communities
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0