Delorean is back to full operation!
Well, I am glad to announce that I have managed to bring back all of Eagleworld's services and get the server back to full operation! Not only are we running on fresh, new hard drives, but I have recovered ALL of the data I previously lost on the solid-state drive, plus we are now upgraded to Debian 8 "Jessie," which is the current stable build.
I figured I would document here what had to be done for posterity, as it was a rather interesting experience which had me biting my teeth at some parts, but was just plain satisfying in others. However, after all is said and done, I successfully survived losing two hard drives, the fan in my backup RAID tower, AND the backup battery the server runs on, and came out back to full capacity!
First off, to fully appreciate everything that happened, you need to know the layout of all of the hard drives in my system. The system, as I outlined in the past, is nothing more than a plain old Dell Inspiron 530 I bought quite a few years ago. With capacity for only two internal hard drives, I had to get rather crafty to get this machine up to a spec that it could keep running for any perceivable period of time. In addition to having it run on an APC UPS (uninterruptable power supply), I added extra hard drives in it using a SATA RAID enclosure that connects to it via eSATA (external SATA). Here are the hard drives I had in it, and what they stored:
- Sata Drive A (/dev/sda) - "Flux" - 500GB Internal Disk Drive
- The drive that came with the computer. It had the following paritions:
- / (root) - The main parition of the operating system
- /boot, /var, /usr, and /tmp - Additional OS partitions
- /srv - A custom parition I created, which holds most of my important data (webserver data, etc.)
- Sata Drive B (/dev/sdb) - "Fusion" - 750GB Internal Disk Drive
- A drive I added in the spare slot. Used for the following:
- /srv/smb - My Samba Fileshare. Holds all of my personal data that I back up regularly and can share among all the computers in my house.
- Sata Drive C (/dev/sdc) - "Plutonium" - 60GB Internal Solid-State Drive
- Added in the spare 5 1/4" Slot. Used for the following:
- /srv/minecraft - Holds all Minecraft server data
- /srv/mysql - Holds my databases (for Minecraft, the Minecraft website, and both of my forums).
- Sata Drive D (/dev/sdd) - 1.5TB External Disk Drive
- Sata Drive E (/dev/sde) - 1.5TB External Disk Drive
- Mirrored together into a RAID-1 Array (so they are copies of one another)
- /srv/backup - Holds all my backup data
Delorean loses ANOTHER hard drive...
I KNOW I am going to lose my system drive soon in this thing if I do not act quickly. So, I need to take action. Instead of plopping any random drive in to replace my data drive, I will be buying two new 2GB drives and creating a new RAID array. I'll migrate the system drive data over to the RAID array, and recreate a samba partition and recover my data. This will give me redundancy for the future, but it's going to be a difficult move.
Not only do I have to do all of this without losing data, but I have to do it without REBOOTING THE MACHINE. Why? I'm convinced that this damn system drive WILL die if I reboot. So, this will be an adventure for sure.
So, stay tuned. I'll be recovering my data soon. I also have to see if I cannot recover my Minecraft Wordpress database from the failed SSD. When (and if) this works, I will post a full workup on how I did it.