Linux

A Series of Unfortunate Events

(with apologies to Lemony Snicket)

Last week was not all that great for me computer wise.

I had three events that pretty much threw me for a loop. If it hadn’t been for Ginny, I would have probably been a drooling, babbling, idiot.

Where should I start? Hmmm, the beginning, maybe?

(cue fade to memory music)

It all started the week before last … I was figuring out what to do with the week off between Christmas & new years. My linux servers (Gondor and Rivendell) are both running Fedora Core 4, and Core 6 was released a little while ago. I figured it would be a fine time to upgrade both servers.

The original plan was to upgrade the servers xmas eve, while Ginny was at church. That plan got changed, however, because Ginny didn’t want her churches website down over Christmas. Ok, later in the week would be fine.

So on Christmas eve, we went to my brother Mitch’s house for dinner. While there, Mitch showed me his Hauppauge MediaMVP . It’s a pretty slick device that lets you play videos (and music & pictures) on the TV. Since we have a SnapStream Beyond TV software running on a system in the basement, we have a lot of video files that can be played. Currently the only way we can play them is to hook up a laptop to the TV and play the videos that way. I figured it would be a nice addition to our A/V setup.

Christmas comes and goes … everything is nice, had a fun time we the family.

On December 26th, my friend Steve calls and asks if I want to take a trip down to Frys to see if they have any deals or specials going on. Sure I say, that could be fun. I’ve been wanting to get the wire to connect my Dell PE430SC server’s AUX_LED to the PCI IDE controller … just so I can see disk activity.

Since I wanted Steve to be able to help me find this cable, I opened the case to take pictures of the connectors. Got my pictures and proceeded to close the case. This is when the trouble happened. I accidentally bumped the ArcoIDE Duplidisk mirroring adapter. This must have caused a temporary problem because the unit started beeping as if one of the drives had failed. A drive hadn’t failed … but the drives were no longer in sync. Well, this shot the trip to Frys.

I spent the next couple of hours trying to figure out what what happened … and get the drives back in sync. I had to call Arco because there were problems getting the adapter recognized by the management software. I ended up attaching the primary hard drive of the mirrored pair directly to the ide controller. Rebuilding the mirror would have to wait a few days.

I decided to fix the mirror and do the Linux upgrade on Saturday the 30th.

I woke up bright and early on Saturday, fed the cat, fed myself, and went down to start my work.

First thing to do is get the mirror re-established. It took a bit of work, because I still couldn’t get the software to recognize the adapter. I determined that it was because the adapter was attached to a add-on PCI IDE card. When I hooked the adapter up to the motherboard’s IDE (it only has one), it worked fine. This is going to take 3 hours for the mirror to be rebuilt. It has to duplicate all the contents of the primary drive to the mirror drive.

While this is running, I started to do the upgrade on Gondor.

While the servers were down, I had setup a dummy “This site is under construction” page on another machine (shadowfax, for what it’s worth). My plan was to assign all the IP’s I have to that one machine so nobody would think my sites were just gone.

Got it all setup … but I couldn’t hit the dummy page. I figured it was because the DSL router still had the old machine’s mac addresses in memory. This has happened before. No problem, I telnet into the router and select the “Restart device” option. The device restarts … but then the 4 lights that correspond to the ethernet ports on the router start blinking at the same time … and the DSL light is not on. I try telneting to the router again, but get no response. I try using the RS232 connection to log into the router, but get no response. Nothing is working.

I also tried connecting my laptop to one of the ethernet ports on the router and notice that windows alternates between saying that my ethernet is Connected and The cable is unplugged at the same frequency as the blinking lights.

I call Covad and talk to a tech … it’s pretty quickly determined that the router had died. The earliest they can get someone out to replace the router is Monday the 1st.

Well, the midrange.com activity will be light because of the holiday … so I shouldn’t expect too many panic calls because there isn’t any list traffic.

So Sunday I decide to get the MediaMVP unit setup.

I had ordered it from amazon.com earlier in the week, but hadn’t had a chance to set it up. My original plan was to use my D-Link DWL-G730AP Wireless Pocket Router in client mode. In this mode I can connect a wired ethernet device to it and it connects to my wireless router.

Unfortunately, I couldn’t get the two devices to play together. Well, my boss had given me a $30 gift card for BestBuy, so I figured getting a wireless gaming adapter would be a fine use for it. I went to the Schaumburg BestBuy, but they didn’t have any wireless gaming adapters in stock at all. I then went to the Arlington Heights BestBuy and found they had one. The Linksys WGA54AG Game Adapter 802.11A/G. I didn’t really need the wireless “A”, but it should work.

I got the unit home and started to configure it … but then found out that it didn’t support WPA encryption, just WEP. That’s stupid. Half the reason to use 802.11G is to have better encryption.

I bit of searching online and I found that CDW had the D-Link DWL-G820 Wireless Gaming Adapter, 802.11g, 108Mbps. Luckily, there is a BestBuy near CDW.

So off I go to Vernon Hills. I return the Linksys device and go to CDW. Unfortunately, although CDW has the DWL-G820 on site, it’s not off the truck yet. So I have to wait … about 2 hours.

I finally get the new gaming adapter and get it home … it works, although I still haven’t gotten the MediaMVP to work correctly yet. I need to futz with it a bit more.

HAPPY NEW YEAR!

Ok, New Years Eve is over … New Years Day is here. The Covad guy is supposed to be here between 8am and noon. Around 10:45 he calls to say he’s in the city, but on his way out … he should be there a little before noon. That’s refreshing.

He gets here exactly when he said he would … runs a few tests, says “Yeah, the router is fried”. Gets the replacement out and starts to configure it. About 30 minutes later, it’s all configured and we’re back on the air. All I have to do is re-implement my firewall filters. Sure wish I had backed them up a while ago. Oh well, they aren’t that complicated … it’s just a bunch of typing.

It takes me about 2 hours to get the router reconfigured the way it should … then I can get the networking on the linux boxes working again.

All in all, I did a lot of work … but ended up right were I started.

Seems to me that this kind of thing has happened before … a few days off from work, a grand plan to upgrade a system or two, and a bunch of failures that cause nothing but problems.

I’m cursed, I tell you, cursed!

[tags]Linux, MediaMVP, Fedora Core, DLink, Linksys, Covad, DSL, Upgrades, RAID, Wireless[/tags]

Popularity: 1%

Noisy

The new tape drive arrived today … I was a bit worried that it would ever show up, as the vendor I purchased from seemed to be pretty disorganized.

It seems to be ok. It’s even new. I was half expecting the unit to be refurbished, which I would be OK with, because the price I paid was $200 less then the next higher vendor.

The only problem with the tape drive is that it’s noisy as hell … louder than both of the servers put together.

One oddity I found … the old tape drive would erase tapes in under a minute … but the new tape drive wants to write over the entire length of the tape. When I issue the mt erase command, it spins for a very long time.

Anyone want a marginally working SCSI DDS3 tape drive? ;)

[tags]linux, backup, tape, dds3, usb, scsi[/tags]

Popularity: 1%

Hard Drives Hell

Oh boy … talk about a bad day for hard drives.

Thursday evening I noticed my laptop was getting REALLY sluggish … I ran uptime (part of the MKS Toolkit, which I can’t do without) and found out my system had been up for 15 days straight without a reboot. Almost a record for a Windows machine, I think. So I decided to reboot the system.

I started the reboot process and noticed it was taking a VERY long time to boot … so I powered off again and ran diagnostics. The hard drive was failing. Unfortunately, because this hard drive was an upgrade I purchased from Dell, it was not covered under my laptop’s warranty. And, even though Hitachi offers a 3 year warranty on the drive, Dell only gives a 1 year warranty.

Oh well, hard drives aren’t that expensive … so I figured I would swing by Frys after work on Friday. I was confident that I had a solid backup of the system, that was only a few days old.

During the day on Friday, someone told me that the midrange.com archive server was very very slow. I checked it out and saw that that system (gondor) was showing disk errors on the 2nd hard drive. Oh joy, another drive to buy at Frys.

I went to Frys to get replacement drives … I wasn’t super happy with the selection (although they had a huge quantity of the drives they had available). I ended up getting a 100gb, 5400 rpm, Seagate PATA drive for the laptop and a 300gb, 7200 rpm, Seagate SATA drive for gondor.

Getting gondor restored wasn’t a big deal … it gets backed up on a daily basis to a usb removable drive.

The laptop is a different story … as Windows doesn’t really have the best backup mechanism I rely on Norton Ghost. It’s always worked OK for me in the past.

It’s in the process of restoring right now … it’s VERY slow going, but I have high hopes.

Luckily I keep the Quicken data files on a LAN drive, so I haven’t lost anything there.


Update … 5:30 pm … oh boy, talk about a long day.

I had to do the restore from the Ghost image 4 times … I couldn’t get it to restore the image and allocate the extra 20gb to my primary Windows partition.

I ended up just restoring the partitions with the same size they were before the failure. I’ll have to get a partition resizing utility in the next week or so so I can resize the partition to allocate the new space.

[tags]Windows, Hard Drives, Backup, Norton Ghost[/tags]

Popularity: 1%

Watchdog

I need to figure out how to setup a watchdog timer on my linux systems.

While Ginny and I were out on a mini-vacation this weekend … gondor, my main web server, went weird on me.

It would respond to pings … but none of the servers would respond. I could telnet into the specific port, and it would connect, but the server itself would not respond.

So we cut our vacation a little short and came home (Gondor is kind of important) … when I looked at the system it wasn’t locked up (which I didn’t expect, because it still responded to some things), but it wasn’t doing anything.

Nothing serious was showing up on syslog, but the system was still hung up.

I ran a bunch of diagnostics, which didn’t indicate any problems… so I’m pretty much at a loss. Diagnostics were run on memory, the main-board, and the hard drives. I should note that one of the hard drives is making an odd whining noise … which indicates to me a potential problem … but the diagnostics didn’t indicate a problem.

Once I can figure out how to get a watchdog running, if the system goes weird on me again, it will at reboot itself. Not a prefect solution, but workable until I can figure out what is going wrong.

[tags]Linux, hardware, diagnostics, Dell, watchdog[/tags]

Popularity: 1%

Another upgrade

This weekend I’m going to attempt another hardware upgrade.

The main mail server for midrange.com (rivendell) is showing it’s age … mind you, it’s still performing fine, but it’s pretty loud and it’s maintenance contract expired last month. A new contract would have costed $300, while a new server only cost $600.

So I got a new Dell PowerEdge SC430 (Pentium D, 3ghz, dual core, 1gb ram, 80gb DASD) to replace the current Dell PowerEdge SC600.

My plan is to backup the drives (mirrored pair) to a USB hard drive and then transplant the drives into the new server.

The only hitch is that the new server primarily runs on SATA, while my current drives are PATA. This I’m solving by getting an add-on ATA/133 card. I’ll set the system to boot off of the drives on the add-on card and use the SATA drive as swap space and a backup drive.

In the past, this kind of hardware upgrade went very smoothly. Mainly because Linux was able to detect the changed hardware and reconfigure itself appropriately. This is why I upgraded rivendell to Fedora Core 4. Redhat 8 would not have been able to deal with the new hardware.

Keep your fingers crossed.


Update 3/25 @ 3pm

Looks like the upgrade was a success!

There were a few minor glitches with the file system table (fstab) … since I put the main drives on the ATA/133 adapter card, they changed from being /dev/hda to /dev/hdc. I had to twiddle with the config to get it to boot properly.

And, in case you’re wondering, I did back it up to the USB drive. That took a bit of doing though, because the USB ports on the old system are ‘full’ speed (10mps). I had to put the USB drive on gondor and do the backup via the network in order to get it done in a reasonable amount of time.

[tags]Linux, Fedora, Dell, Poweredge, Upgrade[/tags]

Popularity: 1%