« The Boss? | Main | Pulling Rank »

August 15, 2004

Bad blocks (mirrored disk)

What a long day... the main problem was some bad blocks in a filesystem.

No big deal. Run fsck. Fsck complains that there are five bad blocks.

Hmmm. This is a RAID filesystem. It contains two 18Gb disks mirrored. A bad block shouldn't happen (after all there are two copies of the data).

Checked the partitions using format (this is a Solaris system). Interesting, no over-lapping partitions in use (the biggest cause of bad blocks on mirrored disks). Check the alternate superblocks using newfs -N (try doing that without the -N and you could end up back on the unemployment queue).

Hmmm. There is another similar filesystem, also raid with 2 x 18Gb and it is fine. BUT according to df -k it is smaller than our dodgy filesystem. Check the partition maps again. There are the same. Most of the disk is allocated to the data partition with a small slice for a SDS metadisk. Why would they be different sizes?

I have a sneaking suspicion that I know. Let me grab a vanilla 18Gb disk and shove it in the system (thank goodness for hot swap drives and spare slots in the drive tray). Create only **one** partition and newfs it. When I mount it, it is the same size as the dodgy partition.

Ah-ha. In the dim distant past, someone created **one** partition and newfs'd it. Then they changed the partition map and gave the small slice to SDS (which uses it as a raw partition). They did not newfs the changed partition. End result is that we have a mounted filesystem which is **bigger** than the underlying partition. Naughty, naughty.

That explains why the five bad blocks, which happens to be the right size for two copies of a metadatabase (i.e. created with the -c 2 option) are outside the partition. I was curious about that and was wondering if I had confused some maths somewhere.

Fix: Copy all the data from the dodgy partition to the spare disk, newfs the dodgy one (which is now the same size as the good one) and the copy all the data back again. Phew.

Posted by Ozguru at August 15, 2004 06:00 AM