Crashing Xen

Right about the time I finished migrating everything over to my Xen-running Linux box things fell apart. It started crashing in one form or another every night when the backups ran. Sometimes, the whole machine rebooted. Other times, one of the domains (usually the one being backe up) would crash and burn and remain hung until I destroyed it, manually removed the snapshot against its partition, and then restarted it. I investigated a little and found some evidence that Xen had some Issues with some motherboard chipsets when the machine was under load (as when it’s running the backups, for example). Curious, though, since I ran backups for an entire day before the last migration without a problem. Sadly, this happened as I ran out of energy for doing home sysadmin work, so my response after brief investigation was: migrate everything important back to the still-running “normal” Linux box and shut the Xen machine off.

Xen 3.0.3 was supposed to be out at he end of July and apparently will contain a fix for the bug that might have been causing my crashes. It might also have been LVM2/snapshot-related — I never relaly got the warm fuzzy feeling about LVM2 and snapshots after I had some trouble getting it working and it was only reported to work in very recent kernel revisions. I can’t believe the LVM folks rewrote everything (LVM -> LVM2) but broke snapshots for several releases.

Anyway, my Xen experiment has come to a close for the time being.