LVM incorrectly reported missing after power failure

Posted by mensi on Server Fault See other posts from Server Fault or by mensi
Published on 2012-06-29T15:06:24Z Indexed on 2012/06/29 15:17 UTC
Read the original article Hit count: 129

Filed under:

lvm

|

mdadm

We have had a major power failure in the data-center. We are using a set of servers for our storage needs. The main server has several pairs of disks mirrored with mdadm. The resulting /dev/mdX are LVM physical volumes and belong to a big volume-group with all our data.

After the powerloss, we had the problem that one of the mdadm devices was not auto-detected due to a missing entry in mdadm.conf. As a consequence, the volumegroup had inactive logical volumes due to the missing PV.

We were able to fix the mdadm config and reboot. pvscan shows all expected PVs but one LV still does not come up. vgdisplay shows:

[...]
Cur PV:  3
Act PV:  2
[...]

Neither vgscan nor pvscan show any missing devices.

What went wrong? How can we force LVM to activate all PVs?

© Server Fault or respective owner

Related posts about lvm

Unix LVM: how to resize root lvm

as seen on Server Fault - Search for 'Server Fault'
I took over a virtual server at work after a co-worker left. He, however, setup the server incorrectly at multiple stages and im cleaning them up as I run into them... Currently I realized that the file system is broken in half onto 2 logical volumes both at 50gb. One is mounted as the root directory… >>> More
RAID5 over LVM on Ubuntu Server 12.04.3

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I'm trying to create a RAID5 software array using LVM. I use VirtualBox as I'm only learning how LVM works. So I've created 4 virtual SCSI drives and then did the following: pvcreate /dev/sd[b-e] vgcreate /dev/sd[b-e] raid5_vg lvcreate --type raid5 -i 3 -L 1G -n raid_lv raid5_vg However, I get… >>> More
Preseeding Ubuntu partman recipe using LVM and RAID

as seen on Server Fault - Search for 'Server Fault'
I'm trying to preseed Ubuntu 12.04 server installation and created a recipe that would create RAID 1 on 2 drives and then partition that using LVM. Unfortunately partman complains when creating LVM volumes saying there no partitions in recipe that could be used with LVM (in console it complains about… >>> More
Restore data from overwritten LVM

as seen on Server Fault - Search for 'Server Fault'
I lost all of my data (8 TB) which I collected over the past few years yesterday because I made some seriuos mistakes during the remounting of my LVM. I run a XenServer5.6 installation with additional 4 harddisks for data storage. An LVM over those 4 HDDs was used to store all of my data. Yesterday… >>> More
how to multi-boot/upgrade linux from LVM-based partition

as seen on Super User - Search for 'Super User'
i currently have FC3 linux which installed itself on the hard disk using LVM partitioning, so it is basically all one big partition. i would like to try some other distributions and upgrade to something newer, but don't want to lose my current capabilities and data files, and i know nothing or less… >>> More

Related posts about mdadm

apt-get update mdadm scary warnings

as seen on Server Fault - Search for 'Server Fault'
Just ran an apt-get update on one of my dedicated servers to be left with a relatively scary warning: Processing triggers for initramfs-tools ... update-initramfs: Generating /boot/initrd.img-2.6.26-2-686-bigmem W: mdadm: the array /dev/md/1 with UUID c622dd79:496607cf:c230666b:5103eba0 W: mdadm:… >>> More
mdadm+zfs vs mdadm+lvm

as seen on Server Fault - Search for 'Server Fault'
This may be a naive question since I'm new to this and I cannot find any results about mdadm+zfs, but after some testing it seems it might work: The use case is a server with RAID6 for some data that is backed-up somewhat infrequently. I think I'm well served by any of ZFS or RAID6. Platform is Linux… >>> More
Upon reboot, Linux software raid fails to include one device of a RAID1 array

as seen on Super User - Search for 'Super User'
One of my four Linux software raid arrays drops one of its two devices when I reboot my system. The other three arrays work fine. I am running RAID1 on kernel version 2.6.32-5-amd64 (Debian Squeeze). Every time I reboot, /dev/md2 comes up with only one device. I can manually add the device by saying… >>> More
Linux software raid fails to include one device for one RAID1 array

as seen on Server Fault - Search for 'Server Fault'
One of my four Linux software raid arrays drops one of its two devices when I reboot my system. The other three arrays work fine. I am running RAID1 on kernel version 2.6.32-5-amd64. Every time I reboot, /dev/md2 comes up with only one device. I can manually add the device by saying $ sudo mdadm /dev/md2… >>> More
What tells initramfs or the Ubuntu Server boot process how to assemble RAID arrays?

as seen on Server Fault - Search for 'Server Fault'
The simple question: how does initramfs know how to assemble mdadm RAID arrays at startup? My problem: I boot my server and get: Gave up waiting for root device. ALERT! /dev/disk/by-uuid/[UUID] does not exist. Dropping to a shell! This happens because /dev/md0 (which is /boot, RAID 1) and /dev/md1… >>> More