| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Fri, 26 Dec 2008 16:42:26 +0100 From: Andreas Sundstrom <sunkan@...pa.cx> To: linux-kernel@...r.kernel.org Subject: Re: 2.6.28 ext4, xen and lvm volume becomes ro after snapshot Theodore Tso wrote: > On Fri, Dec 26, 2008 at 12:06:19PM +0100, Andreas Sundstrom wrote: > >> I use a little script that backup this xen VM from xen dom0. >> >> Here is the interesting part of my script: >> /usr/sbin/xm sysrq xenfw1 s >> /usr/sbin/xm pause xenfw1 >> sync >> /sbin/lvm lvcreate --snapshot --permission rw --size 1G --name xenfw1_s >> /dev/3w250g/xenfw1 > /dev/null >> /usr/sbin/xm unpause xenfw1 >> /bin/mount -o ro,noatime /dev/3w250g/xenfw1_s /mnt/snapshots/xenfw1 || >> /bin/rmdir /mnt/snapshots/xenfw1 >> # Here's where the actual backups take place >> /bin/umount /mnt/snapshots/xenfw1 2> /dev/null >> /sbin/lvm lvremove --force /dev/3w250g/xenfw1_s > /dev/null >> > > OK, so this is being run on Xen Dom0, which means your doing the > snapshot from Host OS, while the guest OS is suspended, correct? > Correct > >> After the "lvcreate --snapshot" if I check within the xen VM (with cat >> /proc/mounts) I can see that / changed from rw to ro: >> Before snapshot: >> /dev/root / ext4 rw,noatime,barrier=1,noextents,data=ordered 0 0 >> After snapshot: >> /dev/root / ext4 ro,noatime,barrier=1,noextents,data=ordered 0 0 >> > > But this was done from the guest OS --- what is /dev/root in the guest > OS? Am I right in assuming it is the device /dev/3w250g/xenfw1? > Stupid question --- if this is the case, why are you taking the > snapshot from the Host OS? It won't be a consistent snapshot, given > that it was mounted in the Guest OS, and all you've done in the Guest > OS is to ask it to sync the filesystem. Given that there's no pause > between sysrq s and the pause, the write operations probably haven't > even been completed before the pause takes place, so the snapshot > probably has a chance of in pretty bad shape as it is. > You're probably correct, I already have a "sleep 1" that I missed to include in my e-mail though. Let's not analyze the backup method so much, it's not a very important machine and I mostly want config files to be backed up and they are pretty consistent at all times anyway. >> It was not possible to do "mount -o remount,rw /": >> mount -o remount,rw / >> > > There are multiple potential reasons for this, but I'm guessing this > was caused by an aborted journal, probably caused by an I/O error when > trying to write to the journal. This may very well be related to > pausing guest OS in the middle of a write operations that would have > been caused by the sysrq s, but that's a guess; we need more > information about exactly what is going on. > You are correct about the aborted journal but it's not caused by anythingelse than the lvm snapshot. > If you could reproduce this and send back the messages from dmesg in > the guest OS, that would be quite helpful. You might also try adding > a sleep 2 between the sysrq s and the pause xenfw1 commands, and see > if the problem goes away. That's just a stab in the dark, of course, > but it's a simple enough thing to try. > Ok, I used another Xen VM for testing now. While I was testing now I noticed that I even could take the snapshot while the system is not started and the error would still occur. No problems with ext3 with the same kernel that I've seen yet. Here's one way to reproduce: # lvs | grep xenfw2 xenfw2 3w250g -wi-a- 1.00G # Notice the last "-" which is not "o" xenfw2_swap 3w250g -wi-a- 196.00M # lvcreate --snapshot --permission rw --size 1G --name xenfw2_s /dev/3w250g/xenfw2 Logical volume "xenfw2_s" created # lvs | grep xenfw2 xenfw2 3w250g owi-a- 1.00G xenfw2_s 3w250g swi-a- 1.00G xenfw2 0.00 xenfw2_swap 3w250g -wi-a- 196.00M # xm create xenfw2 -c # To start the Xen VM with console attached Here's what I think is the relevant part of the kernel output [ 0.317797] EXT4-fs warning (device xvda1): ext4_fill_super: extents feature not enabled on this filesystem, use tune2fs. [ 0.317822] [ 0.326805] EXT4-fs: barriers enabled [ 0.336488] kjournald2 starting. Commit interval 5 seconds [ 0.336514] EXT4-fs: delayed allocation enabled [ 0.336610] EXT4-fs: mballoc enabled [ 0.336622] EXT4-fs: mounted filesystem with ordered data mode. [ 0.336651] VFS: Mounted root (ext4 filesystem) readonly. [ 0.336880] Freeing unused kernel memory: 280k freed [ 0.337936] Write protecting the kernel text: 2624k [ 0.338358] Write protecting the kernel read-only data: 1028k [snip] [ 7.636849] blkfront: xvda1: write barrier op failed [ 7.636867] blkfront: xvda1: barriers disabled [ 7.636877] end_request: I/O error, dev xvda1, sector 4512 [ 7.636892] end_request: I/O error, dev xvda1, sector 4512 [ 7.636933] Aborting journal on device xvda1:8. [ 7.637602] ext4_abort called. [ 7.637616] EXT4-fs error (device xvda1): ext4_journal_start_sb: Detected aborted journal [ 7.637633] Remounting filesystem read-only [ 7.652152] EXT4-fs error (device xvda1) in ext4_reserve_inode_write: Journal has aborted Let me know if I can help with more information /Andreas -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists