linux-ext4 - Fwd: Kernel 3.0.0 + ext4 + ceph == ...

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAO47_--yVuodQhMXknd2cEPpd-QkowVPV_D0+u4LM302r6SKFw@mail.gmail.com>
Date:	Sat, 30 Jul 2011 16:53:22 +0200
From:	Christian Brunner <chb@....de>
To:	linux-ext4@...r.kernel.org, ceph-devel@...r.kernel.org
Subject: Fwd: Kernel 3.0.0 + ext4 + ceph == ...

Fyodor and I are struggling to get a fully stable ceph cluster up and running.

When we run an Ceph-Objectstore (OSD) ontop of an ext4 filesystem, we
get fsck errors, when we check the filesystem (see below).

Fyodor is running 3.0.
I am running a RHEL6.1 Kernel (2.6.32-131.6.1.el6.x86_64).

Any help or hints on how to trace the bug would be appreciated.

Thanks,
Christian

2011/7/30 Fyodor Ustinov <ufm@....su>:
> fail. Epic fail.
>
> Absolutely reproducible.
>
> I have ceph cluster with this configuration:
>
> 8 physical servers
> 14 osd servers.
> Each osd server have personal fs.
> 48T total size of ceph cluster.
> 17T used.
>
> Now, step by step:
>
> 1. Stop ceph server osd0
> /etc/init.d/ceph stop
>
> 2. Make fresh fs for osd
> umount /osd.0
> mkfs.ext4 /dev/sdc1
> tune2fs -o journal_data_writeback /dev/sdc1
> mount -a
> # string from /etc/fstab:
> # /dev/sdc1 /osd.0          ext4
>  user_xattr,rw,noexec,nodev,noatime,nodiratime,data=writeback,barrier=0
>    0       2
> ceph mon getmap -o /tmp/monmap
> cosd --mkfs -i 0 --monmap /tmp/monmap
>
> 3. Start ceph server osd0
> /etc/init.d/ceph start
>
> Now, make a big cup of coffee and begin to wait.
>
> After completion of rebalancing do:
> /etc/init.d/ceph stop
> umount /osd.0
> fsck.ext4 -fy /dev/sdc1
>
> and see many-many messages like:
>
> Inode 238551053, i_blocks is 24, should be 32.  Fix? yes
>
> Inode 238551054, i_blocks is 40, should be 32.  Fix? yes
>
> Inode 238551066, i_blocks is 24, should be 32.  Fix? yes
>
> Inode 238944257, i_blocks is 8, should be 16.  Fix? yes
>
> Inode 239206414, i_blocks is 8, should be 16.  Fix? yes
>
> Inode 239206416, i_blocks is 40, should be 32.  Fix? yes
>
> Inode 239206431, i_blocks is 8, should be 16.  Fix? yes
>
> Inode 239206441, i_blocks is 24, should be 32.  Fix? yes
>
> Voila.
>
> P.S. No any message in syslog. No any message in console.
>
> WBR,
>    Fyodor.
>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html