Message-ID: <6bb80c9.11eef.1624156c7fb.Coremail.0121167@fudan.edu.cn>
Date:   Tue, 20 Mar 2018 10:57:09 +0800 (GMT+08:00)
From:   "Jidong Xiao" <0121167@...an.edu.cn>
To:     "Andreas Dilger" <adilger@...ger.ca>, "Ted Ts'o" <tytso@....edu>,
        linux-ext4@...r.kernel.org
Cc:     "Jidong Xiao" <jidong.xiao@...il.com>
Subject: Re: Re: No data blocks at all in my ext4 journal


> -----Original Messages-----
> From:"Jidong Xiao" <jidong.xiao@...il.com>
> Sent Time:2018-03-20 10:46:43 (Tuesday)
> To:"Andreas Dilger" <adilger@...ger.ca>, "Ted Ts'o" <tytso@....edu>
> Cc:"Jidong Xiao" <0121167@...an.edu.cn>, linux-ext4@...r.kernel.org
> Subject:Re: No data blocks at all in my ext4 journal
> 
> 
> 
> 
> On Mon, Mar 19, 2018 at 8:18 PM, Andreas Dilger <adilger@...ger.ca> wrote:
> >
> > On Mar 19, 2018, at 6:19 PM, Jidong Xiao <0121167@...an.edu.cn> wrote:
> > >> -----Original Messages-----
> > >> From: "Andreas Dilger" <adilger@...ger.ca>
> > >> Sent Time: 2018-03-20 07:46:23 (Tuesday)
> > >> To: "Jidong Xiao" <0121167@...an.edu.cn>
> > >> Cc: linux-ext4@...r.kernel.org
> > >> Subject: Re: No data blocks at all in my ext4 journal
> > >>
> > >> On Mar 19, 2018, at 5:31 PM, Jidong Xiao <0121167@...an.edu.cn> wrote:
> > >>>
> > >>> Hi,
> > >>>
> > >>> When I use the dd command to examine the journal, it turns out every block starts with the jbd2 magic number, meaning they are either a superblock, a descriptor block, or a commit block, or a revoke block, but there is no data block.
> > >>>
> > >>> I manually and randomly tried like 50 blocks after that superblock, but I haven't seen any data blocks.
> > >>>
> > >>> The kernel version I am using is, 4.14. Is this a serious bug or am I doing something wrong? Anyone has any suggestions, or tell me how to find the data blocks in my journal, or are they actually missing from the journal? By data blocks, I mean metadata, as my system is using the default mode, i.e., the ordered journal mode, only metadata will be logged.
> > >>>
> > >>> Thanks. Let me know if more information is needed.
> > >>
> > >> You can decode the journal structure using debugfs, and it will show you the
> > >> blocks more easily:
> > >>
> > >> # debugfs -c -R "logdump -a" /dev/vg_mookie/lv_root
> > >> debugfs 1.42.13.wc5 (15-Apr-2016)
> > >> Journal starts at block 13970, transaction 1103250
> > >> Found expected sequence 1103250, type 1 (descriptor block) at block 13970
> > >> Dumping descriptor block, sequence 1103250, at block 13970:
> > >>  FS block 12058640 logged at journal block 13971 (flags 0x0)
> > >>  FS block 3 logged at journal block 13972 (flags 0x2)
> > >>  FS block 12059015 logged at journal block 13973 (flags 0x2)
> > >>  FS block 12058703 logged at journal block 13974 (flags 0x2)
> > >>  FS block 12066941 logged at journal block 13975 (flags 0x2)
> > >>  FS block 12058641 logged at journal block 13976 (flags 0x2)
> > >>  FS block 12059178 logged at journal block 13977 (flags 0x2)
> > >>  FS block 0 logged at journal block 13978 (flags 0xa)
> > >> Found expected sequence 1103250, type 2 (commit block) at block 13979
> > >>
> > >> Cheers, Andreas
> > >>
> > >>
> > >
> > > Hi, Andreas,
> > >
> > > Thanks for your kindly response.
> > >
> > > I tried the TSK tool and debugfs, the results are the same, there is no data/metadata block in my journal. Every block is either a superblock, or a descriptor block, or a commit block. For example, using debugfs with the command you provided, I got this:
> > >
> > > [lab7]$ sudo debugfs -c -R "logdump -a" /dev/sda3
> > > Journal starts at block 7757, transaction 8305398
> > > Found expected sequence 8305398, type 1 (descriptor block) at block 7757
> > > Dumping descriptor block, sequence 8305398, at block 7757:
> > >  FS block 36700177 logged at journal block 7758 (flags 0x0)
> > >  FS block 9 logged at journal block 7759 (flags 0x2)
> > >  FS block 36700889 logged at journal block 7760 (flags 0x2)
> > >  FS block 36700501 logged at journal block 7761 (flags 0x2)
> > >  FS block 36709095 logged at journal block 7762 (flags 0x2)
> > >  FS block 36700161 logged at journal block 7763 (flags 0x2)
> > >  FS block 36700907 logged at journal block 7764 (flags 0x2)
> > >  FS block 0 logged at journal block 7765 (flags 0xa)
> > > Found expected sequence 8305398, type 2 (commit block) at block 7766
> > > [REMOVED]
> > >
> > > And my journal superblock is at block 52461568, so journal block 7760 is file system block (52461568+7760)=52469328.
> >
> > This is not totally accurate.  While the journal superblock is at block 52461568,
> > it is not always true that the logical journal block 7760 is at 52461568+7760.
> > Instead you should use "stat" to determine the journal inode number (this will
> > be "8" for almost all filesystems), and then "stat <8>" to dump the blocks
> > allocated to the journal inode, like:
> >
> > debugfs:  stat <8>
> > Inode: 8   Type: regular    Mode:  0600   Flags: 0x80000
> > Generation: 0    Version: 0x00000000:00000000
> > User:     0   Group:     0   Size: 134217728
> > File ACL: 0    Directory ACL: 0
> > Links: 1   Blockcount: 262144
> > Fragment:  Address: 0    Number: 0    Size: 0
> >  ctime: 0x4e1b45e2:00000000 -- Mon Jul 11 12:50:10 2011
> >  atime: 0x4e1b45e2:00000000 -- Mon Jul 11 12:50:10 2011
> >  mtime: 0x4e1b45e2:00000000 -- Mon Jul 11 12:50:10 2011
> > crtime: 0x4e1b45e2:00000000 -- Mon Jul 11 12:50:10 2011
> > Size of extra inode fields: 28
> > EXTENTS:
> > (0-32766):6324224-6356990, (32767):6356991
> >
> > In my case, it shows that all 32768 blocks are physically contiguous on disk,
> > so direct block addition will work, but that may not be the case if the journal
> > was (re)created after the filesystem was formatted for some reason, or if it
> > is so large that it crosses a group descriptor boundary.
> >
> > > Now I use dd command to examine 52469328, and I see this:
> > >
> > > [lab7]$ sudo dd if=/dev/sda3 bs=4096 skip=52469328 | xxd | more
> > > 00000000: c03b 3998 0000 0002 007e a8cd 0000 0000  .;9......~......
> > > 00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> > > 00000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> > > 00000030: 0000 0000 5aaf 8011 16ca 442c 0000 0000  ....Z.....D,....
> > > 00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> > >
> > > As you can see, debugfs says this is a journal metadata block records a file system block 36700889, yet the dd command says this is a journal commit block. I tried many journal blocks, and I don't see any metadata/data blocks in my journal. For example, these are the neighbor journal blocks:
> >
> > I did a similar comparison in my filesystem and things look as expected.
> > In my above logdump, block #3 is at journal block 13972, and the journal
> > superblock is at block 6324224, so it is block 6338196 on the disk:
> >
> > # dd if=/dev/vg_mookie/lv_root count=1 bs=4096 skip=3 | od -Ax -tx4 | head
> > 000000 00800000 00800010 00800020 0751583d
> > 000010 00040694 00000000 00000000 112c0000
> > 000020 00800001 00800011 00800220 1c581692
> > 000030 00040056 00000000 00000000 aae40000
> > 000040 00800002 00800012 00800420 191b214e <==== this is modified in journal
> > 000050 000400a5 00000000 00000000 31771575
> > 000060 00800003 00800013 00800620 20006380
> > 000070 00050000 00000000 00000000 fa6f2000
> > 000080 00800004 00800014 00800820 20006d83
> > 000090 00050000 00000000 00000000 7fff2000
> >
> > # dd if=/dev/vg_mookie/lv_root count=1 bs=4096 skip=6338196 | od -Ax -tx4 | head
> > 000000 00800000 00800010 00800020 0751583d
> > 000010 00040694 00000000 00000000 112c0000
> > 000020 00800001 00800011 00800220 1c581692
> > 000030 00040056 00000000 00000000 aae40000
> > 000040 00800002 00800012 00800420 191b214f <==== is 0x214e in filesystem
> > 000050 000400a5 00000000 00000000 cd261575
> > 000060 00800003 00800013 00800620 200063a9
> > 000070 00050000 00000000 00000000 aeb72000
> > 000080 00800004 00800014 00800820 20006d83
> > 000090 00050000 00000000 00000000 7fff2000
> >
> > The two blocks are nearly identical, except for byte 0x4f, which has changed
> > value from 0x4e to 0x4f in the journal.  According to "dumpe2fs -x", block #3
> > is a group descriptor block, and this block describes the groups 256-377. The
> > word in question (0x214e = 8526) is the group 258 free blocks count:
> >
> > Group 258: (Blocks 0x810000-0x817fff) [ITABLE_ZEROED]
> >   Checksum 0x3177, unused inodes 5493
> >   Block bitmap at 0x800002 (bg #256 + 2), Inode bitmap at 0x800012 (bg #256 + 18)
> >   Inode table at 0x800420-0x80061f (bg #256 + 1056)
> >   8526 free blocks, 6427 free inodes, 165 directories, 5493 unused inodes
> >
> > It looks like some block was allocated or freed in this block group, but there
> > are other journal blocks that must also be changing this value.
> >
> > Cheers, Andreas
> >
> >
> >
> >
> >
> 
> Hi, Andreas and Ted,
> 
> Here is the result of stat <8>.
> 
> [lab7]$ sudo debugfs -R "stat <8>" /dev/sda3
> [sudo] password for jxiao:
> debugfs 1.42.12 (29-Aug-2014)
> Inode: 8   Type: regular    Mode:  0600   Flags: 0x80000
> Generation: 0    Version: 0x00000000:00000000
> User:     0   Group:     0   Size: 134217728
> File ACL: 0    Directory ACL: 0
> Links: 1   Blockcount: 262144
> Fragment:  Address: 0    Number: 0    Size: 0
> ctime: 0x56b02676:00000000 -- Mon Feb  1 20:45:58 2016
> atime: 0x56b02676:00000000 -- Mon Feb  1 20:45:58 2016
> mtime: 0x56b02676:00000000 -- Mon Feb  1 20:45:58 2016
> crtime: 0x56b02676:00000000 -- Mon Feb  1 20:45:58 2016
> Size of extra inode fields: 28
> EXTENTS:
> (0-32766):52461568-52494334, (32767):52494335
> 
> So we can see my journal indeed starts at block 52461568. And my journal size is the default 128MB, block size is the default 4K.
> 
> So I guess the formula I was using was correct. The logical journal block 7760 should be at 52461568+7760=52469328; but then I didn't see what Andreas saw in his system.
> 
> Also, it's easy to confirm that block 52461568 is the starting block of my journal, based on the magic number and type 4.
> 
> [lab7]$ sudo dd if=/dev/sda3 bs=4096 skip=52461568 | xxd | more
> 00000000: c03b 3998 0000 0004 0000 0000 0000 1000  .;9.............
> 00000010: 0000 8000 0000 0001 007e bfa8 0000 3e4f  .........~....>O
> 00000020: 0000 0000 0000 0000 0000 0001 0000 0000  ................
> 00000030: fe2c 4ddf 17a7 4c19 b0cd 1373 6f00 ea65  .,M...L....so..e
> 00000040: 0000 0001 0000 0000 0000 0000 0000 0000  ................
> 
> And Ted, I used the command you told me:
> 
> [lab]# debugfs -R "logdump -ac" /dev/sda3 > /var/journal3.txt
> debugfs 1.42.12 (29-Aug-2014)
> 
> And the output file journal3.txt is here, probably too big to paste or attach in this email.
> 
> http://cs.boisestate.edu/~jxiao/cs554/journal3.txt
> 
> As you can see, there really is only blocks with that signature, 98393bc0.
> 
> -Jidong

Google simply doesn't allow me to send email to the mailing list, even if in my previous several attempts I didn't use any html format or include any links. All the delivery failed, so I have to switch between different email accounts. Sorry about that; I am replying this email again, just to make sure it goes to the mailing list.

Powered by blists - more mailing lists