lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <20090601111755.4F55.A69D9226@jp.fujitsu.com>
Date:	Mon,  1 Jun 2009 11:23:55 +0900 (JST)
From:	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
To:	Hisashi Hifumi <hifumi.hisashi@....ntt.co.jp>
Cc:	kosaki.motohiro@...fujitsu.com,
	Andrew Morton <akpm@...ux-foundation.org>,
	Wu Fengguang <fengguang.wu@...el.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>
Subject: Re: [PATCH] readahead:add blk_run_backing_dev

> 
> At 11:23 09/05/28, KOSAKI Motohiro wrote:
> >> Hi Andrew.
> >> Please merge following patch.
> >> Thanks.
> >> 
> >> ---
> >> 
> >> I added blk_run_backing_dev on page_cache_async_readahead
> >> so readahead I/O is unpluged to improve throughput on 
> >> especially RAID environment. 
> >> 
> >> Following is the test result with dd.
> >> 
> >> #dd if=testdir/testfile of=/dev/null bs=16384
> >> 
> >> -2.6.30-rc6
> >> 1048576+0 records in
> >> 1048576+0 records out
> >> 17179869184 bytes (17 GB) copied, 224.182 seconds, 76.6 MB/s
> >> 
> >> -2.6.30-rc6-patched
> >> 1048576+0 records in
> >> 1048576+0 records out
> >> 17179869184 bytes (17 GB) copied, 206.465 seconds, 83.2 MB/s
> >> 
> >> My testing environment is as follows:
> >> Hardware: HP DL580 
> >> CPU:Xeon 3.2GHz *4 HT enabled
> >> Memory:8GB
> >> Storage: Dothill SANNet2 FC (7Disks RAID-0 Array)
> >> 
> >> The normal case is, if page N become uptodate at time T(N), then
> >> T(N) <= T(N+1) holds. With RAID (and NFS to some degree), there 
> >> is no strict ordering, the data arrival time depends on
> >> runtime status of individual disks, which breaks that formula. So
> >> in do_generic_file_read(), just after submitting the async readahead IO
> >> request, the current page may well be uptodate, so the page won't be locked,
> >> and the block device won't be implicitly unplugged:
> >
> >Please attach blktrace analysis ;)
> 
> Hi, Motohiro.
> 
> I've got blktrace output for both with and without the patch, 
> but I just did not clarify the reason for throuput improvement
> from this result.
> 
> I do not notice any difference except around unplug behavior by dd.
> Comments?

Please makes analysis yourself.
following are summarized log of IO completion.

it doesn't contain IO completion inversion.
Why do you think it is RAID specific behavior?

  8,0    3   177784    50.001437357     0  C   R 8717567 + 512 [0]
  8,0    3   177795    50.003815034     0  C   R 8718079 + 512 [0]
  8,0    3   177806    50.007151480     0  C   R 8718591 + 512 [0]
  8,0    3   177817    50.009885672     0  C   R 8719103 + 512 [0]
  8,0    3   177828    50.013880081     0  C   R 8719615 + 512 [0]
  8,0    3   177839    50.017545281     0  C   R 8720127 + 512 [0]
  8,0    3   177850    50.020674534     0  C   R 8720639 + 512 [0]
  8,0    3   177861    50.024269136     0  C   R 8721151 + 512 [0]
  8,0    3   177872    50.026966145     0  C   R 8721663 + 512 [0]
  8,0    3   177883    50.030406105     0  C   R 8722175 + 512 [0]
  8,0    3   177894    50.032916699     0  C   R 8722687 + 512 [0]
  8,0    3   177905    50.036715133     0  C   R 8723199 + 512 [0]
  8,0    3   177916    50.040521790     0  C   R 8723711 + 512 [0]
  8,0    3   177932    50.044383267     0  C   R 8724223 + 416 [0]
  8,0    3   177955    50.050313976     0  C   R 8724991 + 512 [0]
  8,0    3   177966    50.053380250     0  C   R 8725503 + 512 [0]
  8,0    3   177977    50.056970395     0  C   R 8726015 + 512 [0]
  8,0    3   177988    50.060326743     0  C   R 8726527 + 512 [0]
  8,0    3   177999    50.063922341     0  C   R 8727039 + 512 [0]




> 
> -2.6.30-rc6
>   8,0    3   177784    50.001437357     0  C   R 8717567 + 512 [0]
>   8,0    3   177785    50.001635405  4148  A   R 8718079 + 256 <- (8,1) 8718016
>   8,0    3   177786    50.001635675  4148  Q   R 8718079 + 256 [dd]
>   8,0    3   177787    50.001637517  4148  G   R 8718079 + 256 [dd]
>   8,0    3   177788    50.001638954  4148  P   N [dd]
>   8,0    3   177789    50.001639290  4148  I   R 8718079 + 256 [dd]
>   8,0    3   177790    50.001765339  4148  A   R 8718335 + 256 <- (8,1) 8718272
>   8,0    3   177791    50.001765699  4148  Q   R 8718335 + 256 [dd]
>   8,0    3   177792    50.001766971  4148  M   R 8718335 + 256 [dd]
>   8,0    3   177793    50.001768243  4148  U   N [dd] 1
>   8,0    3   177794    50.001769464  4148  D   R 8718079 + 512 [dd]
>   8,0    3   177795    50.003815034     0  C   R 8718079 + 512 [0]
>   8,0    3   177796    50.004008636  4148  A   R 8718591 + 256 <- (8,1) 8718528
>   8,0    3   177797    50.004008951  4148  Q   R 8718591 + 256 [dd]
>   8,0    3   177798    50.004010787  4148  G   R 8718591 + 256 [dd]
>   8,0    3   177799    50.004012089  4148  P   N [dd]
>   8,0    3   177800    50.004012641  4148  I   R 8718591 + 256 [dd]
>   8,0    3   177801    50.004139944  4148  A   R 8718847 + 256 <- (8,1) 8718784
>   8,0    3   177802    50.004140298  4148  Q   R 8718847 + 256 [dd]
>   8,0    3   177803    50.004141393  4148  M   R 8718847 + 256 [dd]
>   8,0    3   177804    50.004142815  4148  U   N [dd] 1
>   8,0    3   177805    50.004144003  4148  D   R 8718591 + 512 [dd]
>   8,0    3   177806    50.007151480     0  C   R 8718591 + 512 [0]
>   8,0    3   177807    50.007344467  4148  A   R 8719103 + 256 <- (8,1) 8719040
>   8,0    3   177808    50.007344779  4148  Q   R 8719103 + 256 [dd]
>   8,0    3   177809    50.007346636  4148  G   R 8719103 + 256 [dd]
>   8,0    3   177810    50.007347821  4148  P   N [dd]
>   8,0    3   177811    50.007348346  4148  I   R 8719103 + 256 [dd]
>   8,0    3   177812    50.007480827  4148  A   R 8719359 + 256 <- (8,1) 8719296
>   8,0    3   177813    50.007481187  4148  Q   R 8719359 + 256 [dd]
>   8,0    3   177814    50.007482669  4148  M   R 8719359 + 256 [dd]
>   8,0    3   177815    50.007483965  4148  U   N [dd] 1
>   8,0    3   177816    50.007485171  4148  D   R 8719103 + 512 [dd]
>   8,0    3   177817    50.009885672     0  C   R 8719103 + 512 [0]
>   8,0    3   177818    50.010077696  4148  A   R 8719615 + 256 <- (8,1) 8719552
>   8,0    3   177819    50.010078008  4148  Q   R 8719615 + 256 [dd]
>   8,0    3   177820    50.010079841  4148  G   R 8719615 + 256 [dd]
>   8,0    3   177821    50.010081227  4148  P   N [dd]
>   8,0    3   177822    50.010081560  4148  I   R 8719615 + 256 [dd]
>   8,0    3   177823    50.010208686  4148  A   R 8719871 + 256 <- (8,1) 8719808
>   8,0    3   177824    50.010209046  4148  Q   R 8719871 + 256 [dd]
>   8,0    3   177825    50.010210366  4148  M   R 8719871 + 256 [dd]
>   8,0    3   177826    50.010211686  4148  U   N [dd] 1
>   8,0    3   177827    50.010212916  4148  D   R 8719615 + 512 [dd]
>   8,0    3   177828    50.013880081     0  C   R 8719615 + 512 [0]
>   8,0    3   177829    50.014071235  4148  A   R 8720127 + 256 <- (8,1) 8720064
>   8,0    3   177830    50.014071544  4148  Q   R 8720127 + 256 [dd]
>   8,0    3   177831    50.014073332  4148  G   R 8720127 + 256 [dd]
>   8,0    3   177832    50.014074517  4148  P   N [dd]
>   8,0    3   177833    50.014075084  4148  I   R 8720127 + 256 [dd]
>   8,0    3   177834    50.014201763  4148  A   R 8720383 + 256 <- (8,1) 8720320
>   8,0    3   177835    50.014202123  4148  Q   R 8720383 + 256 [dd]
>   8,0    3   177836    50.014203608  4148  M   R 8720383 + 256 [dd]
>   8,0    3   177837    50.014204889  4148  U   N [dd] 1
>   8,0    3   177838    50.014206095  4148  D   R 8720127 + 512 [dd]
>   8,0    3   177839    50.017545281     0  C   R 8720127 + 512 [0]
>   8,0    3   177840    50.017741679  4148  A   R 8720639 + 256 <- (8,1) 8720576
>   8,0    3   177841    50.017742006  4148  Q   R 8720639 + 256 [dd]
>   8,0    3   177842    50.017743848  4148  G   R 8720639 + 256 [dd]
>   8,0    3   177843    50.017745318  4148  P   N [dd]
>   8,0    3   177844    50.017745672  4148  I   R 8720639 + 256 [dd]
>   8,0    3   177845    50.017876956  4148  A   R 8720895 + 256 <- (8,1) 8720832
>   8,0    3   177846    50.017877286  4148  Q   R 8720895 + 256 [dd]
>   8,0    3   177847    50.017878615  4148  M   R 8720895 + 256 [dd]
>   8,0    3   177848    50.017880082  4148  U   N [dd] 1
>   8,0    3   177849    50.017881339  4148  D   R 8720639 + 512 [dd]
>   8,0    3   177850    50.020674534     0  C   R 8720639 + 512 [0]
>   8,0    3   177851    50.020864689  4148  A   R 8721151 + 256 <- (8,1) 8721088
>   8,0    3   177852    50.020865007  4148  Q   R 8721151 + 256 [dd]
>   8,0    3   177853    50.020866900  4148  G   R 8721151 + 256 [dd]
>   8,0    3   177854    50.020868283  4148  P   N [dd]
>   8,0    3   177855    50.020868628  4148  I   R 8721151 + 256 [dd]
>   8,0    3   177856    50.020997302  4148  A   R 8721407 + 256 <- (8,1) 8721344
>   8,0    3   177857    50.020997662  4148  Q   R 8721407 + 256 [dd]
>   8,0    3   177858    50.020998976  4148  M   R 8721407 + 256 [dd]
>   8,0    3   177859    50.021000305  4148  U   N [dd] 1
>   8,0    3   177860    50.021001520  4148  D   R 8721151 + 512 [dd]
>   8,0    3   177861    50.024269136     0  C   R 8721151 + 512 [0]
>   8,0    3   177862    50.024460931  4148  A   R 8721663 + 256 <- (8,1) 8721600
>   8,0    3   177863    50.024461337  4148  Q   R 8721663 + 256 [dd]
>   8,0    3   177864    50.024463175  4148  G   R 8721663 + 256 [dd]
>   8,0    3   177865    50.024464537  4148  P   N [dd]
>   8,0    3   177866    50.024464871  4148  I   R 8721663 + 256 [dd]
>   8,0    3   177867    50.024597943  4148  A   R 8721919 + 256 <- (8,1) 8721856
>   8,0    3   177868    50.024598213  4148  Q   R 8721919 + 256 [dd]
>   8,0    3   177869    50.024599323  4148  M   R 8721919 + 256 [dd]
>   8,0    3   177870    50.024600751  4148  U   N [dd] 1
>   8,0    3   177871    50.024602104  4148  D   R 8721663 + 512 [dd]
>   8,0    3   177872    50.026966145     0  C   R 8721663 + 512 [0]
>   8,0    3   177873    50.027157245  4148  A   R 8722175 + 256 <- (8,1) 8722112
>   8,0    3   177874    50.027157563  4148  Q   R 8722175 + 256 [dd]
>   8,0    3   177875    50.027159351  4148  G   R 8722175 + 256 [dd]
>   8,0    3   177876    50.027160731  4148  P   N [dd]
>   8,0    3   177877    50.027161064  4148  I   R 8722175 + 256 [dd]
>   8,0    3   177878    50.027288745  4148  A   R 8722431 + 256 <- (8,1) 8722368
>   8,0    3   177879    50.027289105  4148  Q   R 8722431 + 256 [dd]
>   8,0    3   177880    50.027290206  4148  M   R 8722431 + 256 [dd]
>   8,0    3   177881    50.027291697  4148  U   N [dd] 1
>   8,0    3   177882    50.027293119  4148  D   R 8722175 + 512 [dd]
>   8,0    3   177883    50.030406105     0  C   R 8722175 + 512 [0]
>   8,0    3   177884    50.030600613  4148  A   R 8722687 + 256 <- (8,1) 8722624
>   8,0    3   177885    50.030601199  4148  Q   R 8722687 + 256 [dd]
>   8,0    3   177886    50.030603269  4148  G   R 8722687 + 256 [dd]
>   8,0    3   177887    50.030604463  4148  P   N [dd]
>   8,0    3   177888    50.030604799  4148  I   R 8722687 + 256 [dd]
>   8,0    3   177889    50.030731757  4148  A   R 8722943 + 256 <- (8,1) 8722880
>   8,0    3   177890    50.030732117  4148  Q   R 8722943 + 256 [dd]
>   8,0    3   177891    50.030733397  4148  M   R 8722943 + 256 [dd]
>   8,0    3   177892    50.030734882  4148  U   N [dd] 1
>   8,0    3   177893    50.030736109  4148  D   R 8722687 + 512 [dd]
>   8,0    3   177894    50.032916699     0  C   R 8722687 + 512 [0]
>   8,0    3   177895    50.033176618  4148  A   R 8723199 + 256 <- (8,1) 8723136
>   8,0    3   177896    50.033177218  4148  Q   R 8723199 + 256 [dd]
>   8,0    3   177897    50.033181433  4148  G   R 8723199 + 256 [dd]
>   8,0    3   177898    50.033184757  4148  P   N [dd]
>   8,0    3   177899    50.033185642  4148  I   R 8723199 + 256 [dd]
>   8,0    3   177900    50.033371264  4148  A   R 8723455 + 256 <- (8,1) 8723392
>   8,0    3   177901    50.033371717  4148  Q   R 8723455 + 256 [dd]
>   8,0    3   177902    50.033374015  4148  M   R 8723455 + 256 [dd]
>   8,0    3   177903    50.033376814  4148  U   N [dd] 1
>   8,0    3   177904    50.033380126  4148  D   R 8723199 + 512 [dd]
>   8,0    3   177905    50.036715133     0  C   R 8723199 + 512 [0]
>   8,0    3   177906    50.036971296  4148  A   R 8723711 + 256 <- (8,1) 8723648
>   8,0    3   177907    50.036972136  4148  Q   R 8723711 + 256 [dd]
>   8,0    3   177908    50.036975673  4148  G   R 8723711 + 256 [dd]
>   8,0    3   177909    50.036978277  4148  P   N [dd]
>   8,0    3   177910    50.036979450  4148  I   R 8723711 + 256 [dd]
>   8,0    3   177911    50.037162429  4148  A   R 8723967 + 256 <- (8,1) 8723904
>   8,0    3   177912    50.037162840  4148  Q   R 8723967 + 256 [dd]
>   8,0    3   177913    50.037164967  4148  M   R 8723967 + 256 [dd]
>   8,0    3   177914    50.037167223  4148  U   N [dd] 1
>   8,0    3   177915    50.037170001  4148  D   R 8723711 + 512 [dd]
>   8,0    3   177916    50.040521790     0  C   R 8723711 + 512 [0]
>   8,0    3   177917    50.040729738  4148  A   R 8724223 + 256 <- (8,1) 8724160
>   8,0    3   177918    50.040730200  4148  Q   R 8724223 + 256 [dd]
>   8,0    3   177919    50.040732060  4148  G   R 8724223 + 256 [dd]
>   8,0    3   177920    50.040733551  4148  P   N [dd]
>   8,0    3   177921    50.040734109  4148  I   R 8724223 + 256 [dd]
>   8,0    3   177922    50.040860173  4148  A   R 8724479 + 160 <- (8,1) 8724416
>   8,0    3   177923    50.040860536  4148  Q   R 8724479 + 160 [dd]
>   8,0    3   177924    50.040861517  4148  M   R 8724479 + 160 [dd]
>   8,0    3   177925    50.040872542  4148  A   R 1055943 + 8 <- (8,1) 1055880
>   8,0    3   177926    50.040872800  4148  Q   R 1055943 + 8 [dd]
>   8,0    3   177927    50.040874849  4148  G   R 1055943 + 8 [dd]
>   8,0    3   177928    50.040875485  4148  I   R 1055943 + 8 [dd]
>   8,0    3   177929    50.040877045  4148  U   N [dd] 2
>   8,0    3   177930    50.040878625  4148  D   R 8724223 + 416 [dd]
>   8,0    3   177931    50.040895335  4148  D   R 1055943 + 8 [dd]
>   8,0    3   177932    50.044383267     0  C   R 8724223 + 416 [0]
>   8,0    3   177933    50.044704725     0  C   R 1055943 + 8 [0]
>   8,0    3   177934    50.044749068  4148  A   R 8724639 + 96 <- (8,1) 8724576
>   8,0    3   177935    50.044749472  4148  Q   R 8724639 + 96 [dd]
>   8,0    3   177936    50.044752184  4148  G   R 8724639 + 96 [dd]
>   8,0    3   177937    50.044753552  4148  P   N [dd]
>   8,0    3   177938    50.044754032  4148  I   R 8724639 + 96 [dd]
>   8,0    3   177939    50.044896095  4148  A   R 8724735 + 256 <- (8,1) 8724672
>   8,0    3   177940    50.044896443  4148  Q   R 8724735 + 256 [dd]
>   8,0    3   177941    50.044897538  4148  M   R 8724735 + 256 [dd]
>   8,0    3   177942    50.044948546  4148  U   N [dd] 1
>   8,0    3   177943    50.044950001  4148  D   R 8724639 + 352 [dd]
>   8,0    3   177944    50.047150137     0  C   R 8724639 + 352 [0]
>   8,0    3   177945    50.047294824  4148  A   R 8724991 + 256 <- (8,1) 8724928
>   8,0    3   177946    50.047295142  4148  Q   R 8724991 + 256 [dd]
>   8,0    3   177947    50.047296978  4148  G   R 8724991 + 256 [dd]
>   8,0    3   177948    50.047298301  4148  P   N [dd]
>   8,0    3   177949    50.047298637  4148  I   R 8724991 + 256 [dd]
>   8,0    3   177950    50.047429027  4148  A   R 8725247 + 256 <- (8,1) 8725184
>   8,0    3   177951    50.047429387  4148  Q   R 8725247 + 256 [dd]
>   8,0    3   177952    50.047430479  4148  M   R 8725247 + 256 [dd]
>   8,0    3   177953    50.047431736  4148  U   N [dd] 1
>   8,0    3   177954    50.047432951  4148  D   R 8724991 + 512 [dd]
>   8,0    3   177955    50.050313976     0  C   R 8724991 + 512 [0]
>   8,0    3   177956    50.050507961  4148  A   R 8725503 + 256 <- (8,1) 8725440
>   8,0    3   177957    50.050508273  4148  Q   R 8725503 + 256 [dd]
>   8,0    3   177958    50.050510139  4148  G   R 8725503 + 256 [dd]
>   8,0    3   177959    50.050511522  4148  P   N [dd]
>   8,0    3   177960    50.050512062  4148  I   R 8725503 + 256 [dd]
>   8,0    3   177961    50.050645393  4148  A   R 8725759 + 256 <- (8,1) 8725696
>   8,0    3   177962    50.050645867  4148  Q   R 8725759 + 256 [dd]
>   8,0    3   177963    50.050647171  4148  M   R 8725759 + 256 [dd]
>   8,0    3   177964    50.050648593  4148  U   N [dd] 1
>   8,0    3   177965    50.050649985  4148  D   R 8725503 + 512 [dd]
>   8,0    3   177966    50.053380250     0  C   R 8725503 + 512 [0]
>   8,0    3   177967    50.053576324  4148  A   R 8726015 + 256 <- (8,1) 8725952
>   8,0    3   177968    50.053576615  4148  Q   R 8726015 + 256 [dd]
>   8,0    3   177969    50.053578994  4148  G   R 8726015 + 256 [dd]
>   8,0    3   177970    50.053580173  4148  P   N [dd]
>   8,0    3   177971    50.053580509  4148  I   R 8726015 + 256 [dd]
>   8,0    3   177972    50.053711503  4148  A   R 8726271 + 256 <- (8,1) 8726208
>   8,0    3   177973    50.053712001  4148  Q   R 8726271 + 256 [dd]
>   8,0    3   177974    50.053713332  4148  M   R 8726271 + 256 [dd]
>   8,0    3   177975    50.053714583  4148  U   N [dd] 1
>   8,0    3   177976    50.053715768  4148  D   R 8726015 + 512 [dd]
>   8,0    3   177977    50.056970395     0  C   R 8726015 + 512 [0]
>   8,0    3   177978    50.057161408  4148  A   R 8726527 + 256 <- (8,1) 8726464
>   8,0    3   177979    50.057161726  4148  Q   R 8726527 + 256 [dd]
>   8,0    3   177980    50.057163718  4148  G   R 8726527 + 256 [dd]
>   8,0    3   177981    50.057165098  4148  P   N [dd]
>   8,0    3   177982    50.057165431  4148  I   R 8726527 + 256 [dd]
>   8,0    3   177983    50.057294630  4148  A   R 8726783 + 256 <- (8,1) 8726720
>   8,0    3   177984    50.057294990  4148  Q   R 8726783 + 256 [dd]
>   8,0    3   177985    50.057296070  4148  M   R 8726783 + 256 [dd]
>   8,0    3   177986    50.057297402  4148  U   N [dd] 1
>   8,0    3   177987    50.057298899  4148  D   R 8726527 + 512 [dd]
>   8,0    3   177988    50.060326743     0  C   R 8726527 + 512 [0]
>   8,0    3   177989    50.060523768  4148  A   R 8727039 + 256 <- (8,1) 8726976
>   8,0    3   177990    50.060524095  4148  Q   R 8727039 + 256 [dd]
>   8,0    3   177991    50.060525910  4148  G   R 8727039 + 256 [dd]
>   8,0    3   177992    50.060527239  4148  P   N [dd]
>   8,0    3   177993    50.060527575  4148  I   R 8727039 + 256 [dd]
>   8,0    3   177994    50.060662280  4148  A   R 8727295 + 256 <- (8,1) 8727232
>   8,0    3   177995    50.060662778  4148  Q   R 8727295 + 256 [dd]
>   8,0    3   177996    50.060663993  4148  M   R 8727295 + 256 [dd]
>   8,0    3   177997    50.060665403  4148  U   N [dd] 1
>   8,0    3   177998    50.060666999  4148  D   R 8727039 + 512 [dd]
>   8,0    3   177999    50.063922341     0  C   R 8727039 + 512 [0]
>   8,0    3   178000    50.064113177  4148  A   R 8727551 + 256 <- (8,1) 8727488
>   8,0    3   178001    50.064113492  4148  Q   R 8727551 + 256 [dd]
>   8,0    3   178002    50.064115373  4148  G   R 8727551 + 256 [dd]
> 
> -2.6.30-rc6-patched
>   8,0    3   257297    50.000760847     0  C   R 9480703 + 256 [0]
>   8,0    3   257298    50.000944399  4139  A   R 9481215 + 256 <- (8,1) 9481152
>   8,0    3   257299    50.000944693  4139  Q   R 9481215 + 256 [dd]
>   8,0    3   257300    50.000946541  4139  G   R 9481215 + 256 [dd]
>   8,0    3   257301    50.000947954  4139  P   N [dd]
>   8,0    3   257302    50.000948368  4139  I   R 9481215 + 256 [dd]
>   8,0    3   257303    50.000948920  4139  U   N [dd] 2
>   8,0    3   257304    50.000950003  4139  D   R 9481215 + 256 [dd]
>   8,0    3   257305    50.000962541  4139  U   N [dd] 2
>   8,0    3   257306    50.003034240     0  C   R 9480959 + 256 [0]
>   8,0    3   257307    50.003076338     0  C   R 9481215 + 256 [0]
>   8,0    3   257308    50.003258111  4139  A   R 9481471 + 256 <- (8,1) 9481408
>   8,0    3   257309    50.003258402  4139  Q   R 9481471 + 256 [dd]
>   8,0    3   257310    50.003260190  4139  G   R 9481471 + 256 [dd]
>   8,0    3   257311    50.003261399  4139  P   N [dd]
>   8,0    3   257312    50.003261768  4139  I   R 9481471 + 256 [dd]
>   8,0    3   257313    50.003262335  4139  U   N [dd] 1
>   8,0    3   257314    50.003263406  4139  D   R 9481471 + 256 [dd]
>   8,0    3   257315    50.003430472  4139  A   R 9481727 + 256 <- (8,1) 9481664
>   8,0    3   257316    50.003430748  4139  Q   R 9481727 + 256 [dd]
>   8,0    3   257317    50.003433065  4139  G   R 9481727 + 256 [dd]
>   8,0    3   257318    50.003434343  4139  P   N [dd]
>   8,0    3   257319    50.003434658  4139  I   R 9481727 + 256 [dd]
>   8,0    3   257320    50.003435138  4139  U   N [dd] 2
>   8,0    3   257321    50.003436083  4139  D   R 9481727 + 256 [dd]
>   8,0    3   257322    50.003447795  4139  U   N [dd] 2
>   8,0    3   257323    50.004774693     0  C   R 9481471 + 256 [0]
>   8,0    3   257324    50.004959499  4139  A   R 9481983 + 256 <- (8,1) 9481920
>   8,0    3   257325    50.004959790  4139  Q   R 9481983 + 256 [dd]
>   8,0    3   257326    50.004961590  4139  G   R 9481983 + 256 [dd]
>   8,0    3   257327    50.004962793  4139  P   N [dd]
>   8,0    3   257328    50.004963153  4139  I   R 9481983 + 256 [dd]
>   8,0    3   257329    50.004964098  4139  U   N [dd] 2
>   8,0    3   257330    50.004965184  4139  D   R 9481983 + 256 [dd]
>   8,0    3   257331    50.004978967  4139  U   N [dd] 2
>   8,0    3   257332    50.006865854     0  C   R 9481727 + 256 [0]
>   8,0    3   257333    50.007052043  4139  A   R 9482239 + 256 <- (8,1) 9482176
>   8,0    3   257334    50.007052331  4139  Q   R 9482239 + 256 [dd]
>   8,0    3   257335    50.007054146  4139  G   R 9482239 + 256 [dd]
>   8,0    3   257336    50.007055355  4139  P   N [dd]
>   8,0    3   257337    50.007055724  4139  I   R 9482239 + 256 [dd]
>   8,0    3   257338    50.007056438  4139  U   N [dd] 2
>   8,0    3   257339    50.007057605  4139  D   R 9482239 + 256 [dd]
>   8,0    3   257340    50.007069963  4139  U   N [dd] 2
>   8,0    3   257341    50.008250294     0  C   R 9481983 + 256 [0]
>   8,0    3   257342    50.008431589  4139  A   R 9482495 + 256 <- (8,1) 9482432
>   8,0    3   257343    50.008431881  4139  Q   R 9482495 + 256 [dd]
>   8,0    3   257344    50.008433921  4139  G   R 9482495 + 256 [dd]
>   8,0    3   257345    50.008435097  4139  P   N [dd]
>   8,0    3   257346    50.008435466  4139  I   R 9482495 + 256 [dd]
>   8,0    3   257347    50.008436213  4139  U   N [dd] 2
>   8,0    3   257348    50.008437296  4139  D   R 9482495 + 256 [dd]
>   8,0    3   257349    50.008450034  4139  U   N [dd] 2
>   8,0    3   257350    50.010008843     0  C   R 9482239 + 256 [0]
>   8,0    3   257351    50.010135287  4139  C   R 9482495 + 256 [0]
>   8,0    3   257352    50.010226816  4139  A   R 9482751 + 256 <- (8,1) 9482688
>   8,0    3   257353    50.010227107  4139  Q   R 9482751 + 256 [dd]
>   8,0    3   257354    50.010229363  4139  G   R 9482751 + 256 [dd]
>   8,0    3   257355    50.010230728  4139  P   N [dd]
>   8,0    3   257356    50.010231097  4139  I   R 9482751 + 256 [dd]
>   8,0    3   257357    50.010231655  4139  U   N [dd] 1
>   8,0    3   257358    50.010232696  4139  D   R 9482751 + 256 [dd]
>   8,0    3   257359    50.010380946  4139  A   R 9483007 + 256 <- (8,1) 9482944
>   8,0    3   257360    50.010381264  4139  Q   R 9483007 + 256 [dd]
>   8,0    3   257361    50.010383358  4139  G   R 9483007 + 256 [dd]
>   8,0    3   257362    50.010384429  4139  P   N [dd]
>   8,0    3   257363    50.010384741  4139  I   R 9483007 + 256 [dd]
>   8,0    3   257364    50.010385395  4139  U   N [dd] 2
>   8,0    3   257365    50.010386364  4139  D   R 9483007 + 256 [dd]
>   8,0    3   257366    50.010397869  4139  U   N [dd] 2
>   8,0    3   257367    50.014210132     0  C   R 9482751 + 256 [0]
>   8,0    3   257368    50.014252938     0  C   R 9483007 + 256 [0]
>   8,0    3   257369    50.014430811  4139  A   R 9483263 + 256 <- (8,1) 9483200
>   8,0    3   257370    50.014431105  4139  Q   R 9483263 + 256 [dd]
>   8,0    3   257371    50.014433139  4139  G   R 9483263 + 256 [dd]
>   8,0    3   257372    50.014434520  4139  P   N [dd]
>   8,0    3   257373    50.014435110  4139  I   R 9483263 + 256 [dd]
>   8,0    3   257374    50.014435674  4139  U   N [dd] 1
>   8,0    3   257375    50.014436770  4139  D   R 9483263 + 256 [dd]
>   8,0    3   257376    50.014592117  4139  A   R 9483519 + 256 <- (8,1) 9483456
>   8,0    3   257377    50.014592573  4139  Q   R 9483519 + 256 [dd]
>   8,0    3   257378    50.014594391  4139  G   R 9483519 + 256 [dd]
>   8,0    3   257379    50.014595504  4139  P   N [dd]
>   8,0    3   257380    50.014595876  4139  I   R 9483519 + 256 [dd]
>   8,0    3   257381    50.014596366  4139  U   N [dd] 2
>   8,0    3   257382    50.014597368  4139  D   R 9483519 + 256 [dd]
>   8,0    3   257383    50.014609521  4139  U   N [dd] 2
>   8,0    3   257384    50.015937813     0  C   R 9483263 + 256 [0]
>   8,0    3   257385    50.016124825  4139  A   R 9483775 + 256 <- (8,1) 9483712
>   8,0    3   257386    50.016125116  4139  Q   R 9483775 + 256 [dd]
>   8,0    3   257387    50.016127162  4139  G   R 9483775 + 256 [dd]
>   8,0    3   257388    50.016128569  4139  P   N [dd]
>   8,0    3   257389    50.016128983  4139  I   R 9483775 + 256 [dd]
>   8,0    3   257390    50.016129538  4139  U   N [dd] 2
>   8,0    3   257391    50.016130627  4139  D   R 9483775 + 256 [dd]
>   8,0    3   257392    50.016143077  4139  U   N [dd] 2
>   8,0    3   257393    50.016925304     0  C   R 9483519 + 256 [0]
>   8,0    3   257394    50.017111307  4139  A   R 9484031 + 256 <- (8,1) 9483968
>   8,0    3   257395    50.017111598  4139  Q   R 9484031 + 256 [dd]
>   8,0    3   257396    50.017113410  4139  G   R 9484031 + 256 [dd]
>   8,0    3   257397    50.017114835  4139  P   N [dd]
>   8,0    3   257398    50.017115213  4139  I   R 9484031 + 256 [dd]
>   8,0    3   257399    50.017115765  4139  U   N [dd] 2
>   8,0    3   257400    50.017116839  4139  D   R 9484031 + 256 [dd]
>   8,0    3   257401    50.017129023  4139  U   N [dd] 2
>   8,0    3   257402    50.017396693     0  C   R 9483775 + 256 [0]
>   8,0    3   257403    50.017584595  4139  A   R 9484287 + 256 <- (8,1) 9484224
>   8,0    3   257404    50.017585018  4139  Q   R 9484287 + 256 [dd]
>   8,0    3   257405    50.017586866  4139  G   R 9484287 + 256 [dd]
>   8,0    3   257406    50.017587997  4139  P   N [dd]
>   8,0    3   257407    50.017588393  4139  I   R 9484287 + 256 [dd]
>   8,0    3   257408    50.017589105  4139  U   N [dd] 2
>   8,0    3   257409    50.017590173  4139  D   R 9484287 + 256 [dd]
>   8,0    3   257410    50.017602614  4139  U   N [dd] 2
>   8,0    3   257411    50.020578876     0  C   R 9484031 + 256 [0]
>   8,0    3   257412    50.020721857  4139  C   R 9484287 + 256 [0]
>   8,0    3   257413    50.020803183  4139  A   R 9484543 + 256 <- (8,1) 9484480
>   8,0    3   257414    50.020803507  4139  Q   R 9484543 + 256 [dd]
>   8,0    3   257415    50.020805256  4139  G   R 9484543 + 256 [dd]
>   8,0    3   257416    50.020806672  4139  P   N [dd]
>   8,0    3   257417    50.020807065  4139  I   R 9484543 + 256 [dd]
>   8,0    3   257418    50.020807668  4139  U   N [dd] 1
>   8,0    3   257419    50.020808733  4139  D   R 9484543 + 256 [dd]
>   8,0    3   257420    50.020957132  4139  A   R 9484799 + 256 <- (8,1) 9484736
>   8,0    3   257421    50.020957423  4139  Q   R 9484799 + 256 [dd]
>   8,0    3   257422    50.020959205  4139  G   R 9484799 + 256 [dd]
>   8,0    3   257423    50.020960276  4139  P   N [dd]
>   8,0    3   257424    50.020960594  4139  I   R 9484799 + 256 [dd]
>   8,0    3   257425    50.020961062  4139  U   N [dd] 2
>   8,0    3   257426    50.020961959  4139  D   R 9484799 + 256 [dd]
>   8,0    3   257427    50.020974191  4139  U   N [dd] 2
>   8,0    3   257428    50.023987847     0  C   R 9484543 + 256 [0]
>   8,0    3   257429    50.024093062  4139  C   R 9484799 + 256 [0]
>   8,0    3   257430    50.024207161  4139  A   R 9485055 + 256 <- (8,1) 9484992
>   8,0    3   257431    50.024207434  4139  Q   R 9485055 + 256 [dd]
>   8,0    3   257432    50.024209567  4139  G   R 9485055 + 256 [dd]
>   8,0    3   257433    50.024210728  4139  P   N [dd]
>   8,0    3   257434    50.024211097  4139  I   R 9485055 + 256 [dd]
>   8,0    3   257435    50.024211661  4139  U   N [dd] 1
>   8,0    3   257436    50.024212693  4139  D   R 9485055 + 256 [dd]
>   8,0    3   257437    50.024359266  4139  A   R 9485311 + 256 <- (8,1) 9485248
>   8,0    3   257438    50.024359584  4139  Q   R 9485311 + 256 [dd]
>   8,0    3   257439    50.024361720  4139  G   R 9485311 + 256 [dd]
>   8,0    3   257440    50.024362794  4139  P   N [dd]
>   8,0    3   257441    50.024363106  4139  I   R 9485311 + 256 [dd]
>   8,0    3   257442    50.024363760  4139  U   N [dd] 2
>   8,0    3   257443    50.024364759  4139  D   R 9485311 + 256 [dd]
>   8,0    3   257444    50.024376535  4139  U   N [dd] 2
>   8,0    3   257445    50.026532544     0  C   R 9485055 + 256 [0]
>   8,0    3   257446    50.026714236  4139  A   R 9485567 + 256 <- (8,1) 9485504
>   8,0    3   257447    50.026714524  4139  Q   R 9485567 + 256 [dd]
>   8,0    3   257448    50.026716354  4139  G   R 9485567 + 256 [dd]
>   8,0    3   257449    50.026717791  4139  P   N [dd]
>   8,0    3   257450    50.026718175  4139  I   R 9485567 + 256 [dd]
>   8,0    3   257451    50.026718778  4139  U   N [dd] 2
>   8,0    3   257452    50.026719876  4139  D   R 9485567 + 256 [dd]
>   8,0    3   257453    50.026736383  4139  U   N [dd] 2
>   8,0    3   257454    50.028531879     0  C   R 9485311 + 256 [0]
>   8,0    3   257455    50.028684347  4139  C   R 9485567 + 256 [0]
>   8,0    3   257456    50.028758787  4139  A   R 9485823 + 256 <- (8,1) 9485760
>   8,0    3   257457    50.028759069  4139  Q   R 9485823 + 256 [dd]
>   8,0    3   257458    50.028760884  4139  G   R 9485823 + 256 [dd]
>   8,0    3   257459    50.028762099  4139  P   N [dd]
>   8,0    3   257460    50.028762447  4139  I   R 9485823 + 256 [dd]
>   8,0    3   257461    50.028763038  4139  U   N [dd] 1
>   8,0    3   257462    50.028764268  4139  D   R 9485823 + 256 [dd]
>   8,0    3   257463    50.028909841  4139  A   R 9486079 + 256 <- (8,1) 9486016
>   8,0    3   257464    50.028910156  4139  Q   R 9486079 + 256 [dd]
>   8,0    3   257465    50.028911896  4139  G   R 9486079 + 256 [dd]
>   8,0    3   257466    50.028912964  4139  P   N [dd]
>   8,0    3   257467    50.028913270  4139  I   R 9486079 + 256 [dd]
>   8,0    3   257468    50.028913912  4139  U   N [dd] 2
>   8,0    3   257469    50.028914878  4139  D   R 9486079 + 256 [dd]
>   8,0    3   257470    50.028927497  4139  U   N [dd] 2
>   8,0    3   257471    50.031158357     0  C   R 9485823 + 256 [0]
>   8,0    3   257472    50.031292365  4139  C   R 9486079 + 256 [0]
>   8,0    3   257473    50.031369697  4139  A   R 9486335 + 160 <- (8,1) 9486272
>   8,0    3   257474    50.031369988  4139  Q   R 9486335 + 160 [dd]
>   8,0    3   257475    50.031371779  4139  G   R 9486335 + 160 [dd]
>   8,0    3   257476    50.031372850  4139  P   N [dd]
>   8,0    3   257477    50.031373198  4139  I   R 9486335 + 160 [dd]
>   8,0    3   257478    50.031384931  4139  A   R 1056639 + 8 <- (8,1) 1056576
>   8,0    3   257479    50.031385201  4139  Q   R 1056639 + 8 [dd]
>   8,0    3   257480    50.031388480  4139  G   R 1056639 + 8 [dd]
>   8,0    3   257481    50.031388904  4139  I   R 1056639 + 8 [dd]
>   8,0    3   257482    50.031390362  4139  U   N [dd] 2
>   8,0    3   257483    50.031391523  4139  D   R 9486335 + 160 [dd]
>   8,0    3   257484    50.031403403  4139  D   R 1056639 + 8 [dd]
>   8,0    3   257485    50.033630747     0  C   R 1056639 + 8 [0]
>   8,0    3   257486    50.033690300  4139  A   R 9486495 + 96 <- (8,1) 9486432
>   8,0    3   257487    50.033690810  4139  Q   R 9486495 + 96 [dd]
>   8,0    3   257488    50.033694581  4139  G   R 9486495 + 96 [dd]
>   8,0    3   257489    50.033696739  4139  P   N [dd]
>   8,0    3   257490    50.033697357  4139  I   R 9486495 + 96 [dd]
>   8,0    3   257491    50.033698611  4139  U   N [dd] 2
>   8,0    3   257492    50.033700945  4139  D   R 9486495 + 96 [dd]
>   8,0    3   257493    50.033727763  4139  C   R 9486335 + 160 [0]
>   8,0    3   257494    50.033996024  4139  A   R 9486591 + 256 <- (8,1) 9486528
>   8,0    3   257495    50.033996396  4139  Q   R 9486591 + 256 [dd]
>   8,0    3   257496    50.034000030  4139  G   R 9486591 + 256 [dd]
>   8,0    3   257497    50.034002268  4139  P   N [dd]
>   8,0    3   257498    50.034002820  4139  I   R 9486591 + 256 [dd]
>   8,0    3   257499    50.034003924  4139  U   N [dd] 2
>   8,0    3   257500    50.034006201  4139  D   R 9486591 + 256 [dd]
>   8,0    3   257501    50.034091438  4139  U   N [dd] 2
>   8,0    3   257502    50.034637372     0  C   R 9486495 + 96 [0]
>   8,0    3   257503    50.034841508  4139  A   R 9486847 + 256 <- (8,1) 9486784
>   8,0    3   257504    50.034842072  4139  Q   R 9486847 + 256 [dd]
>   8,0    3   257505    50.034846117  4139  G   R 9486847 + 256 [dd]
>   8,0    3   257506    50.034848676  4139  P   N [dd]
>   8,0    3   257507    50.034849384  4139  I   R 9486847 + 256 [dd]
>   8,0    3   257508    50.034850545  4139  U   N [dd] 2
>   8,0    3   257509    50.034852795  4139  D   R 9486847 + 256 [dd]
>   8,0    3   257510    50.034875503  4139  U   N [dd] 2
>   8,0    3   257511    50.035370009     0  C   R 9486591 + 256 [0]
>   8,0    3   257512    50.035622315  4139  A   R 9487103 + 256 <- (8,1) 9487040
>   8,0    3   257513    50.035622954  4139  Q   R 9487103 + 256 [dd]
>   8,0    3   257514    50.035627101  4139  G   R 9487103 + 256 [dd]
>   8,0    3   257515    50.035629510  4139  P   N [dd]
>   8,0    3   257516    50.035630143  4139  I   R 9487103 + 256 [dd]
>   8,0    3   257517    50.035631058  4139  U   N [dd] 2
>   8,0    3   257518    50.035632657  4139  D   R 9487103 + 256 [dd]
>   8,0    3   257519    50.035656358  4139  U   N [dd] 2
>   8,0    3   257520    50.036703329     0  C   R 9486847 + 256 [0]
>   8,0    3   257521    50.036963604  4139  A   R 9487359 + 256 <- (8,1) 9487296
>   8,0    3   257522    50.036964057  4139  Q   R 9487359 + 256 [dd]
>   8,0    3   257523    50.036967636  4139  G   R 9487359 + 256 [dd]
>   8,0    3   257524    50.036969710  4139  P   N [dd]
>   8,0    3   257525    50.036970586  4139  I   R 9487359 + 256 [dd]
>   8,0    3   257526    50.036971684  4139  U   N [dd] 2
>   8,0    3   257527    50.036973631  4139  D   R 9487359 + 256 [dd]
>   8,0    3   257528    50.036995034  4139  U   N [dd] 2
>   8,0    3   257529    50.038904428     0  C   R 9487103 + 256 [0]
>   8,0    3   257530    50.039161508  4139  A   R 9487615 + 256 <- (8,1) 9487552
>   8,0    3   257531    50.039161934  4139  Q   R 9487615 + 256 [dd]
>   8,0    3   257532    50.039165834  4139  G   R 9487615 + 256 [dd]
>   8,0    3   257533    50.039168561  4139  P   N [dd]
>   8,0    3   257534    50.039169353  4139  I   R 9487615 + 256 [dd]
>   8,0    3   257535    50.039170343  4139  U   N [dd] 2
>   8,0    3   257536    50.039171645  4139  D   R 9487615 + 256 [dd]
>   8,0    3   257537    50.039193195  4139  U   N [dd] 2
>   8,0    3   257538    50.040570003     0  C   R 9487359 + 256 [0]
>   8,0    3   257539    50.040842161  4139  A   R 9487871 + 256 <- (8,1) 9487808
>   8,0    3   257540    50.040842827  4139  Q   R 9487871 + 256 [dd]
>   8,0    3   257541    50.040846803  4139  G   R 9487871 + 256 [dd]
>   8,0    3   257542    50.040849902  4139  P   N [dd]
>   8,0    3   257543    50.040850715  4139  I   R 9487871 + 256 [dd]
>   8,0    3   257544    50.040851642  4139  U   N [dd] 2
>   8,0    3   257545    50.040853658  4139  D   R 9487871 + 256 [dd]
>   8,0    3   257546    50.040876270  4139  U   N [dd] 2
>   8,0    3   257547    50.042081391     0  C   R 9487615 + 256 [0]
>   8,0    3   257548    50.042215837  4139  C   R 9487871 + 256 [0]
>   8,0    3   257549    50.042316192  4139  A   R 9488127 + 256 <- (8,1) 9488064
>   8,0    3   257550    50.042316633  4139  Q   R 9488127 + 256 [dd]
>   8,0    3   257551    50.042319213  4139  G   R 9488127 + 256 [dd]
>   8,0    3   257552    50.042320803  4139  P   N [dd]
>   8,0    3   257553    50.042321412  4139  I   R 9488127 + 256 [dd]
>   8,0    3   257554    50.042322219  4139  U   N [dd] 1
>   8,0    3   257555    50.042323362  4139  D   R 9488127 + 256 [dd]
>   8,0    3   257556    50.042484350  4139  A   R 9488383 + 256 <- (8,1) 9488320
>   8,0    3   257557    50.042484602  4139  Q   R 9488383 + 256 [dd]
>   8,0    3   257558    50.042486744  4139  G   R 9488383 + 256 [dd]
>   8,0    3   257559    50.042487908  4139  P   N [dd]
>   8,0    3   257560    50.042488223  4139  I   R 9488383 + 256 [dd]
>   8,0    3   257561    50.042488754  4139  U   N [dd] 2
>   8,0    3   257562    50.042489927  4139  D   R 9488383 + 256 [dd]
>   8,0    3   257563    50.042502678  4139  U   N [dd] 2
>   8,0    3   257564    50.045166592     0  C   R 9488127 + 256 [0]
>   8,0    3   257565    50.045355163  4139  A   R 9488639 + 256 <- (8,1) 9488576
>   8,0    3   257566    50.045355493  4139  Q   R 9488639 + 256 [dd]
>   8,0    3   257567    50.045357497  4139  G   R 9488639 + 256 [dd]
>   8,0    3   257568    50.045358673  4139  P   N [dd]
>   8,0    3   257569    50.045359267  4139  I   R 9488639 + 256 [dd]
>   8,0    3   257570    50.045359831  4139  U   N [dd] 2
>   8,0    3   257571    50.045360911  4139  D   R 9488639 + 256 [dd]
>   8,0    3   257572    50.045373959  4139  U   N [dd] 2
>   8,0    3   257573    50.046450730     0  C   R 9488383 + 256 [0]
>   8,0    3   257574    50.046641639  4139  A   R 9488895 + 256 <- (8,1) 9488832
>   8,0    3   257575    50.046642086  4139  Q   R 9488895 + 256 [dd]
>   8,0    3   257576    50.046643937  4139  G   R 9488895 + 256 [dd]
>   8,0    3   257577    50.046645092  4139  P   N [dd]
>   8,0    3   257578    50.046645527  4139  I   R 9488895 + 256 [dd]
>   8,0    3   257579    50.046646244  4139  U   N [dd] 2
>   8,0    3   257580    50.046647327  4139  D   R 9488895 + 256 [dd]
>   8,0    3   257581    50.046660234  4139  U   N [dd] 2
>   8,0    3   257582    50.047826305     0  C   R 9488639 + 256 [0]
>   8,0    3   257583    50.048011468  4139  A   R 9489151 + 256 <- (8,1) 9489088
>   8,0    3   257584    50.048011762  4139  Q   R 9489151 + 256 [dd]
>   8,0    3   257585    50.048013793  4139  G   R 9489151 + 256 [dd]
>   8,0    3   257586    50.048014966  4139  P   N [dd]
>   8,0    3   257587    50.048015380  4139  I   R 9489151 + 256 [dd]
>   8,0    3   257588    50.048016112  4139  U   N [dd] 2
>   8,0    3   257589    50.048017202  4139  D   R 9489151 + 256 [dd]
>   8,0    3   257590    50.048029553  4139  U   N [dd] 2
>   8,0    3   257591    50.049319830     0  C   R 9488895 + 256 [0]
>   8,0    3   257592    50.049446089  4139  C   R 9489151 + 256 [0]
>   8,0    3   257593    50.049545199  4139  A   R 9489407 + 256 <- (8,1) 9489344
>   8,0    3   257594    50.049545628  4139  Q   R 9489407 + 256 [dd]
>   8,0    3   257595    50.049547512  4139  G   R 9489407 + 256 [dd]
>   8,0    3   257596    50.049548886  4139  P   N [dd]
>   8,0    3   257597    50.049549318  4139  I   R 9489407 + 256 [dd]
>   8,0    3   257598    50.049550047  4139  U   N [dd] 1
>   8,0    3   257599    50.049551241  4139  D   R 9489407 + 256 [dd]
>   8,0    3   257600    50.049699283  4139  A   R 9489663 + 256 <- (8,1) 9489600
>   8,0    3   257601    50.049699556  4139  Q   R 9489663 + 256 [dd]
>   8,0    3   257602    50.049701266  4139  G   R 9489663 + 256 [dd]
>   8,0    3   257603    50.049702310  4139  P   N [dd]
>   8,0    3   257604    50.049702656  4139  I   R 9489663 + 256 [dd]
>   8,0    3   257605    50.049703118  4139  U   N [dd] 2
>   8,0    3   257606    50.049704020  4139  D   R 9489663 + 256 [dd]
>   8,0    3   257607    50.049715940  4139  U   N [dd] 2
>   8,0    3   257608    50.052662150     0  C   R 9489407 + 256 [0]
>   8,0    3   257609    50.052853688  4139  A   R 9489919 + 256 <- (8,1) 9489856
>   8,0    3   257610    50.052853985  4139  Q   R 9489919 + 256 [dd]
>   8,0    3   257611    50.052855869  4139  G   R 9489919 + 256 [dd]
>   8,0    3   257612    50.052857057  4139  P   N [dd]
>   8,0    3   257613    50.052857423  4139  I   R 9489919 + 256 [dd]
>   8,0    3   257614    50.052858065  4139  U   N [dd] 2
>   8,0    3   257615    50.052859164  4139  D   R 9489919 + 256 [dd]
>   8,0    3   257616    50.052871806  4139  U   N [dd] 2
>   8,0    3   257617    50.053470795     0  C   R 9489663 + 256 [0]
>   8,0    3   257618    50.053661719  4139  A   R 9490175 + 256 <- (8,1) 9490112
>   8,0    3   257619    50.053662097  4139  Q   R 9490175 + 256 [dd]
>   8,0    3   257620    50.053663891  4139  G   R 9490175 + 256 [dd]
>   8,0    3   257621    50.053665034  4139  P   N [dd]
>   8,0    3   257622    50.053665436  4139  I   R 9490175 + 256 [dd]
>   8,0    3   257623    50.053665982  4139  U   N [dd] 2
>   8,0    3   257624    50.053667077  4139  D   R 9490175 + 256 [dd]
>   8,0    3   257625    50.053679732  4139  U   N [dd] 2
>   8,0    3   257626    50.055776383     0  C   R 9489919 + 256 [0]
>   8,0    3   257627    50.055915017  4139  C   R 9490175 + 256 [0]
>   8,0    3   257628    50.055997812  4139  A   R 9490431 + 256 <- (8,1) 9490368
>   8,0    3   257629    50.055998085  4139  Q   R 9490431 + 256 [dd]
>   8,0    3   257630    50.055999867  4139  G   R 9490431 + 256 [dd]
>   8,0    3   257631    50.056001049  4139  P   N [dd]
>   8,0    3   257632    50.056001451  4139  I   R 9490431 + 256 [dd]
>   8,0    3   257633    50.056002189  4139  U   N [dd] 1
>   8,0    3   257634    50.056003197  4139  D   R 9490431 + 256 [dd]
>   8,0    3   257635    50.056149977  4139  A   R 9490687 + 256 <- (8,1) 9490624
>   8,0    3   257636    50.056150279  4139  Q   R 9490687 + 256 [dd]
>   8,0    3   257637    50.056152047  4139  G   R 9490687 + 256 [dd]
>   8,0    3   257638    50.056153109  4139  P   N [dd]
>   8,0    3   257639    50.056153442  4139  I   R 9490687 + 256 [dd]
>   8,0    3   257640    50.056153904  4139  U   N [dd] 2
>   8,0    3   257641    50.056154852  4139  D   R 9490687 + 256 [dd]
>   8,0    3   257642    50.056166948  4139  U   N [dd] 2
>   8,0    3   257643    50.057600660     0  C   R 9490431 + 256 [0]
>   8,0    3   257644    50.057786753  4139  A   R 9490943 + 256 <- (8,1) 9490880
>   8,0    3   257645    50.057787050  4139  Q   R 9490943 + 256 [dd]
>   8,0    3   257646    50.057788865  4139  G   R 9490943 + 256 [dd]
>   8,0    3   257647    50.057790236  4139  P   N [dd]
>   8,0    3   257648    50.057790614  4139  I   R 9490943 + 256 [dd]
>   8,0    3   257649    50.057791169  4139  U   N [dd] 2
>   8,0    3   257650    50.057792246  4139  D   R 9490943 + 256 [dd]
>   8,0    3   257651    50.057804469  4139  U   N [dd] 2
>   8,0    3   257652    50.060322995     0  C   R 9490687 + 256 [0]
>   8,0    3   257653    50.060464005  4139  C   R 9490943 + 256 [0]
>   8,0    3   257654    50.060548216  4139  A   R 9491199 + 256 <- (8,1) 9491136
>   8,0    3   257655    50.060548696  4139  Q   R 9491199 + 256 [dd]
>   8,0    3   257656    50.060550922  4139  G   R 9491199 + 256 [dd]
>   8,0    3   257657    50.060552096  4139  P   N [dd]
>   8,0    3   257658    50.060552531  4139  I   R 9491199 + 256 [dd]
>   8,0    3   257659    50.060553101  4139  U   N [dd] 1
>   8,0    3   257660    50.060554100  4139  D   R 9491199 + 256 [dd]
>   8,0    3   257661    50.060701569  4139  A   R 9491455 + 256 <- (8,1) 9491392
>   8,0    3   257662    50.060701890  4139  Q   R 9491455 + 256 [dd]
>   8,0    3   257663    50.060703993  4139  G   R 9491455 + 256 [dd]
>   8,0    3   257664    50.060705070  4139  P   N [dd]
>   8,0    3   257665    50.060705385  4139  I   R 9491455 + 256 [dd]
>   8,0    3   257666    50.060706012  4139  U   N [dd] 2
>   8,0    3   257667    50.060706987  4139  D   R 9491455 + 256 [dd]
>   8,0    3   257668    50.060718784  4139  U   N [dd] 2
>   8,0    3   257669    50.062964966     0  C   R 9491199 + 256 [0]
>   8,0    3   257670    50.063102772  4139  C   R 9491455 + 256 [0]
>   8,0    3   257671    50.063182666  4139  A   R 9491711 + 256 <- (8,1) 9491648
>   8,0    3   257672    50.063182939  4139  Q   R 9491711 + 256 [dd]
>   8,0    3   257673    50.063184889  4139  G   R 9491711 + 256 [dd]
>   8,0    3   257674    50.063186074  4139  P   N [dd]
>   8,0    3   257675    50.063186440  4139  I   R 9491711 + 256 [dd]
>   8,0    3   257676    50.063187271  4139  U   N [dd] 1
>   8,0    3   257677    50.063188312  4139  D   R 9491711 + 256 [dd]
>   8,0    3   257678    50.063340467  4139  A   R 9491967 + 256 <- (8,1) 9491904
>   8,0    3   257679    50.063340749  4139  Q   R 9491967 + 256 [dd]
>   8,0    3   257680    50.063342529  4139  G   R 9491967 + 256 [dd]
>   8,0    3   257681    50.063343597  4139  P   N [dd]
>   8,0    3   257682    50.063343915  4139  I   R 9491967 + 256 [dd]
>   8,0    3   257683    50.063344374  4139  U   N [dd] 2
>   8,0    3   257684    50.063345313  4139  D   R 9491967 + 256 [dd]
>   8,0    3   257685    50.063357370  4139  U   N [dd] 2
>   8,0    3   257686    50.066605011     0  C   R 9491711 + 256 [0]
>   8,0    3   257687    50.066643587     0  C   R 9491967 + 256 [0]
>   8,0    3   257688    50.066821310  4139  A   R 9492223 + 256 <- (8,1) 9492160
>   8,0    3   257689    50.066821601  4139  Q   R 9492223 + 256 [dd]
>   8,0    3   257690    50.066823605  4139  G   R 9492223 + 256 [dd]
>   8,0    3   257691    50.066825063  4139  P   N [dd]
> 
> 
> 
> >
> >
> >> 
> >>                if (PageReadahead(page))
> >>                         page_cache_async_readahead()
> >>                 if (!PageUptodate(page))
> >>                                 goto page_not_up_to_date;
> >>                 //...
> >> page_not_up_to_date:
> >>                 lock_page_killable(page);
> >> 
> >> Therefore explicit unplugging can help.
> >> 
> >> Signed-off-by: Hisashi Hifumi <hifumi.hisashi@....ntt.co.jp>
> >> Acked-by: Wu Fengguang <fengguang.wu@...el.com> 
> >> 
> >> 
> >>  mm/readahead.c |   10 ++++++++++
> >>  1 file changed, 10 insertions(+)
> >> 
> >> --- linux.orig/mm/readahead.c
> >> +++ linux/mm/readahead.c
> >> @@ -490,5 +490,15 @@ page_cache_async_readahead(struct addres
> >>  
> >>  	/* do read-ahead */
> >>  	ondemand_readahead(mapping, ra, filp, true, offset, req_size);
> >> +
> >> +	/*
> >> +	* Normally the current page is !uptodate and lock_page() will be
> >> +	* immediately called to implicitly unplug the device. However this
> >> +	* is not always true for RAID conifgurations, where data arrives
> >> +	* not strictly in their submission order. In this case we need to
> >> +	* explicitly kick off the IO.
> >> +	*/
> >> +	if (PageUptodate(page))
> >> +		blk_run_backing_dev(mapping->backing_dev_info, NULL);
> >>  }
> >>  EXPORT_SYMBOL_GPL(page_cache_async_readahead); 
> >> 
> 



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ