Message-ID: <a4a81aa4-4570-08c2-14c1-2ca0a0a8fcd0@lp-programming.com>
Date:   Thu, 21 May 2020 20:43:42 -0700
From:   Logan Perkins <logan@...l.lp-programming.com>
To:     minchan@...nel.org, ngupta@...are.org, Jens Axboe <axboe@...nel.dk>
Cc:     linux-block@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: PROBLEM: zram with writeback fails writing to backing_dev


[1.] zram with writeback fails writing to backing_dev
[2.] When I set up /dev/zram0 with a backing device, and then feed it 
sufficient
incompressible data that it has to push some to the backing device, I start
getting Buffer I/O errors showing up in dmesg. If there is a filesystem on
/dev/zram0, it usually gets remounted ro within a couple seconds, and 
whatever
data was written to it gets garbled. This happens either with a loop 
device or
a sata disk partition used as the backing device.
[3.] ZRAM, ZRAM_WRITEBACK
[4.] Kernel information
[4.1.] First encountered: Linux version 5.6.14-gentoo-lp-debiancfg2020.5 
(root@...too-server) (gcc version 9.2.0 (Gentoo 9.2.0-r2 p3)) #1 SMP 
PREEMPT Thu May 21 01:23:37 PDT 2020
Subsequent Testing: Linux version 5.7.0-rc6-x86_64 (root@...-testing) 
(gcc version 9.2.0 (Gentoo Hardened 9.2.0-r2 p3), GNU ld (Gentoo 2.33.1 
p2) 2.33.1) #3 SMP Thu May 21 15:29:23 PDT 2020
[4.2.] Kernel .config file: See Attached (from 5.7.0-rc6)
[5.] 5.0? Maybe 5.1, last used successfully about a year ago.
[6.] Not getting an oops, just the filesystem and block write errors, 
dmesg output attached.
[7.] (as root)
#!/bin/bash
modprobe zram
cd /tmp
dd if=/dev/zero of=loop0.img bs=1M count=1024
losetup /dev/loop0 /tmp/loop0.img
cd /sys/block/zram0
echo /dev/loop0 > backing_dev
echo $((1024*1024*256)) > mem_limit
echo $((1024*1024*1024*10)) > disksize
dd if=/dev/urandom of=/dev/zram0 bs=1M count=1024 &
dmesg -wH
# note that mkfs.ext4 /dev/zram0, followed by mounting it and putting a 
large
# file on it also demonstrates the issue, as does using it as swap and 
consuming
# lots of memory.

[8.] Environment
[8.1.] Software (add the output of the ver_linux script here)
Linux BIT-testing 5.7.0-rc6-x86_64 #3 SMP Thu May 21 15:29:23 PDT 2020 
x86_64 Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz GenuineIntel GNU/Linux

GNU C 9.2.0
GNU Make 4.2.1
Binutils 2.33.1
Util-linux 2.33.2
Mount 2.33.2
Module-init-tools 26
E2fsprogs 1.45.5
PPP 2.4.8
Bison 3.1
Flex 2.6.4
Linux C++ Library 6.0.27
Linux C Library 2.29
Dynamic linker (ldd) 2.29
Procps 3.3.15
Net-tools 2.10
Kbd 2.0.4
Console-tools 2.0.4
Sh-utils 8.31
Udev 243
Modules Loaded 3w_9xxx 3w_xxxx aacraid ahci aic79xx aic7xxx aic94xx 
arcmsr async_memcpy async_pq async_raid6_recov async_tx async_xor 
ata_piix atp870u binfmt_misc btrfs BusLogic coretemp crypto_engine dax 
dc395x dm_bufio dm_crypt dm_log dm_log_writes dm_mirror dm_mod 
dm_region_hash dm_snapshot dmx3191d dm_zero gdth grace i2c_i801 initio 
input_leds joydev led_class libahci libsas linear lockd lpfc lzo 
lzo_compress megaraid megaraid_mbox megaraid_mm megaraid_sas mptbase 
mptfc mptsas mptscsih mptspi multipath nfs nvme nvme_core nvme_fabrics 
nvme_fc nvme_loop nvmet nvmet_fc pata_ali pata_amd pata_artop 
pata_atiixp pata_cmd64x pata_efar pata_hpt366 pata_hpt37x pata_hpt3x2n 
pata_hpt3x3 pata_it8213 pata_it821x pata_jmicron pata_marvell pata_mpiix 
pata_netcell pata_ns87410 pata_ns87415 pata_opti pata_optidma 
pata_pdc2027x pata_pdc202xx_old pata_radisys pata_rz1000 
pata_serverworks pata_sil680 pata_sis pata_sl82c105 pata_triflex 
pata_via pdc_adma qla1280 qla2xxx raid0 raid10 raid1 raid456 raid6_pq 
rtc_cmos sata_inic162x sata_mv sata_nv sata_promise sata_qstor sata_sil 
sata_sil24 sata_sis sata_svw sata_sx4 sata_uli sata_via sata_vsc 
scsi_transport_fc scsi_transport_sas scsi_transport_spi sg sl811_hcd 
sunrpc sym53c8xx uas usbhid virtio_crypto virtio_input 
x86_pkg_temp_thermal xhci_plat_hcd xor xxhash zram zsmalloc 
zstd_compress zstd_decompress

[8.2.] Processor information (from /proc/cpuinfo): 8 cores of
processor : 7
vendor_id : GenuineIntel
cpu family : 6
model : 94
model name : Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
stepping : 3
microcode : 0xc2
cpu MHz : 3824.280
cache size : 8192 KB
physical id : 0
siblings : 8
core id : 3
cpu cores : 4
apicid : 7
initial apicid : 7
fpu : yes
fpu_exception : yes
cpuid level : 22
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov 
pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx 
pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl 
xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor 
ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 
x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm 
abm 3dnowprefetch cpuid_fault invpcid_single pti ibrs ibpb stibp 
tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 
hle avx2 smep bmi2 erms invpcid rtm mpx rdseed adx smap clflushopt 
intel_pt xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts hwp 
hwp_notify hwp_act_window hwp_epp
vmx flags : vnmi preemption_timer invvpid ept_x_only ept_ad ept_1gb 
flexpriority tsc_offset vtpr mtf vapic ept vpid unrestricted_guest ple 
shadow_vmcs pml
bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds 
swapgs taa itlb_multihit
bogomips : 6802.13
clflush size : 64
cache_alignment : 64
address sizes : 39 bits physical, 48 bits virtual
power management:

[8.3.] Module information (from /proc/modules):
lzo 16384 8 - Live 0xffffffffc09ba000
zram 32768 1 - Live 0xffffffffc09ac000
zsmalloc 24576 1 zram, Live 0xffffffffc099d000
binfmt_misc 20480 1 - Live 0xffffffffc093a000
x86_pkg_temp_thermal 16384 0 - Live 0xffffffffc09a7000
coretemp 16384 0 - Live 0xffffffffc0935000
joydev 24576 0 - Live 0xffffffffc092e000
input_leds 16384 0 - Live 0xffffffffc0929000
led_class 16384 1 input_leds, Live 0xffffffffc0924000
rtc_cmos 24576 1 - Live 0xffffffffc091d000
i2c_i801 28672 0 - Live 0xffffffffc0910000
nfs 290816 0 - Live 0xffffffffc09ea000
lockd 94208 1 nfs, Live 0xffffffffc09c9000
grace 16384 1 lockd, Live 0xffffffffc09c4000
sunrpc 376832 2 nfs,lockd, Live 0xffffffffc0940000
btrfs 1376256 1 - Live 0xffffffffc07bf000
zstd_decompress 81920 1 btrfs, Live 0xffffffffc07a5000
zstd_compress 184320 1 btrfs, Live 0xffffffffc0772000
xxhash 16384 2 zstd_decompress,zstd_compress, Live 0xffffffffc074d000
lzo_compress 16384 2 lzo,btrfs, Live 0xffffffffc0498000
multipath 16384 0 - Live 0xffffffffc076d000
linear 16384 0 - Live 0xffffffffc0768000
raid10 57344 0 - Live 0xffffffffc0754000
raid456 143360 0 - Live 0xffffffffc0729000
async_raid6_recov 16384 1 raid456, Live 0xffffffffc0724000
async_memcpy 16384 1 raid456, Live 0xffffffffc071f000
async_pq 16384 1 raid456, Live 0xffffffffc06f5000
async_xor 16384 2 raid456,async_pq, Live 0xffffffffc05ee000
async_tx 16384 5 
raid456,async_raid6_recov,async_memcpy,async_pq,async_xor, Live 
0xffffffffc05b3000
xor 24576 2 btrfs,async_xor, Live 0xffffffffc05cf000
raid6_pq 118784 4 btrfs,raid456,async_raid6_recov,async_pq, Live 
0xffffffffc0701000
raid1 45056 0 - Live 0xffffffffc05a7000
raid0 20480 0 - Live 0xffffffffc06fb000
dm_zero 16384 0 - Live 0xffffffffc0181000
dm_snapshot 53248 0 - Live 0xffffffffc06e7000
dm_mirror 28672 0 - Live 0xffffffffc06df000
dm_region_hash 20480 1 dm_mirror, Live 0xffffffffc06d9000
dm_log_writes 20480 0 - Live 0xffffffffc06d3000
dm_log 20480 2 dm_mirror,dm_region_hash, Live 0xffffffffc06cd000
dm_crypt 45056 0 - Live 0xffffffffc06c1000
dm_bufio 28672 1 dm_snapshot, Live 0xffffffffc06b9000
dm_mod 135168 7 
dm_zero,dm_snapshot,dm_mirror,dm_log_writes,dm_log,dm_crypt,dm_bufio, 
Live 0xffffffffc0697000
dax 28672 1 dm_mod, Live 0xffffffffc059f000
sl811_hcd 28672 0 - Live 0xffffffffc0597000
usbhid 40960 0 - Live 0xffffffffc048d000
uas 28672 0 - Live 0xffffffffc0485000
xhci_plat_hcd 16384 0 - Live 0xffffffffc011d000
pata_sl82c105 16384 0 - Live 0xffffffffc0692000
pata_via 20480 0 - Live 0xffffffffc068c000
pata_jmicron 16384 0 - Live 0xffffffffc0687000
pata_marvell 16384 0 - Live 0xffffffffc039e000
pata_netcell 16384 0 - Live 0xffffffffc0352000
pata_pdc202xx_old 16384 0 - Live 0xffffffffc0205000
pata_triflex 16384 0 - Live 0xffffffffc01a8000
pata_atiixp 16384 0 - Live 0xffffffffc067e000
pata_opti 16384 0 - Live 0xffffffffc0676000
pata_amd 24576 0 - Live 0xffffffffc066b000
pata_ali 20480 0 - Live 0xffffffffc0661000
pata_it8213 16384 0 - Live 0xffffffffc0659000
pata_ns87415 16384 0 - Live 0xffffffffc0651000
pata_ns87410 16384 0 - Live 0xffffffffc0649000
pata_serverworks 16384 0 - Live 0xffffffffc0640000
pata_artop 16384 0 - Live 0xffffffffc063b000
pata_it821x 16384 0 - Live 0xffffffffc0632000
pata_optidma 16384 0 - Live 0xffffffffc062d000
pata_hpt3x2n 16384 0 - Live 0xffffffffc0628000
pata_hpt3x3 16384 0 - Live 0xffffffffc0623000
pata_hpt37x 24576 0 - Live 0xffffffffc061c000
pata_hpt366 16384 0 - Live 0xffffffffc0617000
pata_cmd64x 16384 0 - Live 0xffffffffc0612000
pata_efar 16384 0 - Live 0xffffffffc060d000
pata_rz1000 16384 0 - Live 0xffffffffc0608000
pata_sil680 16384 0 - Live 0xffffffffc0603000
pata_radisys 16384 0 - Live 0xffffffffc05fe000
pata_pdc2027x 16384 0 - Live 0xffffffffc05f9000
pata_mpiix 16384 0 - Live 0xffffffffc05f4000
aic94xx 90112 0 - Live 0xffffffffc05d7000
libsas 86016 1 aic94xx, Live 0xffffffffc05b9000
lpfc 974848 0 - Live 0xffffffffc04a8000
nvmet_fc 32768 1 lpfc, Live 0xffffffffc049f000
qla2xxx 860160 0 - Live 0xffffffffc03b2000
megaraid_mbox 40960 0 - Live 0xffffffffc03a7000
megaraid_mm 20480 1 megaraid_mbox, Live 0xffffffffc02ff000
aacraid 135168 0 - Live 0xffffffffc037c000
3w_9xxx 49152 0 - Live 0xffffffffc036f000
3w_xxxx 32768 0 - Live 0xffffffffc0366000
mptsas 49152 0 - Live 0xffffffffc0359000
scsi_transport_sas 40960 3 aic94xx,libsas,mptsas, Live 0xffffffffc0347000
mptfc 20480 0 - Live 0xffffffffc02df000
atp870u 28672 0 - Live 0xffffffffc02b2000
dc395x 45056 0 - Live 0xffffffffc033b000
qla1280 32768 0 - Live 0xffffffffc0263000
dmx3191d 24576 0 - Live 0xffffffffc021d000
sym53c8xx 90112 0 - Live 0xffffffffc0324000
gdth 90112 0 - Live 0xffffffffc030d000
initio 28672 0 - Live 0xffffffffc0305000
BusLogic 32768 0 - Live 0xffffffffc02f6000
arcmsr 53248 0 - Live 0xffffffffc02e8000
aic7xxx 143360 0 - Live 0xffffffffc02bb000
aic79xx 147456 0 - Live 0xffffffffc028d000
sg 36864 0 - Live 0xffffffffc027e000
megaraid 45056 0 - Live 0xffffffffc026d000
megaraid_sas 159744 0 - Live 0xffffffffc023b000
mptspi 24576 0 - Live 0xffffffffc0230000
mptscsih 28672 3 mptsas,mptfc,mptspi, Live 0xffffffffc0224000
mptbase 69632 4 mptsas,mptfc,mptspi,mptscsih, Live 0xffffffffc020b000
scsi_transport_spi 28672 5 dmx3191d,sym53c8xx,aic7xxx,aic79xx,mptspi, 
Live 0xffffffffc01fd000
pdc_adma 16384 0 - Live 0xffffffffc01f4000
sata_inic162x 16384 0 - Live 0xffffffffc01eb000
sata_mv 36864 0 - Live 0xffffffffc01dc000
sata_qstor 16384 0 - Live 0xffffffffc01d3000
sata_vsc 16384 0 - Live 0xffffffffc01cb000
sata_uli 16384 0 - Live 0xffffffffc01c6000
sata_sis 16384 0 - Live 0xffffffffc01c1000
pata_sis 20480 1 sata_sis, Live 0xffffffffc01b7000
sata_sx4 20480 0 - Live 0xffffffffc01ad000
sata_nv 32768 0 - Live 0xffffffffc019f000
sata_via 20480 0 - Live 0xffffffffc0195000
sata_svw 16384 0 - Live 0xffffffffc0190000
sata_sil24 24576 0 - Live 0xffffffffc0189000
sata_sil 16384 0 - Live 0xffffffffc0171000
sata_promise 20480 0 - Live 0xffffffffc0144000
ata_piix 36864 0 - Live 0xffffffffc0107000
ahci 40960 0 - Live 0xffffffffc0176000
libahci 36864 1 ahci, Live 0xffffffffc0167000
nvme_fc 40960 2 lpfc,qla2xxx, Live 0xffffffffc0157000
nvme_loop 20480 0 - Live 0xffffffffc014d000
nvmet 86016 2 nvmet_fc,nvme_loop, Live 0xffffffffc012e000
nvme_fabrics 24576 2 nvme_fc,nvme_loop, Live 0xffffffffc0123000
nvme 40960 0 - Live 0xffffffffc0112000
nvme_core 94208 4 nvme_fc,nvme_loop,nvme_fabrics,nvme, Live 
0xffffffffc00ef000
scsi_transport_fc 61440 3 lpfc,qla2xxx,mptfc, Live 0xffffffffc00df000
virtio_crypto 24576 0 - Live 0xffffffffc00d4000
crypto_engine 16384 1 virtio_crypto, Live 0xffffffffc00cc000
virtio_input 16384 0 - Live 0xffffffffc00c4000

[8.7.] Tested on two different desktop machines, AMD and Intel, and a 
raspberry pi,
bug present on all. Desktops are one stable desktop machine, and one 
minimal install
specifically for testing. Raspberry pi is a pi 4, running rasbian, with 
zram built
from latest

View attachment "kernel-config" of type "text/plain" (141647 bytes)

View attachment "dmesg.txt" of type "text/plain" (150285 bytes)

Powered by blists - more mailing lists