Kernel memory leak in version 004

[vojtad]

We've upgraded to 004 version a month ago and since that upgrade we've seen a lot of weird memory issues.

After weeks of debugging I am sure it is because of 004 version. When I downgraded to 003 all went back to normal.

These memory issues caused kernel dynamic memory reported by smem -w to grow steadily. In the end the server always froze completely because OOM killer was not able to kill any process to free memory. Unfortunately, I have no more debug details from kernel since I am not able to compile custom kernel on this server.

We are running Debian Buster with this kernel:

$ uname   -a
Linux starkiller 4.19.0-12-amd64 #1 SMP Debian 4.19.152-1 (2020-10-18) x86_64 GNU/Linux

I've also tried upgrading kernel from Debian Buster backports but it didn't help.

Our network card details from lspci:

09:00.0 Ethernet controller: Tehuti Networks Ltd. TN9510 10GBase-T/NBASE-T Ethernet Adapter
	Subsystem: Tehuti Networks Ltd. Ethernet Adapter
	Flags: bus master, fast devsel, latency 0, IRQ 119, NUMA node 0
	Memory at ba000000 (64-bit, prefetchable) [size=64K]
	Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit+
	Capabilities: [78] Power Management version 3
	Capabilities: [80] Express Endpoint, MSI 00
	Capabilities: [100] Virtual Channel
	Kernel driver in use: tn40xx
	Kernel modules: tn40xx

[leo5593]

I had the same issue with the pex10000sfp card on the Ubuntu version 20.04.

There is a pull request to the 003 branch to solve kernel error below : #44

[34668.364735] Driver unloaded [34694.919695] Tehuti Network Driver, 0.3.6.17.2 [34694.919698] Supported phys : QT2025 TLK10232 AQR105 MUSTANG [34694.919931] srom 0x0 HWver 16 build 0 lane# 4 max_pl 0x1 mrrs 0x2 [34695.152404] PHY detected on port 0 ID=40005100 - TLK10232 10Gbps SFP+ [34695.152407] PHY type by svid 7 found 6 [34695.269758] ------------[ cut here ]------------ [34695.269765] WARNING: CPU: 4 PID: 14879 at net/ethtool/common.c:348 ethtool_check_ops+0x18/0x30 [34695.269766] Modules linked in: tn40xx(OE+) iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi md4 cmac nls_utf8 cifs libarc4 fscache libdes snd_soc_skl_hda_dsp ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 xt_tcpudp xt_state xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 snd_hda_codec_hdmi libcrc32c snd_soc_hdac_hdmi snd_hda_codec_realtek iptable_filter bpfilter snd_hda_codec_generic snd_soc_dmic snd_sof_pci snd_sof_intel_byt snd_sof_intel_ipc snd_sof_intel_hda_common snd_soc_hdac_hda snd_sof_xtensa_dsp snd_sof_intel_hda snd_sof snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi ledtrig_audio snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel snd_intel_dspcfg intel_rapl_msr intel_rapl_common snd_hda_codec x86_pkg_temp_thermal intel_powerclamp snd_hda_core snd_hwdep snd_pcm snd_seq_midi coretemp snd_seq_midi_event snd_rawmidi kvm_intel kvm snd_seq snd_seq_device crct10dif_pclmul ghash_clmulni_intel aesni_intel crypto_simd snd_timer cryptd glue_helper [34695.269784] rapl mei_hdcp i915 intel_cstate snd wmi_bmof intel_wmi_thunderbolt ee1004 drm_kms_helper soundcore cec rc_core i2c_algo_bit fb_sys_fops mei_me syscopyarea sysfillrect mei sysimgblt mac_hid acpi_pad sch_fq_codel parport_pc ppdev lp parport drm ip_tables x_tables autofs4 uas usb_storage crc32_pclmul e1000e r8169 i2c_i801 ahci xhci_pci i2c_smbus realtek libahci xhci_pci_renesas wmi video [last unloaded: tn40xx] [34695.269797] CPU: 4 PID: 14879 Comm: insmod Tainted: G OE 5.8.0-53-generic #60~20.04.1-Ubuntu [34695.269798] Hardware name: LENOVO 10SQ003JSP/312A, BIOS M1UKT16A 06/15/2018 [34695.269799] RIP: 0010:ethtool_check_ops+0x18/0x30 [34695.269801] Code: 41 bd f4 ff ff ff eb e4 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 31 c0 48 83 7f 70 00 48 89 e5 74 0d 8b 17 85 d2 75 07 <0f> 0b b8 ea ff ff ff 5d c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f [34695.269802] RSP: 0018:ffffa907c0adb960 EFLAGS: 00010246 [34695.269803] RAX: 0000000000000000 RBX: ffff912013878000 RCX: ffff911fb5f86130 [34695.269803] RDX: 0000000000000000 RSI: ffffa907c1a61270 RDI: ffffffffc10dd000 [34695.269804] RBP: ffffa907c0adb960 R08: 0000000000000000 R09: 0000000000000000 [34695.269805] R10: 0000000000000000 R11: 0000000000000000 R12: ffff911fd5c45000 [34695.269805] R13: ffffa907c01a7000 R14: ffffffff82dfdfc0 R15: ffff911fd5c45000 [34695.269806] FS: 00007f772b221540(0000) GS:ffff912016300000(0000) knlGS:0000000000000000 [34695.269807] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [34695.269808] CR2: 00007fb92aa25940 CR3: 0000000051274005 CR4: 00000000003606e0 [34695.269808] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [34695.269809] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [34695.269809] Call Trace: [34695.269814] register_netdevice+0x71/0x5f0 [34695.269816] ? _cond_resched+0x19/0x30 [34695.269817] register_netdev+0x1f/0x40 [34695.269823] bdx_probe+0x28d/0x920 [tn40xx] [34695.269827] local_pci_probe+0x48/0x80 [34695.269828] pci_device_probe+0x10f/0x1b0 [34695.269830] really_probe+0x159/0x3d0 [34695.269831] driver_probe_device+0xe9/0x160 [34695.269833] device_driver_attach+0x5d/0x70 [34695.269834] __driver_attach+0x8f/0x150 [34695.269835] ? device_driver_attach+0x70/0x70 [34695.269836] bus_for_each_dev+0x7e/0xc0 [34695.269837] driver_attach+0x1e/0x20 [34695.269838] bus_add_driver+0x152/0x1f0 [34695.269839] driver_register+0x74/0xd0 [34695.269840] __pci_register_driver+0x54/0x60 [34695.269843] bdx_module_init.cold+0x79/0x82 [tn40xx] [34695.269846] ? bdx_tx_timeout+0x30/0x30 [tn40xx] [34695.269848] do_one_initcall+0x4a/0x200 [34695.269850] ? _cond_resched+0x19/0x30 [34695.269852] ? kmem_cache_alloc_trace+0x177/0x240 [34695.269854] do_init_module+0x62/0x250 [34695.269855] load_module+0x10f5/0x12a0 [34695.269857] ? ima_post_read_file+0x108/0x120 [34695.269859] __do_sys_finit_module+0xc9/0x130 [34695.269860] ? __do_sys_finit_module+0xc9/0x130 [34695.269861] __x64_sys_finit_module+0x1a/0x20 [34695.269863] do_syscall_64+0x49/0xc0 [34695.269864] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [34695.269865] RIP: 0033:0x7f772b36689d [34695.269867] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d c3 f5 0c 00 f7 d8 64 89 01 48 [34695.269867] RSP: 002b:00007ffed2b91ff8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [34695.269868] RAX: ffffffffffffffda RBX: 000055eb50a977b0 RCX: 00007f772b36689d [34695.269869] RDX: 0000000000000000 RSI: 000055eb4fa1d358 RDI: 0000000000000003 [34695.269869] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007f772b43a260 [34695.269870] R10: 0000000000000003 R11: 0000000000000246 R12: 000055eb4fa1d358 [34695.269871] R13: 0000000000000000 R14: 000055eb50a97780 R15: 0000000000000000 [34695.269872] ---[ end trace 631f61e2f48e4570 ]--- [34695.269873] register_netdev failed [34695.269881] ------------[ cut here ]------------ [34695.269883] WARNING: CPU: 4 PID: 14879 at net/core/dev.c:8934 rollback_registered_many+0xb0/0x510 [34695.269884] Modules linked in: tn40xx(OE+) iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi md4 cmac nls_utf8 cifs libarc4 fscache libdes snd_soc_skl_hda_dsp ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 xt_tcpudp xt_state xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 snd_hda_codec_hdmi libcrc32c snd_soc_hdac_hdmi snd_hda_codec_realtek iptable_filter bpfilter snd_hda_codec_generic snd_soc_dmic snd_sof_pci snd_sof_intel_byt snd_sof_intel_ipc snd_sof_intel_hda_common snd_soc_hdac_hda snd_sof_xtensa_dsp snd_sof_intel_hda snd_sof snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi ledtrig_audio snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel snd_intel_dspcfg intel_rapl_msr intel_rapl_common snd_hda_codec x86_pkg_temp_thermal intel_powerclamp snd_hda_core snd_hwdep snd_pcm snd_seq_midi coretemp snd_seq_midi_event snd_rawmidi kvm_intel kvm snd_seq snd_seq_device crct10dif_pclmul ghash_clmulni_intel aesni_intel crypto_simd snd_timer cryptd glue_helper [34695.269892] rapl mei_hdcp i915 intel_cstate snd wmi_bmof intel_wmi_thunderbolt ee1004 drm_kms_helper soundcore cec rc_core i2c_algo_bit fb_sys_fops mei_me syscopyarea sysfillrect mei sysimgblt mac_hid acpi_pad sch_fq_codel parport_pc ppdev lp parport drm ip_tables x_tables autofs4 uas usb_storage crc32_pclmul e1000e r8169 i2c_i801 ahci xhci_pci i2c_smbus realtek libahci xhci_pci_renesas wmi video [last unloaded: tn40xx] [34695.269899] CPU: 4 PID: 14879 Comm: insmod Tainted: G W OE 5.8.0-53-generic #60~20.04.1-Ubuntu [34695.269900] Hardware name: LENOVO 10SQ003JSP/312A, BIOS M1UKT16A 06/15/2018 [34695.269901] RIP: 0010:rollback_registered_many+0xb0/0x510 [34695.269902] Code: 69 49 8b 44 24 68 49 8d 54 24 68 4c 89 e3 48 83 e8 68 49 39 d5 74 55 49 89 c4 0f b6 83 d8 04 00 00 84 c0 75 d1 0f 1f 44 00 00 <0f> 0b 48 8b 53 68 48 8b 43 70 48 89 42 08 48 89 10 4c 89 43 68 4c [34695.269903] RSP: 0018:ffffa907c0adb908 EFLAGS: 00010246 [34695.269904] RAX: 0000000000000000 RBX: ffff911fd5c45000 RCX: ffffa907c0adb978 [34695.269904] RDX: ffff911fc2d33000 RSI: 0000000000000000 RDI: ffffffff82e00000 [34695.269905] RBP: ffffa907c0adb968 R08: dead000000000100 R09: ffffffff831cd264 [34695.269905] R10: ffffffff831b0988 R11: ffffa907c0adb848 R12: ffffa907c0adb910 [34695.269906] R13: ffffa907c0adb978 R14: ffffa907c0adb908 R15: dead000000000122 [34695.269907] FS: 00007f772b221540(0000) GS:ffff912016300000(0000) knlGS:0000000000000000 [34695.269907] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [34695.269908] CR2: 00007fb92aa25940 CR3: 0000000051274005 CR4: 00000000003606e0 [34695.269909] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [34695.269909] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [34695.269910] Call Trace: [34695.269911] ? vprintk_default+0x1f/0x30 [34695.269913] unregister_netdevice_queue+0x94/0x120 [34695.269914] unregister_netdev+0x1d/0x30 [34695.269917] bdx_probe+0x31a/0x920 [tn40xx] [34695.269919] local_pci_probe+0x48/0x80 [34695.269920] pci_device_probe+0x10f/0x1b0 [34695.269922] really_probe+0x159/0x3d0 [34695.269923] driver_probe_device+0xe9/0x160 [34695.269924] device_driver_attach+0x5d/0x70 [34695.269925] __driver_attach+0x8f/0x150 [34695.269926] ? device_driver_attach+0x70/0x70 [34695.269927] bus_for_each_dev+0x7e/0xc0 [34695.269928] driver_attach+0x1e/0x20 [34695.269929] bus_add_driver+0x152/0x1f0 [34695.269930] driver_register+0x74/0xd0 [34695.269932] __pci_register_driver+0x54/0x60 [34695.269935] bdx_module_init.cold+0x79/0x82 [tn40xx] [34695.269937] ? bdx_tx_timeout+0x30/0x30 [tn40xx] [34695.269939] do_one_initcall+0x4a/0x200 [34695.269940] ? _cond_resched+0x19/0x30 [34695.269942] ? kmem_cache_alloc_trace+0x177/0x240 [34695.269943] do_init_module+0x62/0x250 [34695.269944] load_module+0x10f5/0x12a0 [34695.269945] ? ima_post_read_file+0x108/0x120 [34695.269947] __do_sys_finit_module+0xc9/0x130 [34695.269948] ? __do_sys_finit_module+0xc9/0x130 [34695.269949] __x64_sys_finit_module+0x1a/0x20 [34695.269950] do_syscall_64+0x49/0xc0 [34695.269951] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [34695.269952] RIP: 0033:0x7f772b36689d [34695.269953] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d c3 f5 0c 00 f7 d8 64 89 01 48 [34695.269954] RSP: 002b:00007ffed2b91ff8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [34695.269954] RAX: ffffffffffffffda RBX: 000055eb50a977b0 RCX: 00007f772b36689d [34695.269955] RDX: 0000000000000000 RSI: 000055eb4fa1d358 RDI: 0000000000000003 [34695.269956] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007f772b43a260 [34695.269956] R10: 0000000000000003 R11: 0000000000000246 R12: 000055eb4fa1d358 [34695.269957] R13: 0000000000000000 R14: 000055eb50a97780 R15: 0000000000000000 [34695.269958] ---[ end trace 631f61e2f48e4571 ]--- [34695.270086] network todo 'eth%d' but state 0 [34695.270088] CPU: 4 PID: 14879 Comm: insmod Tainted: G W OE 5.8.0-53-generic #60~20.04.1-Ubuntu [34695.270089] Hardware name: LENOVO 10SQ003JSP/312A, BIOS M1UKT16A 06/15/2018 [34695.270089] Call Trace: [34695.270092] dump_stack+0x74/0x92 [34695.270094] netdev_run_todo.cold+0x34/0x39 [34695.270095] rtnl_unlock+0xe/0x10 [34695.270097] unregister_netdev+0x22/0x30 [34695.270100] bdx_probe+0x31a/0x920 [tn40xx] [34695.270102] local_pci_probe+0x48/0x80 [34695.270104] pci_device_probe+0x10f/0x1b0 [34695.270105] really_probe+0x159/0x3d0 [34695.270106] driver_probe_device+0xe9/0x160 [34695.270108] device_driver_attach+0x5d/0x70 [34695.270109] __driver_attach+0x8f/0x150 [34695.270110] ? device_driver_attach+0x70/0x70 [34695.270111] bus_for_each_dev+0x7e/0xc0 [34695.270112] driver_attach+0x1e/0x20 [34695.270113] bus_add_driver+0x152/0x1f0 [34695.270114] driver_register+0x74/0xd0 [34695.270115] __pci_register_driver+0x54/0x60 [34695.270119] bdx_module_init.cold+0x79/0x82 [tn40xx] [34695.270121] ? bdx_tx_timeout+0x30/0x30 [tn40xx] [34695.270123] do_one_initcall+0x4a/0x200 [34695.270124] ? _cond_resched+0x19/0x30 [34695.270126] ? kmem_cache_alloc_trace+0x177/0x240 [34695.270127] do_init_module+0x62/0x250 [34695.270128] load_module+0x10f5/0x12a0 [34695.270129] ? ima_post_read_file+0x108/0x120 [34695.270131] __do_sys_finit_module+0xc9/0x130 [34695.270132] ? __do_sys_finit_module+0xc9/0x130 [34695.270134] __x64_sys_finit_module+0x1a/0x20 [34695.270135] do_syscall_64+0x49/0xc0 [34695.270136] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [34695.270137] RIP: 0033:0x7f772b36689d [34695.270138] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d c3 f5 0c 00 f7 d8 64 89 01 48 [34695.270139] RSP: 002b:00007ffed2b91ff8 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [34695.270140] RAX: ffffffffffffffda RBX: 000055eb50a977b0 RCX: 00007f772b36689d [34695.270141] RDX: 0000000000000000 RSI: 000055eb4fa1d358 RDI: 0000000000000003 [34695.270141] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007f772b43a260 [34695.270142] R10: 0000000000000003 R11: 0000000000000246 R12: 000055eb4fa1d358 [34695.270143] R13: 0000000000000000 R14: 000055eb50a97780 R15: 0000000000000000 [34695.284465] tn40xx: probe of 0000:01:00.0 failed with error -22 [35400.332865] Driver unloaded [39615.707576] perf: interrupt took too long (3129 > 3126), lowering kernel.perf_event_max_sample_rate to 63750 [47317.367872] CIFS: Attempting to mount //10.130.15.11/GTS-BCK [51489.096701] audit: type=1400 audit(1622584808.183:59): apparmor="DENIED" operation="capable" profile="/usr/sbin/cups-browsed" pid=16148 comm="cups-browsed" capability=23 capname="sys_nice" [83047.812332] CIFS: Attempting to mount //172.26.130.2/b [83171.663264] CIFS: Attempting to mount //172.26.131.80/e [84247.148884] Tehuti Network Driver, 0.3.6.17.2 [84247.148887] Supported phys : QT2025 TLK10232 AQR105 MUSTANG [84247.148992] srom 0x0 HWver 16 build 0 lane# 4 max_pl 0x1 mrrs 0x2 [84247.385279] PHY detected on port 0 ID=40005100 - TLK10232 10Gbps SFP+ [84247.385285] PHY type by svid 7 found 6 [84247.499864] ------------[ cut here ]------------ [84247.499871] WARNING: CPU: 3 PID: 20423 at net/ethtool/common.c:348 ethtool_check_ops+0x18/0x30 [84247.499872] Modules linked in: tn40xx(OE+) iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi md4 cmac nls_utf8 cifs libarc4 fscache libdes snd_soc_skl_hda_dsp ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 xt_tcpudp xt_state xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 snd_hda_codec_hdmi libcrc32c snd_soc_hdac_hdmi snd_hda_codec_realtek iptable_filter bpfilter snd_hda_codec_generic snd_soc_dmic snd_sof_pci snd_sof_intel_byt snd_sof_intel_ipc snd_sof_intel_hda_common snd_soc_hdac_hda snd_sof_xtensa_dsp snd_sof_intel_hda snd_sof snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi ledtrig_audio snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel snd_intel_dspcfg intel_rapl_msr intel_rapl_common snd_hda_codec x86_pkg_temp_thermal intel_powerclamp snd_hda_core snd_hwdep snd_pcm snd_seq_midi coretemp snd_seq_midi_event snd_rawmidi kvm_intel kvm snd_seq snd_seq_device crct10dif_pclmul ghash_clmulni_intel aesni_intel crypto_simd snd_timer cryptd glue_helper [84247.499889] rapl mei_hdcp i915 intel_cstate snd wmi_bmof intel_wmi_thunderbolt ee1004 drm_kms_helper soundcore cec rc_core i2c_algo_bit fb_sys_fops mei_me syscopyarea sysfillrect mei sysimgblt mac_hid acpi_pad sch_fq_codel parport_pc ppdev lp parport drm ip_tables x_tables autofs4 uas usb_storage crc32_pclmul e1000e r8169 i2c_i801 ahci xhci_pci i2c_smbus realtek libahci xhci_pci_renesas wmi video [last unloaded: tn40xx] [84247.499901] CPU: 3 PID: 20423 Comm: insmod Tainted: G W OE 5.8.0-53-generic #60~20.04.1-Ubuntu [84247.499901] Hardware name: LENOVO 10SQ003JSP/312A, BIOS M1UKT16A 06/15/2018 [84247.499903] RIP: 0010:ethtool_check_ops+0x18/0x30 [84247.499905] Code: 41 bd f4 ff ff ff eb e4 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 55 31 c0 48 83 7f 70 00 48 89 e5 74 0d 8b 17 85 d2 75 07 <0f> 0b b8 ea ff ff ff 5d c3 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f [84247.499905] RSP: 0018:ffffa907c34d7960 EFLAGS: 00010246 [84247.499906] RAX: 0000000000000000 RBX: ffff912013878000 RCX: ffff91200a422e50 [84247.499907] RDX: 0000000000000000 RSI: ffffa907c15c1270 RDI: ffffffffc10fb000 [84247.499908] RBP: ffffa907c34d7960 R08: 0000000000000000 R09: 0000000000000000 [84247.499908] R10: 0000000000000000 R11: 0000000000000000 R12: ffff91200cd6d000 [84247.499909] R13: ffffa907c01a7000 R14: ffffffff82dfdfc0 R15: ffff91200cd6d000 [84247.499910] FS: 00007fc9ee3be540(0000) GS:ffff9120162c0000(0000) knlGS:0000000000000000 [84247.499911] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [84247.499911] CR2: 000055cc4baac700 CR3: 00000000029ac005 CR4: 00000000003606e0 [84247.499912] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [84247.499913] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [84247.499913] Call Trace: [84247.499917] register_netdevice+0x71/0x5f0 [84247.499919] ? _cond_resched+0x19/0x30 [84247.499920] register_netdev+0x1f/0x40 [84247.499926] bdx_probe+0x28d/0x920 [tn40xx] [84247.499929] local_pci_probe+0x48/0x80 [84247.499931] pci_device_probe+0x10f/0x1b0 [84247.499933] really_probe+0x159/0x3d0 [84247.499934] driver_probe_device+0xe9/0x160 [84247.499935] device_driver_attach+0x5d/0x70 [84247.499936] __driver_attach+0x8f/0x150 [84247.499938] ? device_driver_attach+0x70/0x70 [84247.499938] bus_for_each_dev+0x7e/0xc0 [84247.499940] driver_attach+0x1e/0x20 [84247.499941] bus_add_driver+0x152/0x1f0 [84247.499942] driver_register+0x74/0xd0 [84247.499943] __pci_register_driver+0x54/0x60 [84247.499946] bdx_module_init.cold+0x79/0x82 [tn40xx] [84247.499949] ? bdx_tx_timeout+0x30/0x30 [tn40xx] [84247.499951] do_one_initcall+0x4a/0x200 [84247.499953] ? _cond_resched+0x19/0x30 [84247.499955] ? kmem_cache_alloc_trace+0x177/0x240 [84247.499957] do_init_module+0x62/0x250 [84247.499958] load_module+0x10f5/0x12a0 [84247.499959] ? ima_post_read_file+0x108/0x120 [84247.499961] __do_sys_finit_module+0xc9/0x130 [84247.499962] ? __do_sys_finit_module+0xc9/0x130 [84247.499964] __x64_sys_finit_module+0x1a/0x20 [84247.499965] do_syscall_64+0x49/0xc0 [84247.499967] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [84247.499968] RIP: 0033:0x7fc9ee50389d [84247.499969] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d c3 f5 0c 00 f7 d8 64 89 01 48 [84247.499970] RSP: 002b:00007ffeeb827a58 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [84247.499971] RAX: ffffffffffffffda RBX: 000055cc4d1637b0 RCX: 00007fc9ee50389d [84247.499971] RDX: 0000000000000000 RSI: 000055cc4bab7358 RDI: 0000000000000003 [84247.499972] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007fc9ee5d7260 [84247.499973] R10: 0000000000000003 R11: 0000000000000246 R12: 000055cc4bab7358 [84247.499973] R13: 0000000000000000 R14: 000055cc4d163780 R15: 0000000000000000 [84247.499974] ---[ end trace 631f61e2f48e4572 ]--- [84247.499976] register_netdev failed [84247.499983] ------------[ cut here ]------------ [84247.499986] WARNING: CPU: 3 PID: 20423 at net/core/dev.c:8934 rollback_registered_many+0xb0/0x510 [84247.499986] Modules linked in: tn40xx(OE+) iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi md4 cmac nls_utf8 cifs libarc4 fscache libdes snd_soc_skl_hda_dsp ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 xt_tcpudp xt_state xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 snd_hda_codec_hdmi libcrc32c snd_soc_hdac_hdmi snd_hda_codec_realtek iptable_filter bpfilter snd_hda_codec_generic snd_soc_dmic snd_sof_pci snd_sof_intel_byt snd_sof_intel_ipc snd_sof_intel_hda_common snd_soc_hdac_hda snd_sof_xtensa_dsp snd_sof_intel_hda snd_sof snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi ledtrig_audio snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine snd_hda_intel snd_intel_dspcfg intel_rapl_msr intel_rapl_common snd_hda_codec x86_pkg_temp_thermal intel_powerclamp snd_hda_core snd_hwdep snd_pcm snd_seq_midi coretemp snd_seq_midi_event snd_rawmidi kvm_intel kvm snd_seq snd_seq_device crct10dif_pclmul ghash_clmulni_intel aesni_intel crypto_simd snd_timer cryptd glue_helper [84247.499995] rapl mei_hdcp i915 intel_cstate snd wmi_bmof intel_wmi_thunderbolt ee1004 drm_kms_helper soundcore cec rc_core i2c_algo_bit fb_sys_fops mei_me syscopyarea sysfillrect mei sysimgblt mac_hid acpi_pad sch_fq_codel parport_pc ppdev lp parport drm ip_tables x_tables autofs4 uas usb_storage crc32_pclmul e1000e r8169 i2c_i801 ahci xhci_pci i2c_smbus realtek libahci xhci_pci_renesas wmi video [last unloaded: tn40xx] [84247.500002] CPU: 3 PID: 20423 Comm: insmod Tainted: G W OE 5.8.0-53-generic #60~20.04.1-Ubuntu [84247.500002] Hardware name: LENOVO 10SQ003JSP/312A, BIOS M1UKT16A 06/15/2018 [84247.500004] RIP: 0010:rollback_registered_many+0xb0/0x510 [84247.500005] Code: 69 49 8b 44 24 68 49 8d 54 24 68 4c 89 e3 48 83 e8 68 49 39 d5 74 55 49 89 c4 0f b6 83 d8 04 00 00 84 c0 75 d1 0f 1f 44 00 00 <0f> 0b 48 8b 53 68 48 8b 43 70 48 89 42 08 48 89 10 4c 89 43 68 4c [84247.500005] RSP: 0018:ffffa907c34d7908 EFLAGS: 00010246 [84247.500006] RAX: 0000000000000000 RBX: ffff91200cd6d000 RCX: ffffa907c34d7978 [84247.500007] RDX: ffff911e401e9800 RSI: 0000000000000000 RDI: ffffffff82e00000 [84247.500007] RBP: ffffa907c34d7968 R08: dead000000000100 R09: ffffffff831d0818 [84247.500008] R10: ffffffff831b0988 R11: ffffa907c34d7848 R12: ffffa907c34d7910 [84247.500009] R13: ffffa907c34d7978 R14: ffffa907c34d7908 R15: dead000000000122 [84247.500009] FS: 00007fc9ee3be540(0000) GS:ffff9120162c0000(0000) knlGS:0000000000000000 [84247.500010] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [84247.500011] CR2: 000055cc4baac700 CR3: 00000000029ac005 CR4: 00000000003606e0 [84247.500011] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [84247.500012] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [84247.500012] Call Trace: [84247.500014] ? vprintk_default+0x1f/0x30 [84247.500016] unregister_netdevice_queue+0x94/0x120 [84247.500017] unregister_netdev+0x1d/0x30 [84247.500020] bdx_probe+0x31a/0x920 [tn40xx] [84247.500022] local_pci_probe+0x48/0x80 [84247.500023] pci_device_probe+0x10f/0x1b0 [84247.500025] really_probe+0x159/0x3d0 [84247.500026] driver_probe_device+0xe9/0x160 [84247.500027] device_driver_attach+0x5d/0x70 [84247.500028] __driver_attach+0x8f/0x150 [84247.500029] ? device_driver_attach+0x70/0x70 [84247.500030] bus_for_each_dev+0x7e/0xc0 [84247.500031] driver_attach+0x1e/0x20 [84247.500032] bus_add_driver+0x152/0x1f0 [84247.500033] driver_register+0x74/0xd0 [84247.500035] __pci_register_driver+0x54/0x60 [84247.500038] bdx_module_init.cold+0x79/0x82 [tn40xx] [84247.500040] ? bdx_tx_timeout+0x30/0x30 [tn40xx] [84247.500041] do_one_initcall+0x4a/0x200 [84247.500043] ? _cond_resched+0x19/0x30 [84247.500044] ? kmem_cache_alloc_trace+0x177/0x240 [84247.500045] do_init_module+0x62/0x250 [84247.500047] load_module+0x10f5/0x12a0 [84247.500048] ? ima_post_read_file+0x108/0x120 [84247.500049] __do_sys_finit_module+0xc9/0x130 [84247.500050] ? __do_sys_finit_module+0xc9/0x130 [84247.500052] __x64_sys_finit_module+0x1a/0x20 [84247.500053] do_syscall_64+0x49/0xc0 [84247.500054] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [84247.500055] RIP: 0033:0x7fc9ee50389d [84247.500056] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d c3 f5 0c 00 f7 d8 64 89 01 48 [84247.500056] RSP: 002b:00007ffeeb827a58 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [84247.500057] RAX: ffffffffffffffda RBX: 000055cc4d1637b0 RCX: 00007fc9ee50389d [84247.500058] RDX: 0000000000000000 RSI: 000055cc4bab7358 RDI: 0000000000000003 [84247.500058] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007fc9ee5d7260 [84247.500059] R10: 0000000000000003 R11: 0000000000000246 R12: 000055cc4bab7358 [84247.500060] R13: 0000000000000000 R14: 000055cc4d163780 R15: 0000000000000000 [84247.500061] ---[ end trace 631f61e2f48e4573 ]--- [84247.500149] network todo 'eth%d' but state 0 [84247.500150] CPU: 3 PID: 20423 Comm: insmod Tainted: G W OE 5.8.0-53-generic #60~20.04.1-Ubuntu [84247.500151] Hardware name: LENOVO 10SQ003JSP/312A, BIOS M1UKT16A 06/15/2018 [84247.500151] Call Trace: [84247.500153] dump_stack+0x74/0x92 [84247.500155] netdev_run_todo.cold+0x34/0x39 [84247.500156] rtnl_unlock+0xe/0x10 [84247.500158] unregister_netdev+0x22/0x30 [84247.500160] bdx_probe+0x31a/0x920 [tn40xx] [84247.500162] local_pci_probe+0x48/0x80 [84247.500164] pci_device_probe+0x10f/0x1b0 [84247.500165] really_probe+0x159/0x3d0 [84247.500166] driver_probe_device+0xe9/0x160 [84247.500167] device_driver_attach+0x5d/0x70 [84247.500169] __driver_attach+0x8f/0x150 [84247.500170] ? device_driver_attach+0x70/0x70 [84247.500171] bus_for_each_dev+0x7e/0xc0 [84247.500172] driver_attach+0x1e/0x20 [84247.500173] bus_add_driver+0x152/0x1f0 [84247.500174] driver_register+0x74/0xd0 [84247.500175] __pci_register_driver+0x54/0x60 [84247.500178] bdx_module_init.cold+0x79/0x82 [tn40xx] [84247.500180] ? bdx_tx_timeout+0x30/0x30 [tn40xx] [84247.500182] do_one_initcall+0x4a/0x200 [84247.500183] ? _cond_resched+0x19/0x30 [84247.500185] ? kmem_cache_alloc_trace+0x177/0x240 [84247.500186] do_init_module+0x62/0x250 [84247.500187] load_module+0x10f5/0x12a0 [84247.500188] ? ima_post_read_file+0x108/0x120 [84247.500190] __do_sys_finit_module+0xc9/0x130 [84247.500191] ? __do_sys_finit_module+0xc9/0x130 [84247.500192] __x64_sys_finit_module+0x1a/0x20 [84247.500193] do_syscall_64+0x49/0xc0 [84247.500194] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [84247.500195] RIP: 0033:0x7fc9ee50389d [84247.500196] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d c3 f5 0c 00 f7 d8 64 89 01 48 [84247.500197] RSP: 002b:00007ffeeb827a58 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 [84247.500198] RAX: ffffffffffffffda RBX: 000055cc4d1637b0 RCX: 00007fc9ee50389d [84247.500198] RDX: 0000000000000000 RSI: 000055cc4bab7358 RDI: 0000000000000003 [84247.500199] RBP: 0000000000000000 R08: 0000000000000000 R09: 00007fc9ee5d7260 [84247.500200] R10: 0000000000000003 R11: 0000000000000246 R12: 000055cc4bab7358 [84247.500200] R13: 0000000000000000 R14: 000055cc4d163780 R15: 0000000000000000 [84247.513173] tn40xx: probe of 0000:01:00.0 failed with error -22