Differences

This shows you the differences between two versions of the page.

--- linux:general:troubleshooting [2019/01/24 17:04] – [Cant mount Windowsshares (CIFS) on Linux] lunetikk
+++ linux:general:troubleshooting [2020/12/03 15:12] (current) – [Linux starts in emergency mode - faulty logical volume (xfs)] lunetikk
@@ Line 94: / Line 94: @@
 Finally restart your system and pray...
-Have a look at this website for more xfs_repair related info\\
-[[http://fibrevillage.com/storage/666-how-to-repair-a-xfs-filesystem|fibrevillage.com - How to repair a xfs filesystem]]
 \\
 \\
@@ Line 166: / Line 164: @@
 \\
-==== Cant mount Windowsshares (CIFS) on Linux ====
+==== Bug: soft lockup in messages ====
 === Description ===
-Mounting your share leads to the following error
+You can find multiple "Bug: soft lockup" entries in /var/log/messages or journalctl
 <code>
-mount error(9): Bad file descriptor
+May 25 07:23:59 XXXXXXX kernel: [13445315.881356] BUG: soft lockup - CPU#16 stuck for 23s! [yyyyyyy:81602]
 </code>
 === Reason ===
-Microsoft released an Update (January 8, 2019 (Monthly Rollup)) which broke smb2.0 and 2.1 connections.\\
+>A 'soft lockup' is defined as a bug that causes the kernel to loop in kernel mode for more than 20 seconds without giving other tasks a chance to run. The watchdog daemon will send an non-maskable interrupt (NMI) to all CPUs in the system who, in turn, print the stack traces of their currently running tasks.
+-SUSE KB [[https://www.suse.com/support/kb/doc/?id=7017652|7017652]]
-The issue affect Windows 7 and Windows Server 2008 (R2).
+=== Fix ===
-[[https://www.theregister.co.uk/2019/01/09/windows_7_network_broken/|https://www.theregister.co.uk/2019/01/09/windows_7_network_broken/]]
+__Solution 1:__
-[[https://serverfault.com/questions/948263/kb-4480970-and-or-kb-4480960-breaks-smb2-connections-to-windows-7-shares|https://serverfault.com/questions/948263/kb-4480970-and-or-kb-4480960-breaks-smb2-connections-to-windows-7-shares]]
+Restart your system and/ or decrease your CPU load.
-=== Fix ===
+__Solution 2:__
-Microsoft released a fix to fix their update... m(
+Increase the time (default 10) before soft lockups are fired.
+<code bash >echo 20 > /proc/sys/kernel/watchdog_thresh</code>
+or
+<code bash>
+echo "kernel.watchdog_thresh=20" > /etc/sysctl.d/99-watchdog_thresh.conf
+sysctl -p  /etc/sysctl.d/99-watchdog_thresh.conf
+</code>
+\\
+\\
+==== systemctl runs in timeout ====
+=== Description ===
+In this example, installation of docker-ce with the following command doesnt work
 <code>
-This update resolves the issue where local users who are part of the local “Administrators“ group may not be able to remotely access shares on Windows 7 SP1 and Windows Server 2008 R2 machines after installing the January 8th, 2019 security updates.
+curl -sSL https://get.docker.com | sh
-This does not affect domain accounts in the local "Administrators" group.
+# Executing docker install script, commit: f45d7c11389849ff46a6b4d94e0dd1ffebca32c1
++ sh -c apt-get update -qq >/dev/null
++ sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null
++ sh -c curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" | apt-key add -qq - >/dev/null
++ sh -c echo "deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable" > /etc/apt/sources.list.d/docker.list
++ sh -c apt-get update -qq >/dev/null
++ [ -n  ]
++ sh -c apt-get install -y -qq --no-install-recommends docker-ce >/dev/null
+Broadcast message from systemd-journald@lunetikk (Wed 2019-10-23 00:22:12 CEST):
+systemd[1]: Caught <SEGV>, dumped core as pid 26368.
+Broadcast message from systemd-journald@lunetikk (Wed 2019-10-23 00:22:12 CEST):
+systemd[1]: Freezing execution.
+E: Sub-process /usr/bin/dpkg returned an error code (1)
 </code>
-Download KB4487345 and install the fix: [[http://www.catalog.update.microsoft.com/Search.aspx?q=KB4487345|KB4487345]]
+Rerun "apt-get install docker-ce" shows the following
+<code>
+apt-get install docker-ce
+Reading package lists... Done
+Building dependency tree
+Reading state information... Done
+docker-ce is already the newest version (5:19.03.4~3-0~ubuntu-xenial).
+After this operation, 0 B of additional disk space will be used.
+Do you want to continue? [Y/n]
+Setting up docker-ce (5:19.03.4~3-0~ubuntu-xenial) ...
+Failed to execute operation: Connection timed out
+Failed to execute operation: Connection timed out
+Failed to retrieve unit state: Connection timed out
+Failed to start docker.service: Connection timed out
+See system logs and 'systemctl status docker.service' for details.
+invoke-rc.d: initscript docker, action "start" failed.
+Failed to get properties: Connection timed out
+dpkg: error processing package docker-ce (--configure):
+ subprocess installed post-installation script returned error exit status 1
+Errors were encountered while processing:
+ docker-ce
+E: Sub-process /usr/bin/dpkg returned an error code (1)
+</code>
+You cant reconfigure
+<code>
+dpkg-reconfigure docker-ce
+/usr/sbin/dpkg-reconfigure: docker-ce is broken or not fully installed
+</code>
+Listing the units for "systemctl status" runs in timeout
+<code>
+systemctl status docke<TAB>
+Failed to list unit files: Connection timed out
+Failed to list units: Connection timed out
+Failed to list unit files: Connection timed out
+</code>
+=== Reason ===
 \\
+In my case, my disk was "inconsistent". Reboot got me stuck in busybox. \\
+{{:linux:general:pasted:20191023-005512.png}}\\
+=== Fix ===
+\\
+I was able to run "fsck.ext4 /dev/vda2" to fix the orphaned inodes
+{{:linux:general:pasted:20191023-005629.png}}
+{{:linux:general:pasted:20191023-005648.png}}
+Reboot after this got me back onto my system and "systemctl" was working again.
 \\

Lunetikk's IT Wiki

User Tools

Site Tools

Differences

Page Tools