Lunetikk's IT Wiki

apt-get autoremove leads to a broken symlink which requires a reload of grub

apt-get autoremove
...
The link /vmlinuz.old is a damaged link
Removing symbolic link vmlinuz.old
 you may need to re-run your boot loader[grub]
The link /initrd.img.old is a damaged link
Removing symbolic link initrd.img.old
 you may need to re-run your boot loader[grub]

Broken symlinks

Run “update-grub”

update-grub
 Generating grub configuration file ...
 Found linux image: /boot/vmlinuz-3.13.0-157-generic
 Found initrd image: /boot/initrd.img-3.13.0-157-generic
 Found linux image: /boot/vmlinuz-3.13.0-153-generic
 Found initrd image: /boot/initrd.img-3.13.0-153-generic
 Found memtest86+ image: /boot/memtest86+.elf
 Found memtest86+ image: /boot/memtest86+.bin
 done

After entering your root password and opening “journalctl -xb” you find red entries, something like

kernel: [299102] XFS (dm-2): First 64 bytes of corrupted metadata buffer:
kernel: [299102] ffff880308ed2000: c7 00 00 00 48 89 5c 24 08 48 89 74 24 10 57 48  ....H.\$.H.t$.WH
kernel: [299103] ffff880308ed2010: 83 ec 30 48 8d 54 24 50 e8 03 38 c9 ff 85 c0 0f  ..0H.T$P..8.....
kernel: [299104] ffff880308ed2020: 88 97 00 00 00 48 8b 5c 24 50 48 8d 54 24 58 48  .....H.\$PH.T$XH
kernel: [299104] ffff880308ed2030: 8b cb e8 e1 25 d2 ff 85 c0 78 74 48 8b 7c 24 58  ....%....xtH.|$X
kernel: [299114] XFS (dm-2): Metadata corruption detected at xfs_inode_buf_verify+0x66/0xc0 [xfs], xfs_inode block 0x13bfa0
kernel: [299115] XFS (dm-2): Unmount and run xfs_repair
kernel: [299115] XFS (dm-2): First 64 bytes of corrupted metadata buffer:
kernel: [299116] ffff880308ed2000: c7 00 00 00 48 89 5c 24 08 48 89 74 24 10 57 48  ....H.\$.H.t$.WH
kernel: [299116] ffff880308ed2010: 83 ec 30 48 8d 54 24 50 e8 03 38 c9 ff 85 c0 0f  ..0H.T$P..8.....
kernel: [299117] ffff880308ed2020: 88 97 00 00 00 48 8b 5c 24 50 48 8d 54 24 58 48  .....H.\$PH.T$XH
kernel: [299117] ffff880308ed2030: 8b cb e8 e1 25 d2 ff 85 c0 78 74 48 8b 7c 24 58  ....%....xtH.|$X
kernel: [299189] XFS (dm-2): metadata I/O error: block 0x13bfa0 ("xfs_trans_read_buf_map") error 117 numblks 16
kernel: [299195] XFS (dm-2): xfs_imap_to_bp: xfs_trans_read_buf() returned error -117.
kernel: [299196] XFS (dm-2): xfs_do_force_shutdown(0x8) called from line 3519 of file ../fs/xfs/xfs_inode.c.  Return address = 0xffffffffa02d4192
kernel: [299200] XFS (dm-2): Corruption of in-memory data detected.  Shutting down filesystem
kernel: [299200] XFS (dm-2): Please umount the filesystem and rectify the problem(s)
kernel: [372139] XFS (dm-2): xfs_log_force: error -5 returned

The server might have KVM installed which messed something up…
Serverfault

Check “journalctl -xb” to find out which LV is corrupted
Get the right LV for “dm-X”

dmsetup info /dev/dm-2

then have a look at your disks

df -h

and mounts

mount

the LV should not be mounted.

Try to repair the filesystem

xfs_repair /dev/mapper/VG02-LVdata

If this fails because of the journal log, try resetting the log

xfs_repair -L /dev/mapper/VG02-LVdata

If completed, rerun the first xfs_repair (without -L) again.
If successful, try to mount the device

mount -a

and check your filesystem

df -h
ll /data

Finally restart your system and pray…

Have a look at this website for more xfs_repair related info
fibrevillage.com - How to repair a xfs filesystem

You can connect to your MySQL with 127.0.0.1 but not with localhost

mysql -h127.0.0.1 -uroot -p
#Welcome to the MySQL monitor...

mysql -hlocalhost -uroot -p
#ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock'

MySQL will try to connect to the unix socket if you tell it to connect to localhost. If you tell it to connect to 127.0.0.1 you are forcing it to connect to the network socket.

Verify if the socket is really your problem.

mysql --print-defaults

mysql would have been started with the following arguments:
--port=3306 --socket=/var/run/mysqld/mysqld.sock

mysqld --print-defaults

mysqld would have been started with the following arguments:
--user=mysql --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock --port=3306...

Compare the defaults for “socket”, both should be the same. If you get a different socket for your client, try to connect to your database by using the same as specified for mysqld

mysql --socket=/var/run/mysqld/mysqld.sock -hlocalhost -uroot -p

If that worked, check if you have specified a socket for both, the client and the daemon in your my.cnf

...
[client]
port            = 3306
socket          = /var/run/mysqld/mysqld.sock
...
[mysqld]
port            = 3306
socket          = /var/run/mysqld/mysqld.sock
...

If this is the case, check if your my.cnf is in one of the following folders. The default options are read from these files in the given order:

/etc/mysql/my.cnf 
/etc/my.cnf 
~/.my.cnf       

If you dont have your my.cnf in one of the folders, create a symlink at one of these locations referencing your config file

example:
ln -s /usr/local/mysql/etc/my.cnf /etc/my.cnf

The connection should work now.

Mounting your share leads to the following error

mount error(9): Bad file descriptor

Microsoft released an Update (January 8, 2019 (Monthly Rollup)) which broke smb2.0 and 2.1 connections.

The issue affect Windows 7 and Windows Server 2008 (R2).

https://www.theregister.co.uk/2019/01/09/windows_7_network_broken/

https://serverfault.com/questions/948263/kb-4480970-and-or-kb-4480960-breaks-smb2-connections-to-windows-7-shares

Microsoft released a fix to fix their update…

update-grub
This update resolves the issue where local users who are part of the local “Administrators“ group may not be able to remotely access shares on Windows 7 SP1 and Windows Server 2008 R2 machines after installing the January 8th, 2019 security updates. This does not affect domain accounts in the local "Administrators" group.

Download KB4487345 and install the fix: KB4487345