====== VMware Troubleshooting ======
==== Cant apply or remediate a host profile ====
A host profile is attached by the cluster and there is no way to apply or remediate. \\
__Reason:__ \\
It seems like you are using the HTML webclient which doesnt have this option (yet). \\
__Fix:__ \\
Either use the flash version (if you have vCenter 6.0 the software client will also work) \\
or\\
apply it via PowerCLI
#remove everything after to use your windows authentication
Connect-VIServer -User -Password
Apply-VMHostProfile -Entity
#if you are not sure what name to use, just list all your esx hosts
(Get-VMHost).name
Recheck the compliance of the host profile. If some issues still exist, execute "Apply-VMHostProfile ..." again.
\\
==== Cant configure vSphere HA for host ====
Trying to activate vSphere HA fails with the following messages: \\
vSphere HA agent cannot be correctly installed or configured
Reconfigure vSphere HA host
Status: Operation timed out.
__Reason:__ \\
There could be different reasons... \\
__Fix:__ \\
At first, if you dont want to get VMs migrated right away, activate maintenance before executing something from below.\\
Check if your service "vSphere High Availability Agent" is running on your host. (its stopped by default if you are in maintenance mode)\\
If it isnt running, start it and execute "Reconfigure for vSphere HA".\\
Still not working? alright then...\\
Go to your cluster configuration and select "Services -> vSphere Availability"\\
On the right side of "vSphere HA is Turned ON" click edit and turn it off. \\
Apply and then turn it back on.\\
Your vSphere should now work again.\\
If it still doesnt, [[http://www.giyf.com|www.giyf.com]]
==== Search in vSphere Client doesnt work ====
Trying to search a system fails with the following message: \\
Login to the query service failed.
The server could not interpret the communication from the client. (The remote server returned an error: (500) Internal Server Error.)
__Reason:__ \\
VMware says that this is "//an expected behavior//"
//Searching for Inventory objects when logged in to the vSphere Client using the Use Windows session credentials option is not supported.//
For sure this is expected... why would you let lazy people search in your vCenter? m(
__Fix:__ \\
At the login screen deselect "Use Windows session credentials" and log in with your username and password.
[[https://kb.vmware.com/s/article/2143566|https://kb.vmware.com/s/article/2143566]]
==== Cant connect a VMDK backup because of duplicated UUIDs ====
Connecting a backed up VMDK to the same machine results in a duplicated UUID \\
msg.disk.duplicateUUID:Virtual disks
"/vmfs/volumes/DATASTORE/VM_FOLDER/VM_NAME.vmdk" and "/vmfs/volumes/DATASTORE/VM_FOLDER/VM_NAME2.vmdk" have the same UUID. Virtual disks with the same UUID should not be assigned to a virtual machine because duplications can lead to problems when the guest boots. Make sure the virtual disks have unique UUIDs.
__Reason:__ \\
Well... Duplicated UUIDs...
__Fix:__ \\
Login on the ESX host to change the UUID of the VMDK.\\
Check the current and set a new one afterwards.
vmkfstools -J getuuid .vmdk
vmkfstools -J setuuid .vmdk
[[https://kb.vmware.com/s/article/2006865|https://kb.vmware.com/s/article/2006865]]
==== Snapshot Manager fails with the error: Operation not allowed in current state ====
Creating / removing a snapshot fails with the following message
Cannot create snapshot
Operation not allowed in current state
On the ESX itself you may find the following error in /var/log/hostd.log
2019-10-08T07:16:52.016Z error hostd[14B81B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/acde0a05-cd36531b/VMNAME/VMNAME.vmx] Invalid transition requested (VM_STATE_ON_SHUTTING_DOWN -> VM_STATE_CREATE_SCREENSHOT): Invalid state
or
[07:43:03.404 F64A8B90 error 'vm:/vmfs/volumes/4acc704a-2b59540f-116b-0017a477fc18/VMNAME/VMNAME.vmx'] Invalid transition requested (VM_STATE_ON_SHUTTING_DOWN -> VM_STATE_CREATE_SNAPSHOT): Invalid state
__Reason:__ \\
This issue occurs if the vCenter Server management agents have stopped causing the snapshot attempt to move in to the ON_SHUTTING_DOWN state.
__Fix:__ \\
Connect to your ESX host via SSH and restart the management agents
/etc/init.d/hostd restart
/etc/init.d/vpxa restart
You should be able to create or remove the snapshot afterwards
[[https://kb.vmware.com/s/article/1027707|VMware KB 1027707 - Invalid state]]\\
[[https://kb.vmware.com/s/article/1003490|VMware KB 1003490 - Restart management agents]]\\
[[https://kb.vmware.com/s/article/2004746|VMware KB 2004746 - Activate SSH]]\\
==== No coredump target has been configured ====
You see a yellow warning on your ESX hosts with the following message
No coredump target has been configured. Host core dumps cannot be saved
{{:vmware:pasted:20191008-152919.png}}
__Reason:__ \\
Just like the warning says, there is no coredump target configured.
__Fix:__ \\
Solution 1:\\
Create the target -> [[https://kb.vmware.com/s/article/2004299|VMware KB 2004299]]
Solution 2:\\
Suppress the warning (WARNING: this will "disable" core dumps when you run into purple screens!)
Go to your "Advanced Settings" and set "SuppressCoredumpWarning" to "1"
Host > Configuration > Advanced Settings > UserVars > SuppressCoredumpWarning > 1
[[https://www.techieshelp.com/disable-vmware-no-coredump-target-has-been-configured/|Techieshelp.com]]
==== Lost connectivity to the device naa.60xxxx backing the boot filesystem ====
You see a yellow warning on your ESX hosts with the following message
Error: Lost connectivity to the device naa.60xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx backing the boot filesystem /vmfs/devices/disks/naa.60xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
__Reason:__ \\
Your ESX lost connection to its bootdevice, maybe because of a network outage (switch reboot, cable disconnect,...)
__Fix:__ \\
Solution 1:\\
Put the host into maintenance mode to migrate all VMs to another host and reboot.
Solution 2:\\
Access your ESX via SSH and restart the managementagent
/etc/init.d/hostd restart
[[https://kb.vmware.com/s/article/50441|VMware KB 50441]]
==== No space left on device (vCenter) ====
* vCenter AD Login doesnt work
* “shell.set –enabled True” shows the error “Unknown command: `shell.set'“
* creating files will show “No space left on device”
You can also check this by connecting via SSH and entering the following commands
com.vmware.vimtop
com.vmware.rvc
>Host to connect to (user@host): root@localhost
>Are you sure you want to continue connecting (y/n)? y
Warning: Permanently added 'localhost' (vim) to the list of known hosts
Failed to connect to localhost: No space left on device @ dir_s_mkdir - /root/.rvc
__Reason:__ \\
You have no space left on your device, mostly /var/log/ and mostly because of audit.log
__Fix:__ \\
Because you cant use PuTTY to operate the shell, you need to open the shell with a kernelparameter in the GRUB bootloader. You need to reboot your vCenter (DOWNTIME!). Before starting your VM, set the "Power On Boot Delay" to 10 seconds.
Fatclient:\\
{{:vmware:pasted:20191011-140448.png}}
Webclient:\\
{{:vmware:pasted:20191011-140602.png}}
After starting the VM, check the VMware console and wait for the bootloader. You can stop the autostart by hitting "spacebar"
Select the vCenter appliance and hit "e"\\
Then select the right kernel and hit "e" again\\
Add "init=/bin/bash" to the parameters (seperated by space "ARG1 ARG2") and boot with "b".\\
You should now be in the shell. Check the drives with "df -h"\\
If audit.log is the problem, empty it with
echo "" > audit.log
To fix the logrotate/cron, check [[https://kb.vmware.com/s/article/2149278|VMware KB 2149278]]
==== Restore from different datastore ====
A VM is broken and needs to be restored. To do so you need to copy the VM from the backup datastore to your productive one \\
__Reason:__ \\
You dont want to run VMs from your backup datastore \\
__Fix:__ \\
Connect to your esx host via ssh and copy the files with "cp" or "vmkfstools"\\
Make sure you either rename your old .vmdk + flatfile to .old or your restore to something different (rename flat in .vmdk)
cp -a myvm.vmdk /vmfs/volumes/prod_datastore/myvm/myvm.vmdk
cp -a myvm-flat.vmdk /vmfs/volumes/prod_datastore/myvm/myvm-flat.vmdk
With vmkfstools you dont need to edit the flat name inside .vmdk, the tool does this for you if you choose another name
vmkfstools -i myvm.vmdk /vmfs/volumes/prod_datastore/myvm/myvm.vmdk
\\