Get-VMHostTimeReport Reporting time from vmhost system

I wanted to check if all of my vmhost system are keeping time with ntp. I wrote recently how to query time from vmhost system using esxcli. Today i want to show how to do this without esxcli, and also produce nice report and a summary which will help investigate issue with host time. I am comparing host time with local time o/s that is converted to UTC. If you are not sure whether your time is accurate then it might be a problem. Solution for this is to query remote utc time. In internet you can find sites that offer time web services that can be easily integrated to this script.
Have in mind that property DiffToUTC is in seconds.
I have tried to optimize this report as much as i can. It was possible to execute this report in 1 minute and 8 seconds. But it was not that readable and did not contain all of this information. On 100 vmhost+ it should fit in 1 – 2 minutes. Before i had approach with obtaining time using esxcli, but that was slower then this approach.
If you know how to query it faster, post a comment. Have in mind that this function is not perfect. I believe that there are better ways to show weather vmhost system is out of time or not. Anyway, this function helps a lot in diagnosing problems time on esx/esxi host systems.
It will state if ntpd is running or not, or for example if you have >100 host you can spot a pattern for ntp servers. Sometimes it is easy to overlook something, for example ntp servers:
172.16.x.y
72.16.x.y
Where you can easily see that you had a typo.
Sometimes you are just not aware that ntp service is running.
Or sometimes… 😉
Few screenshots from running this function:
From this screenshot where the report was stored in $myreport, we can see that we have some issues with host time, as there is a difference of 76 second compared to my local os time.
srh2
If you see DiffToUTC like 0.0xxxxx that would mean that you are ok. If you get readings > 1 seconds that could indicate something is wrong.
We can also do report for a single vmhost system.
srh1
We can do a full report and using the -summary switch, make function to return some description about the report.
trsum
Documentation for
HostDateTimeSystem -> Managed Object – HostDateTimeSystem
HostServiceSystem -> Managed Object – HostServiceSystem
I have added now option to check your local time which will be converted to UTC and compare it to UTC time taken from internet. Internet connection should be in place in order to use it. That was added if you suspect that your local time might not be correct and you would like to check it with other source. Please also read disclaimer,usage restrictions before using this time web service. So if you spot that your time differs to much from time taken from that ‘internet returned time’, it might indicate some issues, maybe the web service has issues, or our system.
Function below:

function Get-VMHostTimeReport {
<#
	.SYNOPSIS
		Gets time from VM Hosts and checks with local time(to utc).

	.DESCRIPTION
		This function might help investigating issues with vm host time. If it is running without any
		parameter it checks time for all hosts registered in virtual center to which user is currently
		connected. Using parameter SingleVMHost will produce report for single vm host system.
		VMHost should be the name of the host, string.
		Report returns colums : Name (vmhost system name), VMHostTime (Time from vmhost),
		UTCTime (this is the utc time from our local os), NTPServers ( if any are in the vmhost
		configuration), NTPServiceRunning (checks if the ntp service is running on the vmhost),
		DiffToUTC (that's the difference in seconds between time reported by vmhost and our os)
		By default it sorts report by from lowest to highest time difference reported.
		

	.PARAMETER  SingleVMHost
		Specify single vmhost name that is registered in VC. This should be a string.
		
	.PARAMETER  Summary
		Indicate if you would like to receive short summary about produced report.

	.PARAMETER  CheckTimeFromInternet
		Indicate if you would like to see in summary information about your local time and remote 
		time. Time from remote web service will be checked and comapred to your system utc time.

	.EXAMPLE
		PS C:\> Get-VMHostTimeReport
		Will produce report for all vmhosts that are registered within VirtualCenter to which user
		is currently connected. It is possible to close the report into a variable for example:
		$timereport=Get-VMHostTimeReport
		You can then export this report to csv if needed for example, or view it again :
		$timereport | format-table -autosize
		For viewing convenience
		
		
	.EXAMPLE
		PS C:\> Get-VMHostTimeReport -SingleVMHost 'myesxhost.local.lan'
		Will produce report for given vmhost that is registered within VirtualCenter to which user
		is currently connected.
		

	.EXAMPLE
		PS C:\> Get-VMHostTimeReport -Summary
		Will produce report . and show a small summary which might indicate if there is a problem
		with time sync on vmhost systems. Example of the summary below:
		UTC time from the current system :Min and Max times during reporting period
		Min: 9/3/2013 4:17:38 PM
		Max: 9/3/2013 4:19:20 PM
		While function was creating this report, first date that was returned by local os was the Min
		and the last date that was returned by local os was the Max value.
		VMHosts reported Times :Min and Max date / time while creating report
		Min: 9/3/2013 4:16:35 PM
		Max: 9/3/2013 4:19:20 PM
		Our vmhost systems reported their date/time. If this time span is too big, this might indicate 
		issues with vmhost ntp sync.
		Time Difference between VMHost and UTC from local os time Min, Max, Avg
		Min: 0.03075
		Max: 76.20985
		Avg: 1.36604993421053
		This is summary for comparing UTC vm host time to UTC time from local os. If you see 
		big Max value >1/2 sec that it might indicate that there is an issue with vmhost ntp time sync
		If a switch parameter CheckTimeFromInternet is present in the summary section there will
		be small report generated about your local time converted to UTC and time taken from
		http://www.earthtools.org
		You can then quickly see if there is an issue with your local time

	.NOTES
		NAME:  Get-VMHostTimeReport
		
		AUTHOR: Grzegorz Kulikowski
		
		NOT WORKING ? #powercli @ irc.freenode.net 
		
		THANKS: http://www.earthtools.org

	.LINK

https://psvmware.wordpress.com

#>

   param(
   [string]$SingleVMHost,
   [switch]$Summary,
   [switch]$CheckTimeFromInternet
   )
$TimeReport=@()
if ($SingleVMHost) { $VMHosts=Get-View -ViewType HostSystem -Filter @{'name'=$SingleVMHost} }
else{
$VMHosts=get-view -viewtype hostsystem -property name,ConfigManager.DateTimeSystem,ConfigManager.ServiceSystem -Filter @{'runtime.ConnectionState'='connected'}
}
Foreach($VMHost in $VMHosts){
$VMHostDateTimeSystem=get-view -id $VMHost.ConfigManager.DateTimeSystem
$VMHostServiceSystem=get-view -id $VMHost.ConfigManager.ServiceSystem
$VMHostTime=$VMHostDateTimeSystem.QueryDateTime()
$NtpServiceState=($VMHostServiceSystem.ServiceInfo.Service|Where-Object {$_.Key -eq 'ntpd'}).Running
$NtpServers=$VMHostDateTimeSystem.DateTimeInfo.NtpConfig.Server
$UTCTime=(Get-Date).ToUniversalTime() 
$TimeReport+=$vmhost| Select-Object -Property Name, @{n='VMHostTime';e={$VMHostTime}},@{n='UTCTime';e={$UTCTime}},@{n='NTPServers';e={$NtpServers}},@{n='NTPServiceRunning';e={$NtpServiceState}},@{n='DiffToUTC';e={[Math]::Round([math]::abs(($VMHostTime - $UTCTime).TotalSeconds),5)}}
}
if($Summary){
$SummaryUTCTime=$TimeReport|Measure-Object -Property UTCTime -Min -Max
$SummaryVMHostTime=$TimeReport|Measure-Object -Property VMHostTime -Min -Max
$SummaryDiffToUTC=$TimeReport|Measure-Object -Property DiffToUTC -Min -Max -Average
Write-Host "UTC time from the current system :Min and Max date/time while creating this report."
Write-Host "Min: $($SummaryUTCTime.Minimum.ToString())"
Write-Host "Max: $($SummaryUTCTime.Maximum.ToString())"
Write-Host "VMHosts reported Times :Min and Max date/time while creating this report"
Write-Host "Min: $($SummaryVMHostTime.Minimum.ToString())"
Write-Host "Max: $($SummaryVMHostTime.Maximum.ToString())"
Write-Host "Time Difference between VMHost and UTC from local os time Min, Max, Avg"
Write-Host "Min: $($SummaryDiffToUTC.Minimum.ToString())"
Write-Host "Max: $($SummaryDiffToUTC.Maximum.ToString())"
Write-Host "Avg: $($SummaryDiffToUTC.Average.ToString())"
if($CheckTimeFromInternet){
[datetime]$TimeFromInternet=(Invoke-RestMethod -Uri 'http://www.earthtools.org/timezone/52.35000/4.86660').timezone.utctime
$CurrentSystemTimeUTC=(Get-Date).ToUniversalTime()
Write-Host "We took UTC time from internet and compared it to your local time converted to UTC time"
$TimeDiff=[math]::abs(($CurrentSystemTimeUTC-$TimeFromInternet).TotalSeconds)
Write-Host "Reported local time converted to UTC $CurrentSystemTimeUTC"
Write-Host "Reported time taken from internet(http://www.earthtools.org/webservices.htm) $TimeFromInternet"
Write-Host "Difference: $TimeDiff seconds"
}
}
return $TimeReport | Sort-Object -Property DiffToUTC
}

‘Cannot complete the HA configuration’ yellow popup message via powercli

I have noticed some day that issue like on screenshot below is not going to the Alert tab. 99% that some other alert created this message, for example host is loosing uplink will result in having this popup to show up. I was wondering how it is possible for me to monitor those yellow popups in summary tab 😉

The easiest way to get information about this popup is to check the configissue property of vmhost object.

$myhost=get-vmhost 'myhost'
$myhost.extensiondata.configissue
-
Message              :
Reason               : agentFailed
Key                  : 0
ChainId              : 0
CreatedTime          : 8/9/2012 12:27:41 PM
UserName             :
Datacenter           : VMware.Vim.DatacenterEventArgument
ComputeResource      : VMware.Vim.ComputeResourceEventArgument
Host                 : VMware.Vim.HostEventArgument
Vm                   :
Ds                   :
Net                  :
Dvs                  :
FullFormattedMessage : HA agent on myhost in cluster X in Y DC has an error :  HA agent on the host failed
ChangeTag            :
DynamicType          :
DynamicProperty      :

That’s pretty much it 😉 In this object there will be FullFormattedMessage property that will describe current issue. From here is now easy to write some little report that will inform us if we have any config issues on our hosts.

vmxnet3 Ethernet Adapter #2 #3 #4

So what to do when you add a vmnic vmxnet3 to vm and it results in having a network adapter with vmxnet3 Ethernet Adpater #2 or #3 for example. Well i tried many different workarounds, solutions but for some reasons they were not working for me. I visited around 10-20 different pages with ms,vmware kb and still it was not working.  For some reason my network cards were not on the list even if i used ms kb to see hidden devices as described here:

http://support.microsoft.com/kb/315539

How did i do it ? 🙂

First of all, let’s grab some knowledge about devcon.

http://msdn.microsoft.com/en-us/library/windows/hardware/ff544707%28v=vs.85%29.aspx

Download devcon tool. Make sure you use the right version on the system. If you are running for example w2k8r2 64bit you will not be able to remove hidden device. You will be able to list it though using

devcon.exe findall =net

It will show you hidden devices, but when you will try to remove it using 32bit devcon.exe nothing will happen 😉

I could not find standalone version of devcon.exe so i had to download the whole tools iso file from Microsoft.

http://www.microsoft.com/download/en/details.aspx?id=11800 => Windows Driver Kit

Open the iso file image and look for

GRMWDK_EN_7600_1.ISO\WDK\setuptools_x64fre.msi

GRMWDK_EN_7600_1.ISO\WDK\setuptools_x64fre_cab001.cab

Copy those 2 files to some test folder like c:\test1

Then extract the msi package ( some info about msi http://msdn.microsoft.com/en-us/library/windows/desktop/cc185688%28v=vs.85%29.aspx )

cd c:\test1

msiexec /a setuptools_x64fre.msi TARGETDIR=”C:\sth\x\y”

And grab fresh 64bit devcon from

c:\sth\x\y\winddk\7600.16385.win7_wdk.100208-1538\tools\devcon\amd64\devcon.exe

Copy the devcon.exe to the vm. If vm does not have any network use powercli for example.

Copy-VMGuestFile –LocalToGuest –Source C:\sth\x\y\winddk\7600.16385.win7_wdk.100208-1538\tools\devcon\amd64\devcon.exe –Destination c:\ -VM yourVM_NAME -GuestUser Administrator -GuestPassword “admin_password”

When connected to esxhost i believe you will have to use -HostUser <host user> -HostPassword <host password>  as well. I was doing this copy connected to vc via powercli.

Now run devcon.exe findall =net  if will print you the list with all network devices. In my case i found 2 which were not used and were causing the #2 #3 .

You will have to remove them all with: devcon.exe remove “@Your_unused_vmxnet_Device” , just copy/paste what’s on the left after executing devcon.exe findall =net regarding your vmxnet adapter.

Then repeat this step until will delete all of them. Reboot.

Now add the vmxnet3 adapter and see what happens 😉 Fresh clean nice and fluffy “vmxnet3 Ethernet Adapter” 🙂

By the way… If you want to have a template from which you will have a vm with clean vmxnet network adapter, then on the template make sure you have cleaned adapters using devcon, make sure that there are no vmnics added to that template vm. Shutdown the template vm. Then add a vmxnet3 network card. Save the vm as a template, and you can deploy vms from it. After vm will be deployed it will have a vmxnet3 network adapter without the #2 or #3 😉

I hope that this will save you a lot of time 😉

Laczymy sie do wielu esx’ow naraz podczas 1 sesji powercli / sposob nr2

Mmmm, w sumie ten sposob ktory wczesniej podalem dziala, ale z tego co zauwazylem z racji ze vmware w nowej wersji wprowadzil ulatwienia jest juz malo uzywany.
Wcale nie musimy budowac tablicy z hostami na ktorych bedziemy chcieli wykonywac nasze cmdlety. Vmware daje nam mozliwosc pracy w trybie multi.
W trybie single wykonywana komenda bedzie uruchamiana wobec ostatniego serwera do ktorego sie podlaczymy, w multi natomiast na kazdym.
Set-PowerCLIConfiguration -DefaultVIServerMode Multiple -Confirm:$false

-> Pisze z -Confirm bo nie chcemy zeby podczas wykonywania skryptow wyskakiwal nam popup.
Po tej komendzie polaczmy sie na 2 esxy:
connect-viserver 10.10.10.1
connect-viserver 10.10.10.2

W rezultacie otrzymamy:
echo $DefaultVIServers

Name Port User
---- ---- ----
10.10.10.1 443 root
10.10.10.2 443 root

Zuwazcie ze gdy mamy tryb single(by default), zmienna $DefaultVIServers pokazuje tylko 1 aktywny host.

Laczymy sie do VC / ESX/ESXi uzywajac credentials z pliku

Zamiast za kazdym razem utozsamiac sie podajac login/pass , mozemy wykorzystac mechanizm credentials ktore jest do naszej dyspozycji.
Connect-VIserver posiada przelacznik -credentials.
Zgodnie z opisem:
http://www.vmware.com/support/developer/windowstoolkit/wintk40u1/html/Connect-VIServer.html
“Specify a PSCredential object that contains credentials for authenticating with the server”
Zatem musimy najpierw stworzyc obiekt PSCredential:
$mycred=get-credential
write-host “Please type file location to store credential”
$file=read-host
$mycred.Password | ConvertFrom-SecureString | Set-content $file
write-host “Credential file was created”

-> Od teraz dysponujemy plikiem z haslem zapisanym w miejscu podanym w $file.

Mozemy zatem napisac funkcje ktora nas autoryzuje :
function authorize
{
if(test-path -PathType leaf $file)
{
write-host “Using authorization file “$file
$password = Get-Content $file | ConvertTo-SecureString
$cred = New-Object System.Management.Automation.PsCredential(“USER”,$password)
return $cred
}
else
{
write-host “No authorization file found”`n”Terminating.”
break
}
}

Potem mozemy w skryptach uzywac juz polaczenie wywolywanego:
$credential=authorize()
connect-viserver A.B.C.D -credential $credential

Jak szybko/sprawnie zmienic nazwe VM na datastore

Bedac zalogowanym na esx/i wydac komendy po uprzedniej zmianie katalogu :

pierwszy=stara_nazwa

drugi=nowa_nazwa

for i in `ls $pierwszy*`; do mv $i `echo $i|sed "s/${pierwszy}/$drugi/g" `; done

sed -i "s/$pierwszy/$drugi/g" ${drugi}.vmdk ${drugi}.vmx ${drugi}_1.vmdk ${drugi}.vmxf