VCAC Archives - VMtoday

Since joining VMware, I’ve built, rebuilt and updated several lab environments (home lab and work labs). One of the problems I keep running into is incorrect/missing/disabled NTP on my ESXi hosts. Because lab gear is often ~~abused~~ well-loved, BIOS time is not always correct (long periods of being powered off), settings are all sorts of jacked up (it’s a lab, we’re engineers, let’s break stuff!!!), and documentation is non-existent. The problem is not just in a lab environment – the number of customer environments I’ve seen that aren’t set to use NTP is much higher than I would have thought (I wrote about a case here where incorrect time on hosts caused some of the problems with a VMware Horizon View environment). Getting ESXi host time set correctly is one of the first steps in building out a new vSphere environment as authentication fails, VM guests get incorrect time, HA configuration can fail, log files get incorrect time stamps making security analysis or troubleshooting difficult, etc. This TechTarget tip (https://searchvmware.techtarget.com/tip/Network-time-synchronization-for-VMware-ESXi-Timing-is-everything) does a good job explaining the basics of ESXi time synchronization, including how to manually set time and NTP on ESXi hosts. Manually setting time, NTP, and DNS forwarding servers (if you’re using hostnames for NTP instead of IP addresses) on a bunch of hosts is a waste of time – let’s automate!

Before we get too carried away, let’s first make sure our vCenter server has the correct time. If you installed your vCenter Server on a Windows host, it should get it’s time from your Active Directory. For a vCenter Server Virtual Appliance (VCVA), you need to set the NTP settings manually. To do this, log into the admin interface of the VCVA (https://<VCVA-name-or-IP>/5480/). On the vCenter Server tab, click either the ‘Time’ menu button or the ‘Configure Time’ link.

You have several options for time synchronization on the VMware vCenter Server Appliance:

No synchronization – just don’t…
NTP synchronization – sync with an upstream NTP server. Use an authoritative source on your own network if available, otherwise use something like ntp.org’s servers.
VMware Tools synchronization – Use the pre-installed VMware tools package to read time from the ESXi host that the VCVA is running on. If the host’s time is incorrect or changes, the VCVA time will change too. If you vMotion the VCVA from one host to another and the hosts time is not in sync, the VCVA will experience a time shift.
Active Directory synchronization – If you joined your VCVA to your Microsoft Active Directory, VCVA will read the time from your domain’s authoritative time source (usually the domain controller holding the PDC Emulator FSMO role).

I prefer Active Directory if one is available and correctly configured to pull time from a trusted higher stratum server. See this TechNet article for tips on how to configure an Active Directory forest for reliable time synchronization. Next I would choose the NTP synchronization option, followed by VMware tools sync, and never the no time synchronization.

Once the VCVA has accurate time, we can focus on the ESXi hosts. As I mentioned above, this can be done manually in either the vSphere Client (the legacy C# client) or the vSphere Web Client. Here’s the screenshot for where to click in the vSphere Web Client for a single host.

That’s too many clicks for me, so I put together a quick script in PowerCLI that will do several things for me:

Clear existing NTP servers from ESXi hosts.
Manually set the time on all ESXi hosts to match the local system time (the system you are running the script on). This is very helpful if you have servers with date-time values that are way out of whack. The ESXi NTP client will not correct time on the host when the offset (i.e. difference) between the host time and the NTP server time is greater than the preset sanity limit of 1000 seconds. Per VMware KB 1005092, If you have a greater than 1000 second offset you may find entries like the following in /var/log/messages or /var/log/hostd.log (better yet, see them in VMware Log Insight!!!)
```
 ntpd[263140]: time correction of  seconds exceeds sanity limit (1000); set clock manually to the correct UTC time.
 [info 'ha-eventmgr'] Event 91 : NTP daemon stopped. Time correction 1206 > 1000 seconds. Manually set the time and restart ntpd.
```
Configure DNS servers on the management network (useful if you find your NTP servers by name instead of IP.
Configure NTP servers on ESXi hosts, configure the NTP daemon to start automatically with the host, open the ESXi firewall for outbound NTP requests, and restart the NTP service to be sure everything is working correctly.

I have a simple menu driving the script to make changes easy without having to remember to pass variables and whatnot. You’ll see the first menu option for ‘Delete all existing DNS Servers values’ is not working. PowerCLI can’t write null values with the Set-VMHostNetwork -DNSAddress command, and the UpdateDnsConfig method in VMware.Vim.HostDnsConfig was not behaving for me (probably because my PowerCLIfu is weak). Adding new DNS servers will overwrite any existing values, so the option is kinda silly to have I guess….

Feel free to modify or suggest improvements. The code is on github (https://github.com/joshuatownsend/set-vmware-ntp-dns), and displayed below for you to use:

Code below ↓
001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197	`<######################################################################` `Menu-driven` `PowerCLI script to delete, add, and update DNS and NTP on` `VMware ESXi hosts.` `Save as` `set-ntp.ps1` `Switch to directory ps1 is saved in and run` `** MAKE SURE LOCAL SYSTEM TIME IS CORRECT!!!!` `v1.0 27 February 2014: Initial draft` `v1.1 27 February 2014: Changed menu option 1 to dark gray cause no worky.` `Written By Josh Townsend` `All Code provided as is and used at your own risk.` `######################################################################>` `$xAppName` `= ‘set-ntpdns’` `[BOOLEAN]$global:xExitSession=$false` `# Add PowerCLI snapin if not already loaded` `if ((Get-PSSnapin` `-Name` `VMware.VimAutomation.Core` `-ErrorAction` `SilentlyContinue)` `-eq` `$null` `) {Add-PsSnapin` `VMware.VimAutomation.Core;` `$bSnapinAdded` `=` `$true}` `# Prompt for vCenter or ESXi server` `$vCenter` `=` `Read-host` `"Enter the name of your vCenter Server or ESXi host"` `$domain` `=` `Read-host` `"Domain name (leave blank if using local ESXi account)"` `$user` `=` `Read-host` `"vCenter or ESXi user account"` `$password` `=` `Read-host` `-Prompt` `"Enter password"` `-AsSecureString` `$decodedpassword` `= [System.Runtime.InteropServices.Marshal]::PtrToStringAuto([System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($password))` `Connect-VIServer` `$vCenter` `-user` `$domain\$user` `-Password` `$decodedpassword` `#Some other connection options:` `#Option 1 - Show menu of recently used ESXi or vCenter Servers` `#Write-host "Choose from the list below or enter the name of your vCenter Server or ESXi Host, then enter credentials as prompted" -ForegroundColor Yellow` `#Connect-VIServer -Menu $true` `#option 2 - Hard code it. This leaves your password in plain text. Consider using the new-VICredentialStore option to securely store your credentials!` `#Connect-VIServer -Server 10.10.10.10 -User root -Password vmware` `$esxHosts` `=` `get-VMHost` `Function` `Clear-DNS{` `#foreach ($esx in $esxHosts) #{` `#Write-Host "Deleting existing DNS servers from $esx" -ForegroundColor Green` `# Get Current Values for required properties` `#$currenthostname = Get-VMHost \| Get-VMHostNetwork \| select hostname` `#$currentdomainName = Get-VMHost \| Get-VMHostNetwork \| select domainname` `# ------- UpdateDnsConfig -------` `#$config = New-Object VMware.Vim.HostDnsConfig` `#$config.dhcp = $false` `#$config.hostName = $currenthostname` `#$config.domainName = $domainname` `#$_this = Get-View -Id 'HostNetworkSystem-networkSystem'` `#$_this.UpdateDnsConfig($config)` `#}` `#Write-Host "Old DNS Values Cleared! Press any key to return to menu..." -ForegroundColor Green` `Write-Host` "Sorry, this function not supported because we cannot delete or write null values for DNS Server Addresses using PowerCLI. `nUse the Add Additional menu option to overwrite existing DNS servers. `nPress any key to return to the menu." `-ForegroundColor` `Magenta` `$x` `=` `$host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")` `LoadMenuSystem` `}` `Function` `Clear-NTP{` `foreach` `($esx` `in` `$esxHosts) {` `Write-Host` `"Deleting existing NTP servers from $esx"` `-ForegroundColor` `Green` `$existingNTParray` `=` `$esxhosts` `\|` `Get-VMHostNTPServer` `Remove-VMHostNTPServer` `-NtpServer` `$existingNTParray` `-Confirm:$false` `}` `Write-Host` `"Old NTP Values Cleared! Press any key to return to menu..."` `-ForegroundColor` `Green` `$x` `=` `$host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")` `LoadMenuSystem` `}` `Function` `Clear-NTPDNS{` `Clear-NTP` `Clear-DNS` `Write-Host` `"Done! Press any key to return to menu...."` `-ForegroundColor` `Green` `$x` `=` `$host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")` `LoadMenuSystem` `}` `Function Manual-Time{` `# First set all hosts in vCenter to use time of local system to prevent too large drift for NTP to correct` `Write-Host` `"Updating manual time on $esx to match local system time"` `-ForegroundColor` `Green` `$esxhosts` `\|` `Where-Object` `{` `$t` `=` `Get-Date` `$dst` `= $_ \| %{` `Get-View` `$_.ExtensionData.ConfigManager.DateTimeSystem }` `$dst.UpdateDateTime((Get-Date($t.ToUniversalTime())` `-format` `u))` `}` `Write-Host` `"Done! Press any key to return to menu...."` `-ForegroundColor` `Green` `$x` `=` `$host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")` `LoadMenuSystem` `}` `Function` `Set-DNS{` `# Prompt for Primary and Alternate DNS Servers` `$dnspri` `=` `read-host` `"Enter Primary DNS"` `$dnsalt` `=` `read-host` `"Enter Alternate DNS"` `# Prompt for Domain` `$domainname` `=` `read-host` `"Enter Domain Name"` `foreach` `($esx` `in` `$esxHosts) {` `Write-Host` `"Configuring DNS and Domain Name on $esx"` `-ForegroundColor` `Green` `Get-VMHostNetwork` `-VMHost` `$esx` `\|` `Set-VMHostNetwork` `-DomainName` `$domainname` `-DNSAddress` `$dnspri` `,` `$dnsalt` `-Confirm:$false` `Write-Host` `"Done! Press any key to return to menu...."` `-ForegroundColor` `Green` `$x` `=` `$host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")` `LoadMenuSystem` `}` `}` `Function` `Set-NTP{` `#Prompt for NTP Servers` `$ntpone` `=` `read-host` `"Enter NTP Server One"` `$ntptwo` `=` `read-host` `"Enter NTP Server Two"` `foreach` `($esx` `in` `$esxHosts) {` `Write-Host` `"Configuring NTP Servers on $esx"` `-ForegroundColor` `Green` `Add-VMHostNTPServer` `-NtpServer` `$ntpone` `,` `$ntptwo` `-VMHost` `$esx` `-Confirm:$false` `Write-Host` `"Allow NTP queries outbound through the firewall on $esx"` `-ForegroundColor` `Green` `Get-VMHostFirewallException` `-VMHost` `$esx` `\|` `where` `{$_.Name` `-eq` `"NTP client"} \|` `Set-VMHostFirewallException` `-Enabled:$true` `Write-Host` `"Starting NTP service on $esx"` `-ForegroundColor` `Green` `Get-VmHostService` `-VMHost` `$esx` `\|` `Where-Object` `{$_.key` `-eq` `"ntpd"} \|` `Start-VMHostService` `Write-Host` `"Configuring NTP Client Policy on $esx"` `-ForegroundColor` `Green` `Get-VMHostService` `-VMHost` `$esx` `\|` `where{$_.Key` `-eq` `"ntpd"} \|` `Set-VMHostService` `-policy` `"on"` `-Confirm:$false` `Write-Host` `"Restarting NTP Client on $esx"` `-ForegroundColor` `Green` `Get-VMHostService` `-VMHost` `$esx` `\|` `where{$_.Key` `-eq` `"ntpd"} \|` `Restart-VMHostService` `-Confirm:$false` `}` `Write-Host` `"Done! Press any key to return to menu...."` `-ForegroundColor` `Green` `$x` `=` `$host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")` `LoadMenuSystem` `}` `Function` `Set-NTPDNS` `{` `Set-DNS` `Set-NTP` `Write-Host` `"Done! Press any key to return to menu...."` `-ForegroundColor` `Green` `$x` `=` `$host.UI.RawUI.ReadKey("NoEcho,IncludeKeyDown")` `LoadMenuSystem` `}` `# Menu to select functions` `function LoadMenuSystem() {` `[int]$xMenuChoiceA` `= 0` `[BOOLEAN]$xValidSelection=$false` `while (` `$xMenuChoiceA` `-lt` `1` `-or` `$xMenuChoiceA` `-gt` `8 ){` `CLS` `# ------ Menu Choices -------` `Write-Host` “Choose an option below to modify all ESXi hosts DNS and NTP settings.`n” `-ForegroundColor` `Magenta` `Write-host` "`t1. Delete all existing DNS Servers values (not working)" `-ForegroundColor` `DarkGray` `Write-host` "`t2. Delete all existing NTP Server values" `-ForegroundColor` `Cyan` `Write-host` "`t3. Delete all existing DNS & NTP Servers (not working)" `-ForegroundColor` `Cyan` `Write-host` "`t4. Manually set time on all hosts to match your local system time." `-ForegroundColor` `Cyan` `Write-host` "`tThis option prevents NTP sync problems due to large offset" `-ForegroundColor` `Gray` `Write-host` "`t5. Add additional DNS Server values" `-ForegroundColor` `Cyan` `Write-host` "`tThis will overwrite any existing values" `-ForegroundColor` `Gray` `Write-host` "`t6. Add additional NTP Server values and set NTP to start with host.*" `-ForegroundColor` `Cyan` `Write-host` "`t*Verifyr DNS values are correct if using names instead of IPs." `-ForegroundColor` `Gray` `Write-host` "`t7. Add both NTP & DNS Server values and set NTP to start wtih host" `-ForegroundColor` `Cyan` `Write-host` "`t8. Quit and exit`n`n" `-ForegroundColor` `Yellow` `# ------ Menu Choices -------` `# Get and Validate Choice` `[Int]$xMenuChoiceA` `=` `read-host` `"Please select an option [1-8]"` `if(` `$xMenuChoiceA` `-lt` `1` `-or` `$xMenuChoiceA` `-gt` `8 ){` `Write-Host` “`tInvalid Selection.`n” `-Fore` `Red;start-Sleep` `-Seconds` `1` `}` `# Survey Says....` `Switch(` `$xMenuChoiceA` `){#… User has selected a valid entry.. load menu` `1{` `Clear-DNS` `}` `2{` `Clear-NTP` `}` `3{` `Clear-NTPDNS` `}` `4{ Manual-Time` `}` `5{` `Set-DNS` `}` `6{` `Set-NTP` `}` `7{` `set-NTPDNS` `}` `default {` `$global:xExitSession=$true;break }` `}` `}` `}` `LoadMenuSystem` `If ($xExitSession){` `Exit-PSSession` `#… User quit & Exit` `} Else {` `.\set-ntp.ps1` `#… Loop the function` `}`