**Update: January 2013: My PAL Threshold files are now included in the PAL 2.3.3 install.  Grab the latest from https://pal.codeplex.com.

Background

I’ve been using the Performance Analysis of Logs (PAL) tool for years (since the first version released in 2007) to assist in analyzing, reporting, and visualizing performance logs in Windows environments.  The real power of PAL is the built-in threshold files that interpret Perfmon counters, providing an explanation of what the counter is, why it matters, and thresholds for warnings and critical to help quickly zero in on performance bottlenecks.  PAL includes templates for Windows, Hyper-V, SQL, Active Directory, IIS, Exchange, .NET, and SharePoint.

I’m often called in to help customers conduct performance analysis and troubleshooting within their VMware View environments.  There are several great tools available for on-going monitoring and alerting in View environments – VMware offers vCenter Operations for VMware View, Xangati makes a good product, and there’s a free PCoIP Log Viewer from Chuck Hirstius at https://mindfluxinc.net/?p=367.  But some customers do not have budget for these products or have an immediate need to do spot troubleshooting for a particular VM, or have problems that extend beyond PCoIP.  I thought why not combine the power of PAL with some VMware View specific data to help analyze Perfmon logs in a View environment?

A Microsoft engineer, Jeff Stokes, wrote a very simple threshold file for VMware Tools counters for VMware guest VM’s.  It was a great start, but I found it didn’t provide enough detail and didn’t include PCoIP statistics for VMware View desktops.  I used Jeff’s template for inspiration and built my own, including VM Processor, VM Memory, and PCoIP Session statistics perfmon counters, along with explanations of each counter, links to resources for each counter, and ideas for remediating performance problems identified with counters that exceed expected thresholds or are suspected of being a bottleneck.  Windows System stats are included in PAL and are included in the PAL report my threshold file generates.  All told, there are 122 different Windows Performance Monitor (perfmon) counters included in my VMware View Desktop PAL report.

I also built a threshold file that can be used to analyze VMware View Connection Servers and Security Servers.  The standard PAL tool with Windows System Overview thresholds can be used to analyze VMware vCenter Server and VMware Composer servers.  All combined, you can use PAL to analyze all components of your VMware View environment, with the exception of Thin or Zero clients.

The full reports can be information overload – you’ll not use it for day-to-day monitoring.  Rather, use it for a deep analysis for specific VM’s with performance issues.

This is my first version of the VMware View specific thresholds.  I plan to add more analysis, calculated thresholds, etc. in future versions.  Also, stay tuned for a threshold file that is capable of analyzing ESXi ESXTOP batch mode output.

As always, I am open for constructive feedback on this tool.  Let me know if it is useful to you, needs some tweaking or tuning, or if you have ideas for cool new features.

Quick tip #1: You can use the View Desktop template to analyze any Windows-based VM, not just View desktops.  In this usage scenario, the PCoIP counters will be ignored.

Quick Tip #2: You can edit my PAL Template file and comment out the inheritance of the SystemOverview.xml file if you just want to collect and report on my VMware counters.

Usage

1.) Download and install PAL Prerequisites:

Microsoft .NET Framework 3.5 Service Pack 1 (Partial package – internet access required)
https://www.microsoft.com/downloads/details.aspx?familyid=AB99342F-5D1A-413D-8319-81DA479AB0D7&displaylang=en

Microsoft .NET Framework 3.5 Service Pack 1 (full package – no internet access required)
https://download.microsoft.com/download/2/0/e/20e90413-712f-438c-988e-fdaa79a8ac3d/dotnetfx35.exe

Microsoft Chart Controls for Microsoft .NET Framework 3.5
https://www.microsoft.com/downloads/details.aspx?FamilyID=130f7986-bf49-4fe5-9ca8-910ae6ea442c&DisplayLang=en

PowerShell v2.0 (Windows Management Framework (Windows PowerShell 2.0, WinRM 2.0, and BITS 4.0))
https://support.microsoft.com/kb/968929

2.) Install PAL

Download and install PAL from https://pal.codeplex.com

3.) Install View Thresholds

Download my View specific threshold files from the links below.  Copy .xml files to your PAL installation directory (C:Program FilesPALPAL).

4.) Export Perfmon Template

Launch PAL from the start menu.  Switch to the ‘Threshold File’ tab.  Choose the ‘VMware View PCoIP and Guest Performance’ or ‘VMWare View Connection Server Performance’ threshold from the Threshold file title dropdown.  Click the ‘Export to Perfmon template file…’ button and save the file for either XP or Windows 7/2008 as appropriate for your environment.  (I also have the exported templates linked below if you need a quick download of them to get a data collection started).

5.) Configure Collector

Copy the exported .html or .xml template to the system you want to analyze.  Create a new data collector using the template you exported.  Run the data collector for a period of time when performance problems are occurring.  You probably don’t want to run this on all of your View desktops, as Perfmon can drive up resource use (CPU, memory, disk IOPS, and disk space).

6.) Analyze

Copy the resulting .blg file to the machine you installed PAL on.  Walk through the PAL wizard to analyze and report on your log data with the appropriate threshold file.  Review the resulting report for issues and take corrective action.

Automate

My fellow vExpert, Jonathan Medd, has a great script for deploying Perfmon Data Collector sets based on an XML template on his blog.  Use this to deploy my template to a batch of View desktops, all Connection Servers, etc.  Find Jonathan’s script here: https://www.jonathanmedd.net/2010/11/managing-perfmon-data-collector-sets-with-powershell.html

Ryen Tang takes a slightly different approach to deploying Perfmon data collectors – his approach will work with a bit of modifications for my counters.

Laerte Junior has another PowerShell approach for gathering Perfmon data from multiple remote computers here: https://www.simple-talk.com/sql/database-administration/gathering-perfmon-data-with-powershell/

Sample Reports

View PAL Desktop Sample Report

View PAL Connection Server Sample Report

Download

VMware View Desktop PAL Threshold File: PAL_VMwareView_PCoIP <-This is a new version as of 14OCT2013 to correct errors during analysis.  Replace your existing in C:\Program Files\PAL\PAL with this copy.

— View desktop Perfmon Template: VMwareViewDesktopPerfmonTemplate

VMware View Connection Server PAL Threshold File: PAL_VMwareView_VDM

— View Connection Server Perfmon template: ViewConnectionServerPerfmonTemplate

Feedback

I welcome your feedback on this tool, including suggestions for warning and critical values, links to resources to be embedded in reports, formatting, etc.  Leave a comment below or email me at josh @ vmtoday.com

Included Perfmon Counters List

• CacheDirty Pages
• CacheLazy Write Flushes/sec
• LogicalDisk(*)% Free Space
• LogicalDisk(*)% Idle Time
• LogicalDisk(*)Avg. Disk Bytes/Read
• LogicalDisk(*)Avg. Disk Bytes/Write
• LogicalDisk(*)Avg. Disk Queue Length
• LogicalDisk(*)Avg. Disk sec/Read
• LogicalDisk(*)Avg. Disk sec/Write
• LogicalDisk(*)Current Disk Queue Length
• LogicalDisk(*)Disk Bytes/sec
• LogicalDisk(*)Disk Reads/sec
• LogicalDisk(*)Disk Transfers/sec
• LogicalDisk(*)Disk Writes/sec
• LogicalDisk(*)Free Megabytes
• Memory% Committed Bytes In Use
• MemoryAvailable MBytes
• MemoryCache Bytes
• MemoryCommit Limit
• MemoryCommitted Bytes
• MemoryFree & Zero Page List Bytes
• MemoryFree System Page Table Entries
• MemoryPages Input/sec
• MemoryPages Output/sec
• MemoryPages/sec
• MemoryPool Nonpaged Bytes
• MemoryPool Paged Bytes
• MemorySystem Cache Resident Bytes
• MemoryTransition Pages RePurposed/sec
• Network Inspection SystemAverage inspection latency (sec/bytes)
• Network Interface(*)Bytes Received/sec
• Network Interface(*)Bytes Sent/sec
• Network Interface(*)Bytes Total/sec
• Network Interface(*)Current Bandwidth
• Network Interface(*)Output Queue Length
• Network Interface(*)Packets Outbound Errors
• Network Interface(*)Packets Received/sec
• Network Interface(*)Packets Sent/sec
• Network Interface(*)Packets/sec
• Paging File(*)% Usage
• PCoIP Session Audio Statistics(PCoIP Session)Audio Bytes Received
• PCoIP Session Audio Statistics(PCoIP Session)Audio Bytes Sent
• PCoIP Session Audio Statistics(PCoIP Session)Audio RX BW kbit/sec
• PCoIP Session Audio Statistics(PCoIP Session)Audio TX BW kbit/sec
• PCoIP Session Audio Statistics(PCoIP Session)Audio TX BW Limit kbit/sec
• PCoIP Session General Statistics(PCoIP Session)Bytes Received
• PCoIP Session General Statistics(PCoIP Session)Bytes Sent
• PCoIP Session General Statistics(PCoIP Session)Packets Received
• PCoIP Session General Statistics(PCoIP Session)Packets Sent
• PCoIP Session General Statistics(PCoIP Session)RX Packets Lost
• PCoIP Session General Statistics(PCoIP Session)Session Duration Seconds
• PCoIP Session General Statistics(PCoIP Session)TX Packets Lost
• PCoIP Session Imaging Statistics(PCoIP Session)Imaging Active Minimum Quality
• PCoIP Session Imaging Statistics(PCoIP Session)Imaging Bytes Received
• PCoIP Session Imaging Statistics(PCoIP Session)Imaging Bytes Sent
• PCoIP Session Imaging Statistics(PCoIP Session)Imaging Decoder Capability kbit/sec
• PCoIP Session Imaging Statistics(PCoIP Session)Imaging Encoded Frames/sec
• PCoIP Session Imaging Statistics(PCoIP Session)Imaging RX BW kbit/sec
• PCoIP Session Imaging Statistics(PCoIP Session)Imaging TX BW kbit/sec
• PCoIP Session Network Statistics(PCoIP Session)Round Trip Latency ms
• PCoIP Session Network Statistics(PCoIP Session)RX BW kbit/sec
• PCoIP Session Network Statistics(PCoIP Session)RX BW Peak kbit/sec
• PCoIP Session Network Statistics(PCoIP Session)RX Packet Loss %
• PCoIP Session Network Statistics(PCoIP Session)TX BW Active Limit kbit/sec
• PCoIP Session Network Statistics(PCoIP Session)TX BW kbit/sec
• PCoIP Session Network Statistics(PCoIP Session)TX BW Limit kbit/sec
• PCoIP Session Network Statistics(PCoIP Session)TX Packet Loss %
• PCoIP Session USB Statistics(PCoIP Session)USB Bytes Received
• PCoIP Session USB Statistics(PCoIP Session)USB Bytes Sent
• PCoIP Session USB Statistics(PCoIP Session)USB RX BW kbit/sec
• PCoIP Session USB Statistics(PCoIP Session)USB TX BW kbit/sec
• PhysicalDisk(*)Avg. Disk Queue Length
• PhysicalDisk(*)Avg. Disk sec/Read
• PhysicalDisk(*)Avg. Disk sec/Write
• PhysicalDisk(*)Current Disk Queue Length
• PhysicalDisk(*)Disk Bytes/sec
• PhysicalDisk(*)Disk Reads/sec
• PhysicalDisk(*)Disk Writes/sec
• Process(*)% Privileged Time
• Process(*)% Processor Time
• Process(*)Handle Count
• Process(*)ID Process
• Process(*)IO Data Operations/sec
• Process(*)IO Other Operations/sec
• Process(*)IO Read Operations/sec
• Process(*)IO Write Operations/sec
• Process(*)Private Bytes
• Process(*)Thread Count
• Process(*)Virtual Bytes
• Process(*)Working Set
• Processor Information(*)% of Maximum Frequency
• Processor Information(*)Parking Status
• Processor(*)% DPC Time
• Processor(*)% Interrupt Time
• Processor(*)% Privileged Time
• Processor(*)% Processor Time
• Processor(*)% User Time
• Processor(*)DPC Rate
• ServerPool Nonpaged Failures
• ServerPool Paged Failures
• SystemContext Switches/sec
• SystemProcessor Queue Length
• SystemSystem Calls/sec
• TCPv4Connection Failures
• VM MemoryMemory Active in MB
• VM MemoryMemory Ballooned in MB
• VM MemoryMemory Limit in MB
• VM MemoryMemory Mapped in MB
• VM MemoryMemory Overhead in MB
• VM MemoryMemory Reservation in MB
• VM MemoryMemory Shared in MB
• VM MemoryMemory Shared Saved in MB
• VM MemoryMemory Shares
• VM MemoryMemory Swapped in MB
• VM MemoryMemory Used in MB
• VM Processor% Processor Time
• VM ProcessorCPU stolen time
• VM ProcessorEffective VM Speed in MHz
• VM ProcessorHost processor speed in MHz
• VM ProcessorLimit in MHz
• VM ProcessorReservation in MHz
• VM ProcessorShares

Connection Server Counters:

• VMware VDMAll Sessions
• VMware VDMAll Sessions High
• VMware VDMCheck Out VMs
• VMware VDMCheck Out VMs High
• VMware VDMFull VM Sessions
• VMware VDMFull VM Sessions High
• VMware VDMLinked Clone Sessions
• VMware VDMLinked Clone Sessions High
• VMware VDMOther Sessions
• VMware VDMOther Sessions High
• VMware VDMPCoIP Gateway Sessions
• VMware VDMPCoIP Gateway Sessions High
• VMware VDMSecure Gateway Sessions
• VMware VDMSecure Gateway Sessions High