Citrix PVS Image Process – Best Practices

On completion of the implementation and handover of any technology to an IT department / client, high quality build and configuration documentation is a must. This may seem obvious but I’ve been in many situations where project time gets squeezed due to budget or to get a deal over the line and a ‘day or two’ gets lobbed off the documentation phase. This leads to inadequate and poor quality documentation which may not be an issue at first, but down the line when someone else has to take over the environment or troubleshoot it can be nothing short of a nightmare.

If documentation is up to scratch and training workshops are provided, it will empower the client to take over and confidently manage the environment, and in some cases, carry out future upgrades. Some integrators may see this as bad business from a revenue perspective, but I believe customers are likely to consume more services in the future when they are confident you are providing them with the correct traning and tools. And, realistically, internal IT departments are typically way too busy to take on more than what they already have on their plates, but it’s good practice to give them the option.

Now back to point of this blog post; one of our clients had the time and resource to invest in the internal IT team following the positive start in documentation and training provided by us. They asked us to provide some best practice tips around the PVS imaging process in preparation for a PVS upgrade they were taking on internally. We, the Citrix Engineering and Consulting team, came up with the following, it was written over a year ago and has had the input and review of around 6 senior Citrix engineers and consultants. I still refer to it and find it relevant so here it is:

Provisioning Services: Image Creation

Assumes that the image is working with an attached local drive. The attached local drive is typically created/placed on high speed or near high speed storage according to environment needs or constrains.

1. Redirect PageFile from %SystemDrive% to a local drive.

2. Crash dumps;

• Totally disable crash dump processing,
• Or, configure as mini dump only and redirect to the local drive.

3. Disable or minimise as much write operations to the “%SystemDrive%” as possible. Ideally, write operations should be restricted to the ““%SystemDrive%\Users” path only.
As a simple example, exclude *ALL* optimisation and scanning tools such as;

• Disable scheduled defragmentation tasks of the “%SystemDrive%”.
• DDL optimisations, including Citrix DDL optimisation.
• Deploy Antivirus exclusions for key operating components and files (refer to current Citrix and vendor recommendations).

4. Windows Event logs redirected from “%SystemRoot%\System32\winevt\Logs” to the local drive. This may be done as either;

• Policy based redirection on a per log basis,
• Or (recommended), using junction point redirection on the “Logs” folder to capture all logs.5. Application log redirection – if required. This may be done via vendor recommended configuration or junction point redirection.

6. Keep the design simple. Complex designs introduce complex problems that are often extremely difficult to diagnose or isolate. In general, always try and use vendor default paths, settings and configuration. Only deviate from this where a vendor has a clearly defined best practice procedure to allow this.

• It is an unfortunate fact of life that programmers often accidentally hard code configuration options into applications and the operating system. Tricky or complex deviations from vendor recommendations often lead to obscure, intermittent or strange behaviours that can be extremely difficult to diagnose.
  If you use only vendor recommended configuration changes, then you have a much higher assurance that the vendor will include full testing in their hotfixes and patching procedures.

7. All applications should be validated to confirm that they do not place ephemeral (temp, batch or data), or user specific files in in any of the following paths; %SystemDrive%, %SystemDrive%\TEMP, %SystemRoot%, %ProgramFiles%, %ProgramFiles(x86)%, %CommonProgramFiles%, %CommonProgramFiles(x86)%, or any other similar path. Note; application global configuration files used by all users are exempt from this rule.

8. All applications should be validated to confirm that they do not create server specific configuration settings. This includes both registry and file based configuration – including INI, XML, CFG, DAT, etc, files.
Note: Special attention should be made to look for server unique settings such as GUID strings like “{BB2E617C-0920-11d1-9A0B-00C04FC2D6C1}”, that may uniquely identify a server for AntiVirus, Firewall, licencing’s, etc, configuration.
If application unique settings are encountered, these unique server settings should be set via the Provisioning Services device TARGET personality strings. Thus, can be configured within the Image during each server’s boot sequence.

9. Do NOT redirect application installation paths. This may include changing %ProgramFiles%, %ProgramFiles(x86)% locations or the installation path of applications.

• It is an unfortunate fact of life that programmers can accidentally hard code program paths into application operate. This may lead to obscure, intermittent or strange application behaviours if an application is not installed in its default path.
  In addition, be extremely careful with changing application configuration settings outside of those provided in the application’s own configuration files or registry settings.

10. Good practice application tips also include;

• Try and minimise any situation where application executables are run from network shares. In addition, this also includes opening database files from network shares.
  ▫ It is understood that occasionally the only option is to place some applications on network shares due to limitations on their design or age.
• Removal of any unnecessary application based Taskbar items or background services.
• Removal of any unnecessary Start Menu or Desktop icons. This includes shell based namespace items held in the registry.

11. Utilise user “home folder” shares to store application per user settings.

12. User based Folder Redirection policy applied via GPO.

13. Deploy Citrix Profile Manager based profile management. This will ensure that multi-session &/or disconnected sessions do not encounter the synchronisation issues.

14. Consider segmenting PVS streaming traffic to a dedicated VLAN if the network has a highly utilized or potentially saturated. This can also be implemented for security purposes if the business require it. Note, segmenting PVS traffic used to be a Citrix best practice but is no longer the case for the vast majority of deployments.

15. TCP Large Send Offload option allows the AIX TCP layer to build a TCP message up to 64 KB long and send it in one call down the stack through IP and the Ethernet device driver. The adapter then re-segments the message into multiple TCP frames to transmit on the wire. The TCP packets sent on the wire are either 1500-byte frames for a Maximum Transmission Unit (MTU) of 1500 or up to 9000-byte frames for a MTU of 9000 (jumbo frames).

16. Re-segmenting and storing packets to send in large frames causes latency and timeouts to the Provisioning Server. This should be disabled on all Provisioning Servers and clients.

17. Disable Large Send Offload, open the Network Interface Card (NIC) properties and select the Advanced tab.
Some NICs do not offer this setting in the properties page. In this case, you must change a registry key change to disable Large Send Offload. To disable, add the following entry to the registry:

• Target Device
  Note: For Provisioning Server 6.0 and beyond, BNNS driver is no longer used for Windows 7 and 2008, so this registry key is not applicable. However, BNNS is still used for windows XP and 2003.HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BNNS\Parameters\
  DWORD = EnableOffload
  Value: “0”
• Provisioning Server and Target Device
  HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TCPIP\Parameters\
  Key: “DisableTaskOffload” (dword)
  Value: “1”

18. Hard code all PVS ports on the servers & clients to avoid auto negotiation network speed, this can cause slow start up times and PXE timeouts for target devices.

19. Firewall and server to server communication ports.

Open the following ports in both directions:

• UDP 6892 and 6904 (For Soap to Soap communication – MAPI and IPC)
• UDP 6905 (For Soap to Stream Process Manager communication)
• UDP 6894 (For Soap to Stream Service communication)
• UDP 6898 (For Soap to Mgmt Daemon communication)
• UDP 6895 (For Inventory to Inventory communication)
• UDP 6903 (For Notifier to Notifier Communication)

Provisioning Services: Image Closure

The provisioned image should typically assume a local drive is attached for the storage of; WriteCache, PageFile redirection, Windows Mini dumps (only if absolutely required), Windows Event Logs, and Application specific logs.

Any other server specific personality settings should be set via the Provisioning Services device TARGET personality strings.
Confirm the following:

1. Refer to the “Image Creation” sequence and validate all items. Specific focus should be on any additional applications that have been added or modified in the environment. Then focus on the best practice steps associated with applications. E.g.

• Personality strings set, as required, for applications that require personality traits.
• Confirmation of shared folder usage. Specifically; applications should not place ephemeral (temp, batch or data), or user specific files in %ProgramFiles%, %ProgramFiles(x86)%, %CommonProgramFiles%, %CommonProgramFiles(x86)%, %SystemDrive%, or %WinDir% paths.

2. Using a Domain account with local Admin rights, login and reboot the server. If you wish, you may also perform a final “GPUPDATE /FORCE”.

3. Login as the local machine Administrator account and remove:

• Any roaming or local user profiles on the server image created or left from the creation or update of the server image. This must be validated in three (3) locations;
  ▫ Remove the extraneous profiles using the Windows Profiles dialog under the “Advance System Settings”.
  ▫ Verify there are no extraneous profile folders in “%SystemDrive%\Users”.
  ▫ Using regedit, verify there are no extraneous profiles in “HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList”. Specific attention should be focused on any “.bak” files.
• Any / all temp files created on the %SystemDrive% &/or sub folders.

4. Using the Citrix Role Manager, perform a XenApp prep for image closure. Do not remove the server from the Farm. Then shutdown gracefully in preparation for image closure in PVS.

5. When using Citrix UPM 4.x in the image, remove the UPM cache file (not applicable to UPM 5.x)

6. Run ‘ngen /ExecuteQueuedItems’ for all CLRs (2.0 x86, 2.0 x64, 4.0 x86, 4.0 x64) in order to precompile assemblies, if this step is not performed, ngen will run the pre-compilation at random times after every reboot in PVS standard mode and therefore cause significant IO load.

Thanks for reading, share if you found this useful.