Frame Image Management: Implementing a Pilot Testing Methodology

This blog series centers around Frame Image Management. Image Management is “How do I set up my environment in the most optimal way, to minimize the number of images I have to manage?” Part 1 of this series covered how to onboard and configure SaaS applications. Part 2 covered how to install persistent applications into a non-persistent workload. Part 3 covered how to preconfigure user-specific settings for all users from the Sandbox. This blog post will cover another common virtual desktop use case, how to implement a pilot testing methodology using Frame. Due to the nature of persistent vs non-persistent Frame accounts, this post will cover this process from a non-persistent perspective.

Terminology​

First, we must cover some terminology, so everybody reading this will understand my wayward thinking.

Pilot Testing​

Pilot Testing is a method of testing that verifies a component of a system or the entire system under real-time operating conditions. While this is a broad definition, for this scenario we are talking specifically about image updates in a DaaS environment. Pilot testing an image can follow many paths, but at the core there are four different scenarios I like to discuss; development, test, UAT (User Acceptance Testing, also called "pilot"), and production.

Development​

Development, or "dev", is the pilot testing step performed by the administrator of the physical, VDI, or DaaS image. The purpose of dev Testing is to verify the changes you implemented are reflected in the end goal, without testing of the changes themselves. This can be something simple such as creating a folder structure or placing files/registry keys, or something a little more complex such as installing/configuring applications. As an admin, you may not need to test these changes to verify functionality, your purpose is just to make sure these changes are implemented.

Test​

"Test" is the pilot testing step that is most often skipped. If test is not part of your requirement, feel free to skip ahead. Test is where changes implemented by the administrator are tested by Information Technology folks who are more familiar with, or closer to, the end users. This can be done by support staff to verify common errors or regression issues weren't inadvertently introduced by the changes made, or by a Subject Matter Expert (SME) with knowledge of the changes being implemented, such as a developer. Anyone who deals with issues brought to their attention by the user population would be best here, as administrators of the image may be too far removed from the user population to truly understand or feel the effect of the problem.

UAT/Pilot​

This is where we get to the meat of the testing. UAT (User Acceptance Testing), or Pilot Testing, is where end users, generally those who agreed to be part of the testing process and therefore understand the somewhat mercurial nature of the process, test the changes made to the image. Pilot testers will generally live inside a pilot image 24/7 and will hopefully use the items that are being changed as to best provide valuable feedback before the image goes to production.

Production​

Production is where the bulk of your end users live and breathe. This is generally the final step in the Pilot Testing methodology, although you can always implement other verification steps at this step, not covered in this blog post, if you prefer.

Alright, I understand. Now what?​

With the terminology out of the way, let's get into the nitty gritty.

First, to stipulate, this blog post will cover implementation of the full Pilot Testing methodology, including the criminally underused Test step. If your process does not use the Test step, please adjust the information here according to your needs.

To apply a Pilot Testing methodology to Frame, you will utilize a minimum of three separate Frame Accounts. We shall call these Frame Accounts Dev/Test, Pilot (or "UAT" if you prefer), and Production.

You said there are four steps, but are only creating three Frame accounts, what's up with that?

You are absolutely correct, and I can see how that would be confusing, so let's discuss a newer feature of Frame called Test Publish. Now, to be clear, as of the writing of this blog the Test Publish feature is Early Access, which I don't normally recommend in a production workflow. While there may be changes in how this feature works behind the scenes, the purpose behind this function will stay relatively the same.

Test Publish is a feature of Frame where, upon a Publish, special workload VMs are created for the admin to verify changes made to the Sandbox are implemented as desired post-publish. This feature effectively eliminates the need for a fourth Frame Account specifically for development purposes. Once the Test Publish is verified, you can promote the publish to the Test users, as assigned through SAML permissions.

Please read the Test Publish documentation linked above very closely as the configuration for it is in a couple different places. If these instructions aren't followed, your Pilot Testing scenario may not work as desired.

Once the Development and Test steps are verified and working as desired, the next step is to make this image update available to UAT/Pilot. For this you will clone your Sandbox to another account, replacing the current Sandbox for the UAT/Pilot Account with the Sandbox from Dev/Test. During this clone process, you are able to immediately publish the image to the workload VMs, which I recommend. If you don't, it is highly recommended to not establish a session to the UAT/Pilot Sandbox, as it potentially introduces a change from the Dev/Test Sandbox.

At this point UAT/Pilot testers have access to the new image and can go through testing any updates to the applications/features as needed by their UAT status. Once approved, you can then repeat the steps above to promote directly from Dev/Test to Production, again with the recommendation to publish during the clone process.

That's pretty straightforward, anything else I should know?​

Potentially! Some applications are more complicated to get working in a VDI environment than in a physical environment. This can be due to a number of factors. Legacy 16-bit applications are problematic in the best of 32-bit operating systems, much less the 64-bit operating system required by Frame. Anti-virus applications are problematic because of how they run at install and what they do immediately afterward. In-house built applications may not be cognizant of a VDI environment and don't handle themselves the same way applications such as Google Chrome and Adobe Reader do, such as self-updating user-specific files in a machine-specific location, like Program Files.