SharePoint projects have a tradition for being tested in a test environment and being deployed to one ore more staging environments before hitting production. I have never been a huge fan of this approach, as it is time consuming and in many cases doesn’t improve the quality of the project enough to justify the more elaborate deployment process. I won’t dig into the details of why I think the quality isn’t necessarily improved by this deployment/testing strategy, instead I just want to introduce what I think is better and more modern approach.
Canary Release
As the title of this post indicates, I think a better approach is testing in production with a so-called canary release. If you are unfamiliar with the concept I can recommend that you read CanaryRelease on Martin Folwers website.
All the big boys in the software industry (Microsoft: Visual Studio Online/SharePoint Online, Facebook etc.) use Canary Releases, it allows them to test new features/bug fixes on a subset of their users, in the production environment. If the changes work they can roll them out to everyone, if they don’t or they have undesired side-effects they can quickly do a roll back.
The positive effect of this deployment/testing strategy is in my opinion many, here is a few:
- It forces you to implement proper logging/feedback logic in your application so you can measure if it works as desired.
- It forces you to have a repeatable and automated deployment process
- Your changes will be tested better, because you are doing it in the production environment, no more excuses like: “ehh it worked in staging, it must be something with the environment”
- You will always, if done correctly, have the possibility to roll back, something which traditionally have been tricky with SharePoint
All of above doesn’t necessary mean that you can go without a test environment completely, it depends on how critical your application is and how big changes you are in the process of rolling out.
How to do it
Firstly, you need to embrace the SharePoint App model, Canary Release are never going to work well with anything but the most simple farm solutions.
Secondly, your life becomes a lot easier if you have a stable data model. If you are doing major changes to the data that is already in use, making a canary release just became a whole lot harder, but I guess that goes for any deployment :).
In the following I will describe how use canary release with a SharePoint provider hosted app, hosted with Azure Web Sites. Using Azure Web Sites as the hosting platform makes it very easy for us to make a canary release at this functionality is available out of the box. If you want to host your app in cloud web role or on a local IIS, you can obtain the same with a little more effort on your part as you have to setup and configure Application Request Routing (ARR) on IIS yourself.
Configuring an Azure Web Site for Canary Releases of a SharePoint Provider Hosted App
Let’s look at how we setup our Azure Web Site to support canary releases. In the following I will do everything through the new Azure Portal as I just want to focus on the concept, but of course you can do the same with PowerShell.
The first requirement we need to fulfill is that we our website must be on a at least a Standard Web Hosting Plan. The testing in production feature is not support by the Basic and Free plan.
Then when you have created your Azure Web Site that is going to host your SharePoint app, we need to configure it for “testing in production” as it’s called in the portal. (Note: it doesn’t matter if you create the website manually or from the wizard in Visual Studio).
Open the web site blade, and scroll down to Configuration and click testing in production.
From the new blade that opens, you can add a new deployment slot by clicking “Add Slot”.
The deployment slots can be considered different environment of the same solution, it’s per default just separate IIS websites and application pools hosted on the same Virtual Machine (VM), so be careful if you e.g. are doing stress testing against a testing slot. To avoid sharing VM, it’s possible to move a deployment slot to a different web hosting plan, which effectively will move it to another VM.
Before you add anything you only have the “production” slot, in my example I have added “test-sjkpsptestapp” which I have set to receive 50 % of the traffic going to the site. Typically you wouldn’t do a canary testing with 50 % of the traffic, 5-10 % would be better, but it of course depends on what you are testing, and how comfortable you are with it, maybe you are doing a ramp up scenario.
Here’s my two versions of the simple test App, in IE I have been served the new “v2” of the app, as v2 is appended to my name, while in chrome (right) I still get the old version without version number.
If you are deploying from within Visual Studio (which I hope you are not, if you are servicing a production application), it’s easy to deploy to the test slot as you simply create another publishing profile.
That is all there is to it from a conceptual point of view. Now you can have two different versions of the same SharePoint app deployed and you can decide the amount of traffic that goes to the new release. Once you are satisfied with the release, e.g. you have measured no errors and got no user complaints you can do a “Swap” which will set the test deployment as the new production deployment and reset the “testing in production” traffic percentages to direct all traffic to the new production deployment.
In another post I hope to be able to deliver more details on how to setup you source control system to support canary testing and give you details on how I would collect quality feedback from the release, and wrap it all together in an automated process. But I hoped that this post have inspired you to think of testing in a different way.