This post is a brain dump of something I discovered working with Azure Cloud Services specifically, when deploying them to Azure with Octopus Deploy.

In the beginning.

Cloud Service deployments have been designed by Microsoft to provide a seamless upgrade experience.  If your cloud service infrastructure comprises of multiple cloud service instances, than the fabric controller in Azure which controls deployments will perform a rolling upgrade. The underlying instances will be gradually upgraded until all are done. When you are deploying to a production slot this all make sense. You want to avoid down time, you want to minimise impact for your customers. However this convenience comes at a cost – time. If you have a large number of Cloud Service instances and web roles the process seems to take for ever.  That is the last thing you want if you are watching over a Live release in an evening.

What other choices are there?

In some deployment scenarios you might deploy to the staging slot of the Cloud Service, do all of your testing and then perform a slot swap to get this version into Live. In this case you don’t want to incur the cost of a rolling upgrade as customers don’t use the staging slot.

The Cloud Service upgrade documentation talks about a deployment mode called Simultaneous. Unfortunately there is not a lot of documentation around that describes what it does. This Stack Overflow question highlights that simultaneous mode is referred to as BlastUpgrade in the topologyChangeDiscovery attribute in the Cloud Service’s Service Definition File. What I determined by experimenting is that in this mode, the fabric controller ignores all upgrade domains meaning all instances are upgraded at once. This was a lot quicker and exactly what I wanted when deploying to Staging slots.

So the obvious answer would be to update the service definition file? Wrong! This didn’t work with Octopus Deploy so I was faced with looking for other options. This led to an investigation of how Octopus Deploy actually deploys Cloud Services. I found that it uses this script by default.

function CreateOrUpdate()
{
    $deployment = Get-AzureDeployment -ServiceName $OctopusAzureServiceName -Slot $OctopusAzureSlot -ErrorVariable a -ErrorAction silentlycontinue
    if (($a[0] -ne $null) -or ($deployment.Name -eq $null))
    {
        CreateNewDeployment
        return
    }
    UpdateDeployment
}

function UpdateDeployment()
{
    Write-Verbose "A deployment already exists in $OctopusAzureServiceName for slot $OctopusAzureSlot. Upgrading deployment..."
    Set-AzureDeployment -Upgrade -ServiceName $OctopusAzureServiceName -Package $OctopusAzurePackageUri -Configuration $OctopusAzureConfigurationFile -Slot $OctopusAzureSlot -Mode Auto -label $OctopusAzureDeploymentLabel -Force
}

function CreateNewDeployment()
{

    Write-Verbose "Creating a new deployment..."
    New-AzureDeployment -Slot $OctopusAzureSlot -Package $OctopusAzurePackageUri -Configuration $OctopusAzureConfigurationFile -label $OctopusAzureDeploymentLabel -ServiceName $OctopusAzureServiceName
}

function WaitForComplete()
{

    $completeDeployment = Get-AzureDeployment -ServiceName $OctopusAzureServiceName -Slot $OctopusAzureSlot
    $completeDeploymentID = $completeDeployment.DeploymentId
    Write-Host "Deployment complete; Deployment ID: $completeDeploymentID"
}

CreateOrUpdate
WaitForComplete

You’ll notice calls to Set-AzureDeploymentwhere the modeparameter is set to Auto.However the documentation for this powershell cmdlet states that the optional mode argument can be set to Simultaneous. How do you get Octopus to do something different. Luckily if you drop a powershell script called DeployToAzure.ps1 into the root of the package you are deploying , Octopus will use your script rather than its own. Therefore you can adjust the script to look like this.

if ($UseSimultaneousUpgradeMode -eq "True")
{
    Write-Verbose "Using Simultaneous Upgrade Mode"
    Set-AzureDeployment -Upgrade -ServiceName $OctopusAzureServiceName -Package $OctopusAzurePackageUri -Configuration $OctopusAzureConfigurationFile -Slot $OctopusAzureSlot -Mode Simultaneous -label $OctopusAzureDeploymentLabel -Force
}
else
{
    Set-AzureDeployment -Upgrade -ServiceName $OctopusAzureServiceName -Package $OctopusAzurePackageUri -Configuration $OctopusAzureConfigurationFile -Slot $OctopusAzureSlot -Mode Auto -label $OctopusAzureDeploymentLabel -Force
}

Where $UseSimultaneousUpgradeMode is an Octopus variable that can be used to control which mode is used when.

One word of warning. You see the function in the script called WaitForComplete()? This is used by Octopus to determine when the release is complete. It works by querying the relevant AzureDeployment. I have found that this reports back as complete before the Cloud Service instances have upgraded. And if you were to swap from Staging to Production whilst they were still upgrading… oops you have a temporary outage. So if you are doing this remember to physically check the status of the staging deployment before swapping.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s