Provision Azure Storage Account and automate file upload and deletion using PowerShell

Microsoft Azure Storage is a cloud offering from Microsoft that provides highly scalable, available, durable storage. Its a part of Microsoft Azure offerings. Azure Storage consists of three data services: Blob storage, File storage, and Queue storage. Blob storage supports both standard and premium storage, with premium storage using only SSDs for the fastest performance possible.

Now as is often the case with the cloud services, it comes at a cost. So you should be very careful in using only the space you need and not paying for extra storage consumption that you should not need to. Along with that, you should be able to automate it. In this blog post, we’ll learn how to create an Azure Storage account, uploads some files to it in the blob storage and then delete them all using PowerShell. 

Login to Azure Resource Manager

For this, we’ll need to run cmdlet Login-AzureRMAccount. This will create a pop-up box where you can enter your credentials to authenticate. Once done, you may need to run cmdlet Set-AzureRMContext to set the azure context to the subscription under which you want to create azure storage account.

Create an Azure Resource Group

First, since everything is a resource in Azure Resource Model, we would need to have a resource group that could be used to hold our storage account. We can do that using below code:

This should throw an output like below:

creates an azure resource group

Creates an Azure Storage Account

Microsoft provides various types of storage for same and you need to be a little bit familiar with kind of options that you would need to use. You can read about it here. We can use below code to create a general purpose storage account for us:

On success, this should give a response like below:

create azure resource account
create azure resource account

Create Azure Blob Storage Container(s)

First we need to identify connection information for the storage account. For this we need to use either storage account key or the endpoint. This information can be found from storage account in the Portal itself:

Identify storage account connection details
Identify storage account connection details

We would need to set azure storage context using above key using below commands:

creates new azure storage container

Upload files to Blob Storage Container

For this demo, I would like to upload everything that’s stored under directory C:\Source\ConsoleApp01 on my local machine. It contains few hidden files, few files and sub-directories with files:

Directory structure on my local machine

We can choose to upload files with preservation of hierarchy structure or not. To do that, we can use below command:

The magic number 23 in remove() function is the number of characters we need to remove from full name of the file. This would change depending upon your file path.

Below is the output for both:

files upload to azure without preserving file hierarchy

And with second option to preserve hierarchy:

files upload to azure with preserving file hierarchy

File Deletion from Blob Storage Container

More often that note, we would need to remove file based on a certain criteria. Like based on a path structure or timestamp etc. For purpose of this demo, we would like to delete everything older than 1 minute (Since we just uploaded files to azure). You can change the filters as per your needs.

First, we can get blob list using below command:

Get-AzureStorageBlob -Container $containerName -Context $strContext

Then we can filter it based on timestamp, say older than 1 minute:

Get-AzureStorageBlob -Container $containerName -Context $strContext | Where-Object {$_.LastModified -le ($(Get-Date).AddMinutes(-1))}

And finally, removing the content:

Remove-AzureStorageBlob -Blob $($blob.Name) -Container $containerName -Context $strContext

So our command would look like:

Removed files from azure blob storage

We have also added a Write-Host command for purpose of auditing.

Once deleted, we can verify the change from Azure Portal too:

Removed files from azure blob storage - 2

One thought on “Provision Azure Storage Account and automate file upload and deletion using PowerShell

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s