Cloud Load Test with Visual Studio Online - part 1: Web panel

This article is the first of a series of 3 where I'll write about the Cloud Load Testing with Visual Studio Online.

In this first part I'm going to talk about simple load tests execution, directly using the using the VSO portal.

Introduction

Before to start with the Cloud Load Test execution, it's important to specify a couple of aspects:

• To use the Cloud Load Test functionality you need a MSDN Ultimate subscription
• The application you want to test has to be public exposed on the web
• You can use a max of 15.000 minutes of load test per month. They are "virtual user minutes", which means thet, i.e., if you execute a 2 minutes test with a constant load of 200 virtual users, you're going to consume 400 minutes..

That said, let's see how to setup and execute those Load Test with the Visual Studio Online web interface.

The start

First of all, you have to log in into your VSO web portal and click on "Load test" on the main dashboard's menu.

Atfer that, it shows the settings page. How you can see, it's a single form: the available settings are a bit limited but more than sufficient to do a generic load test over our app.

The first parameter to set is the url of the page to test (it's not possible to execute a multi-step load test from this interface) which can be the home page or, like in the example, what page you want. The only constraint is that the page must be accessible on the internet without credentials.

The second value to insert is the test name: it a free text, useful as reminder.

Below of these params there are other 4, named "Test settings", useful to better set up the key aspects of the load to apply.

• User Load: with this setting you can define the number of concurrent users that will connect to the give url. Allowed values are 25, 50, 100 and 200
• Run duration: it's the duration of the entire test. Possible selections are from 1 to 5 minutes
• Think-time: it's the consecutive requests' delay time. It's useful to avoid the intervention of anti-hammering and anti DoS systems. It's possible to set a waiting time of 1 second (default) or 5 seconds
• Browser distribution: this selection sets the percentage of browser usage you want to simulate. Choosing, for example,"IE 80%, Chrome 20%" the test will be executed with agents that use Internet Explorer engine and Chrome engine by the given percentages

Setted up these setting, just click on the "Test now" button to start the test.

Test execution

But what happens when you start the test?

Visual Studio Online creates an on demand virtual lab in some Azure datacenter and configures the test agents on the VMs, with the given settings:

As soon as the lab is ready and configured, the test starts. The agents start to generate traffic to the given url and the results are posted back to our browser in near real time. In this way we can have a preview of the test outcome.

In the image we see, i.e., that at about 50 seconds there is a "hole" in the request per second that the site was able to manage, associated with a huge increase of the response time. Using these data we could start an analysis activity over our application or our infrastructure to understand why and where this anomaly happened.

Results

When the test completes, we are given the results and the achieved performances.

Near the chart we have seen before, we have a couple of other informations.

First, we have the average response time. A value under 0,1 seconds is considered good, between 1 second and 0,1 seconds is "not so good", more than 1 second is bad.

After the response time, there is the number of total requests done to the web app.

Finally, there is an indication of any requests that are not completed successfully or generated an error on the application.

Below these values there is also a table with the eventual errors.

In this example we don't have errors, but if they had occured we'll find them with the generic cause, the specific type and the given error message.

Conclusions

This kind of Load Test isn't the most complete we can do but it's however sufficient if we want to have some info about the performances and the load managed by our application

Moreover, the setup of these tests is extremely simple and in a while you can have a huge number of useful info.

Instead, if what you need is a deep load test where you can do a step-by-step navigation, maybe with credentials... don't miss my next article of this series.