Monday, January 19, 2015

Cloud Load Test with Visual Studio Online - part 1: Web panel

This article is the first of a series of 3 where I'll write about the Cloud Load Testing with Visual Studio Online.

In this first part I'm going to talk about simple load tests execution, directly using the using the VSO portal.

Before to start with the Cloud Load Test execution, it's important to specify a couple of aspects:

  • To use the Cloud Load Test functionality you need a MSDN Ultimate subscription
  • The application you want to test has to be public exposed on the web
  • You can use a max of 15.000 minutes of load test per month. They are "virtual user minutes", which means thet, i.e., if you execute a 2 minutes test with a constant load of 200 virtual users, you're going to consume 400 minutes..

That said, let's see how to setup and execute those Load Test with the Visual Studio Online web interface.

The start
First of all, you have to log in into your VSO web portal and click on "Load test" on the main dashboard's menu.

Atfer that, it shows the settings page. How you can see, it's a single form: the available settings are a bit limited but more than sufficient to do a generic load test over our app.

The first parameter to set is the url of the page to test (it's not possible to execute a multi-step load test from this interface) which can be the home page or, like in the example, what page you want. The only constraint is that the page must be accessible on the internet without credentials.

The second value to insert is the test name: it a free text, useful as reminder.

Below of these params there are other 4, named "Test settings", useful to better set up the key aspects of the load to apply.

  • User Load: with this setting you can define the number of concurrent users that will connect to the give url. Allowed values are 25, 50, 100 and 200
  • Run duration: it's the duration of the entire test. Possible selections are from 1 to 5 minutes
  • Think-time: it's the consecutive requests' delay time. It's useful to avoid the intervention of anti-hammering and anti DoS systems. It's possible to set a waiting time of 1 second (default) or 5 seconds
  • Browser distribution: this selection sets the percentage of browser usage you want to simulate. Choosing, for example,"IE 80%, Chrome 20%" the test will be executed with agents that use Internet Explorer engine and Chrome engine by the given percentages

Setted up these setting, just click on the "Test now" button to start the test.

Test execution
But what happens when you start the test?
Visual Studio Online creates an on demand virtual lab in some Azure datacenter and configures the test agents on the VMs, with the given settings:

As soon as the lab is ready and configured, the test starts. The agents start to generate traffic to the given url and the results are posted back to our browser in near real time. In this way we can have a preview of the test outcome.

In the image we see, i.e., that at about 50 seconds there is a "hole" in the request per second that the site was able to manage, associated with a huge increase of the response time. Using these data we could start an analysis activity over our application or our infrastructure to understand why and where this anomaly happened.

When the test completes, we are given the results and the achieved performances.
Near the chart we have seen before, we have a couple of other informations.

First, we have the average response time. A value under 0,1 seconds is considered good, between 1 second and 0,1 seconds is "not so good", more than 1 second is bad.

After the response time, there is the number of total requests done to the web app.

Finally, there is an indication of any requests that are not completed successfully or generated an error on the application.

Below these values there is also a table with the eventual errors.

In this example we don't have errors, but if they had occured we'll find them with the generic cause, the specific type and the given error message.

This kind of Load Test isn't the most complete we can do but it's however sufficient if we want to have some info about the performances and the load managed by our application
Moreover, the setup of these tests is extremely simple and in a while you can have a huge number of useful info.

Instead, if what you need is a deep load test where you can do a step-by-step navigation, maybe with credentials... don't miss my next article of this series.

1 comment:

Hạ Thu said...

Thanks for sharing, nice post!

Tọa lạc tại địa chỉ 61C Tú Xương. P. 7, Quận 3, TP. HCMCasanova là quán cafe dep được thiết kế hoàn toàn theo phong cách độc đáo của nước Ý, đây là quán cafe yen tinh không gian cổ điển đẹp hay ca phe cappuccino với hương vị tuyệt hảo hay đây là quán nhung quan cafe dep sai gon với không gian tuyệt đẹp hay bạn có biết lợi ích của việc uống cà phê đối với sức khỏe chưa, hay Casanova Cafe – không gian cafe làm việc lý tưởng của freelancer hay đây là quán cafe chụp ảnh không tính phí tại sài gòn bạn có thể tha hồ chụp ảnh, đây là cà phê con hẻm nhỏ cực đẹp bạn không nên bỏ qua hay đây là quán cafe tinh nhan o sai gon thích hợp hẹn hò hay đây là quán cafe học nhóm tphcm cực hợp có phòng riêng hay là 1 trong quán cafe lãng mạn sài gòn với đồ uống giá rẻ với thức uống đặc trưng cực ngon là trà đào cam sả uống 1 lần là nghiền hay đây là điểm hẹn cafe cuối tuần ở sài gòn với Casanova Cafe hay meo giup be ngu ngon giúp bé ngủ ngon giấc hay nôi võng giúp bé ngủ ngon hay cafe tổ chức sinh nhật tphcm cực lý tưởng hay cà phê sân vườn sài gòn bạn không nên bỏ qua hay bạn muốn học bài thì quán cafe yên tĩnh để học bài cực thú vị đấy hay quán cafe view dep sai gon bạn không nên bỏ qua hay quán cafe có đàn piano ở tphcm siêu lãng mạn hay casanovacafe là 1 trong các quán cafe đẹp giá rẻ ở tphcm bạn nên đến 1 lần.