In this blog post I’ll show you how you can play back the IIS Logs in Visual Studio to automatically generate the web performance tests. You can also download the sample solution I am demo-ing in the blog post.

Introduction

Performance testing is as important for new websites as it is for evolving websites. If you already have your website running in production you could mine the information available in IIS logs to analyse the dense zones (most used pages) and performance test those pages rather than wasting time testing & tuning the least used pages in your application.

What are IIS Logs

To help with server use and analysis, IIS is integrated with several types of log files. These log file formats provide information on a range of websites and specific statistics, including Internet Protocol (IP) addresses, user information and site visits as well as dates, times and queries. If you are using IIS 7 and above you will find the log files in the following directory C:\Interpub\Logs\

Walkthrough

1. Download and Install Log Parser from the Microsoft download Centre. You should see the LogParser.dll in the install folder, the default install location is C:\Program Files (x86)\Log Parser 2.2. LogParser.dll gives us a library to query the iis log files programmatically. By the way if you haven’t used Log Parser in the past, it is a is a powerful, versatile tool that provides universal query access to text-based data such as log files, XML files and CSV files, as well as key data sources on the Windows operating system such as the Event Log, the Registry, the file system, and Active Directory. More details…

2. Create a new test project in Visual Studio. Let’s call it IISLogsToWebPerfTestDemo.  

3.  Delete the UnitTest1.cs class that gets created by default. Right click the solution and add a project of type class library, name it, IISLogsToWebPerfTestEngine. Delete the default class Program.cs that gets created with the project.

4. Under the IISLogsToWebPerfTestEngine project add a reference to

• Microsoft.VisualStudio.QualityTools.WebTestFramework – c:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PublicAssemblies\Microsoft.VisualStudio.QualityTools.WebTestFramework.dll
• LogParser also called MSUtil - c:\users\tarora\documents\visual studio 2010\Projects\IisLogsToWebPerfTest\IisLogsToWebPerfTestEngine\obj\Debug\Interop.MSUtil.dll

5. Right click IISLogsToWebPerfTestEngine project and add a new classes – IISLogReader.cs

The IISLogReader class queries the iis logs using the log parser.

using System;
using System.Collections.Generic;
using System.Text;
using MSUtil;
using LogQuery = MSUtil.LogQueryClassClass;
using IISLogInputFormat = MSUtil.COMIISW3CInputContextClassClass;
using LogRecordSet = MSUtil.ILogRecordset;
using Microsoft.VisualStudio.TestTools.WebTesting;
using System.Diagnostics;

namespace IisLogsToWebPerfTestEngine
{
    // By making use of log parser it is possible to query the iis log using select queries
  
    public class IISLogReader
    {
        private string _iisLogPath;

        public IISLogReader(string iisLogPath)
        {
            _iisLogPath = iisLogPath;
        }

        public IEnumerable<WebTestRequest> GetRequests()
        {
            LogQuery logQuery = new LogQuery();
            IISLogInputFormat iisInputFormat = new IISLogInputFormat();

            // currently these columns give us suffient information to construct the web test requests

            string query = @"SELECT s-ip, s-port, cs-method, cs-uri-stem, cs-uri-query FROM " + _iisLogPath;

            LogRecordSet recordSet = logQuery.Execute(query, iisInputFormat);

            // Apply a bit of transformation 
            while (!recordSet.atEnd())
            {
                ILogRecord record = recordSet.getRecord();
                if (record.getValueEx("cs-method").ToString() == "GET")
                {
                    string server = record.getValueEx("s-ip").ToString();
                    string path = record.getValueEx("cs-uri-stem").ToString();
                    string querystring = record.getValueEx("cs-uri-query").ToString();

                    StringBuilder urlBuilder = new StringBuilder();
                    urlBuilder.Append("http://");
                    urlBuilder.Append(server);
                    urlBuilder.Append(path);
                    if (!String.IsNullOrEmpty(querystring))
                    {
                        urlBuilder.Append("?");
                        urlBuilder.Append(querystring);
                    }

                    // You could make substitutions by introducing parameterized web tests. 
                    WebTestRequest request = new WebTestRequest(urlBuilder.ToString());
                    Debug.WriteLine(request.UrlWithQueryString);
                    yield return request;
                }

                recordSet.moveNext();
            }

            Console.WriteLine(" That's it! Closing the reader");
            recordSet.close();
        }
    }
}

 

6. Connect the dots by adding the project reference ‘IisLogsToWebPerfTestEngine’ to ‘IisLogsToWebPerfTest’. Right click the ‘IisLogsToWebPerfTest’ project and add a new class ‘WebTest1Coded.cs’

The WebTest1Coded.cs inherits from the WebTest class. By overriding the GetRequestMethod we can inject the log files to the IISLogReader class which uses Log parser to query the log file and extract the web requests to generate the web test request which is yielded back for play back when the test is run.

namespace IisLogsToWebPerfTest
{
    using System;
    using System.Collections.Generic;
    using System.Text;
    using Microsoft.VisualStudio.TestTools.WebTesting;
    using Microsoft.VisualStudio.TestTools.WebTesting.Rules;
    using IisLogsToWebPerfTestEngine;

    // This class is a coded web performance test implementation, that simply passes 
    // the path of the iis logs to the IisLogReader class which does the heavy 
    // lifting of reading the contents of the log file and converting them to tests. 

    // You could have multiple such classes that inherit from WebTest and implement 
    // GetRequestEnumerator Method and pass differnt log files for different tests.
    public class WebTest1Coded : WebTest
    {
        public WebTest1Coded()
        {
            this.PreAuthenticate = true;
        }

        public override IEnumerator<WebTestRequest> GetRequestEnumerator()
        {
            // substitute the highlighted path with the path of the iis log file 
            IISLogReader reader = new IISLogReader(@"C:\Demo\iisLog1.log");
            foreach (WebTestRequest request in reader.GetRequests())
            {
                yield return request;
            }
        }
    }
}

 

7. Its time to fire the test off and see the iis log playback as a web performance test. From the Test menu choose Test View Window you should be able to see the WebTest1Coded test show up. Highlight the test and press Run selection (you can also debug the test in case you face any failures during test execution).

8. Optionally you can create a Load Test by keeping ‘WebTest1Coded’ as the base test.

Conclusion

You have just helped your testing team, you now have become the coolest developer in your organization! Jokes apart, log parser and web performance test together allow you to save a lot of time by not having to worry about what to test or even worrying about how to record the test. If you haven’t already, download the solution from here. You can take this to the next level by using LogParser to extract the log files as part of an end of day batch to a database. See the usage trends by user this solution over a longer term and have your tests consume the web requests now stored in the database to generate the web performance tests. If you like the post, don’t forget to share … Keep RocKiNg!