How to detect the start of a file download using javascript?

Long time ago I had a trouble detecting the start time of a file download with JavaScript. The requirement was to display a "Loading..." message while generating the file at server side. We were using ExpertPdf as a third-party component for converting HTML reports to PDF. The process time for large reports was taking about 7 seconds. So, displaying a loading message was a reasonable requirement we had to address. The following workflow clarifies the requirement:

1- User clicks on a download link
2- Client displays a "Loading..." message
3- Client sends a request to server for generating and returning the file.
(e.g. window.location = '@Url.Content("~/Download/Pdf")';)
4- Server returns the dynamically generated file to the client
5- Client hides the "Loading..." message
6- Client starts to download the file

Unfortunately JavaScript doesn't know when download starts then we cannot hide the message! All the Google searches were not promising at all.

As a first solution, I came up with using TempData. Here is the steps in this approach:

1- User clicks on a download link
2- Client displays a "Loading..." message
3- Client sends an ajax request to server for generating the file. Assume that the url is '@Url.Content("~/Download/PreparePdf")'
4- Server generates the report in pdf format and stores the file stream in TempData. Then returns a json response confirming that the file ready to download
5- Client hides the "Loading..." message and sends another request to download the file (e.g. window.location = '@Url.Content("~/Download/Pdf")';) which is a separate action method.
6- Download starts

At first glimpse everything looks fine and it gives the user a nice experience. After deploying the application to UAT environment and doing some load tests we found that some of the file downloads take more that expected (around 15 seconds). As you know, The more response time, the more requests waiting in thread pool.

Digging into the problem helped me to find out the root cause. If you look at the workflow above, the client calls the server twice (One call for generating the pdf server side and the other for downloading the file). If you have only one web server to serve the requests you won't see any issue. However, using the web-farm and load balancer will replicate the issue! The reason is simple. There is no guarantee for sending the second HttpRequest to the same server utilized for first HttpRequest. Hence, the second request will find TempData empty and then will try to generate the file. (This is just a defensive programming approach). That's why we experience a response time longer that before.

One solution to resolve the issue is using the centralized state server such as AppFabric or SqlServer. In this approach instead of storing the pdf stream in TempData we should store it in the state server which is accessible for all the web servers.

For some reasons I didn't want to use this approach. I still had a bad filling about sending two separate requests for just having a progress message client side!

I ended up with suing the cookies as my hero! Before explaining it let's have a look to a sample code for that.

Client-side code:


The Action code - server side:
public ActionResult Download()
{
     // set a cookie to notify client the server side process is done
     var requestToken = Request.Cookies["fileDownloadToken"];
     if (requestToken != null && long.Parse(requestToken.Value) > 0)
     {
         var responseTokenValue = long.Parse(requestToken.Value) * (-1);
         Response.Cookies["fileDownloadToken"].Value = responseTokenValue.ToString();
     }

     // Just a simulation of generating a pdf file - demo purpose
     Thread.Sleep(7000);
     FileStream fs = new FileStream("C:/Temp/DummyFile.pdf", FileMode.Open);
     byte[] buffer = new byte[fs.Length];
     fs.Read(buffer, 0, (int)fs.Length);
     fs.Close();
            
     return File(buffer, "application/pdf", "test.pdf");
}


Now let's dig into the code. First of all, I am using jQuery cookie to manipulate the cookies client side. The function in client side, creates a unique token (timestamp in this demo) and assign it to a cookie called fileDownloadToken. Then it creates an interval function to check the cookie every one second. It is expecting the same token that is multiplied by (-1). When that happens then downloadStarted function is called to clear the cookie and stop the checker function.

On the other side, inside the Action method, we first check the cookie and if it exists then multiply the value by (-1). After generating the pdf, we use the MVC File type to return the pdf file.

You may think this is end of the story but it is not! Taking 7 seconds for processing a request in ASP.NET is awful and it will cause the deadlocks and will increase the number of requests waiting to be served by the web server.

The simple solution to dramatically affect the performance is to leverage OutputCache feature in ASP.NET MVC. This is a very cool and handy feature that you can use to cache the output result of an Action. Please look at the detail info yourself.

Again, because we are using webfarm approach, I will use the AppFabric cache to persist the download results.

The problem with this scenario is that the Action code won't be called for cached urls. So, the cookie value will never be changed. As a result, the "Loading..." message will be up forever!

This issue can be solved by implementing the following HttpModule:
public class DownloadModule : IHttpModule
    {
        public void Dispose()
        {
        }

        public void Init(HttpApplication context)
        {
            context.PreSendRequestHeaders += new EventHandler(context_PreSendRequestHeaders);
        }

        void context_PreSendRequestHeaders(object sender, EventArgs e)
        {
            HttpApplication app = sender as HttpApplication;
            var requestToken = app.Request.Cookies["fileDownloadToken"];
            long requestTokenValue;
            if (requestToken != null && long.TryParse(requestToken.Value, out requestTokenValue))
            {
                if (requestTokenValue > 0)
                {
                    var responseTokenValue = requestTokenValue * (-1);
                    app.Response.Cookies["fileDownloadToken"].Value = responseTokenValue.ToString();
                }
            }
        }
    }

This module will check the cookie before sending the request headers. It doesn't matter if the request output result is cached or not. This event will be called always in the request pipeline.

The OutputCache will use the IIS output cache by default. We can customize it to use other caching technologies like AppFabric. It is as simple as inheriting from OutputCacheProvider and override 4 main methods. Here is a nice blog from ScottGu that describes how to extend the OutputCache in ASP.Net 4.


Enjoy coding...

Investigating performance bottlenecks in your web application - Part 2

In the previous post we talked about the performance tuning of the application by finding the bottlenecks in the code and modifying them to get a better response time. However, the code is not the only factor in performance tuning process. In this post I will list the other parameters that will affect the overall response time of the web requests.

application Release mode vs Debug mode

Make sure the assemblies are compiled in release mode. Debug and Release are different configurations for building your project. As the name implies, you generally use the Debug mode for debugging your project, and the Release mode for the final build for end users. The Debug mode does not optimize the binary it produces (as optimizations can greatly complicate debugging), and generates additional data to aid debugging. The Release mode enables optimizations and generates less (or no) extra debug data.

You should also make sure to set the debug mode to false in web.config:



 



IIS request pipeline mode
if you are using IIS 7+ to host the application it is highly recommended to have a separate application pool assigned to the web application and set the pipeline mode to Integrated with target .Net framework 4.0. IIS7+ supports both classic and integrated modes. Classic mode follows the request pipeline structure as in IIS6. Integrated mode follows the IIS7 new pipeline structure witch is completely different. The following articles will help you to understand the differences and features of each mode:

How to Take Advantage of the IIS 7.0 Integrated Pipeline
ASP.NET Thread Usage on IIS 7.0 and 6.0

If your application is now working in classic mode then you need to make some changes in config file to support integrated mode. The easiest way is using IIS appcmd command. In command prompt window type the following command :
%SystemRoot%\system32\inetsrv\appcmd migrate config "SiteName/"

[SiteName] is the name of your web site. This command will change the web.config file in the root of web site. By comparing the old and new config file you will find out that this command puts all your httpHandlers and httpModules in section.

For ASP.NET MVC applications make sure the modules section inside the contains the following parts:


      
      
      
      


machine.config
ASP.NET Process Model configuration defines some process level properties like how many number of threads ASP.NET uses, how long it blocks a thread before timing out, how many requests to keep waiting for IO works to complete and so on. The default is in many cases too limiting. Nowadays hardware has become quite cheap and dual core with gigabyte RAM servers have become a very common choice. So, the process model configuration can be tweaked to make ASP.NET process consume more system resources and provide better scalability from each server.

You can tweak the process model in machine.config file located in the $WINDOWS$\Microsoft.NET\Framework\$VERSION$\CONFIG directory. 10 ASP.NET Performance and Scalability Secrets is one of the best articles on the web that guides you very well.

In the Complete Guide .NET Performance and Optimization , Paul Glavich and Chris Farrell offer a comprehensive and essential handbook to anybody looking to set up a .NET testing environment and get the best results out of it, or just learn effective techniques for testing and optimizing their .NET applications. I strongly recommend this book to every web developer.

Investigating performance bottlenecks in your web application - Part 1

Recently I was responsible for improving the performance of a website implemented with ASP.NET MVC. Although the performance improvement does not have a straight forward approach and it strongly depends on your application structure and physical topology, here are the steps I chose and I believe they are valid in most applications with the same structure. Before I list these steps let me describe the structure of our case study:

We are using:
2 Web Servers: Windows Server 2008 R2, ASP.NET MVC 3 hosted in IIS 7
2 App Servers: Windows Server 2008 R2, WCF services hosted in IIS 7
2 DB Server: SQL Server 2008 R2
2 Reporting Servers: Sql Server Reporting Servers (SSRS)


The preliminary load test results revealed that the system was unable to cope beyond 50 concurrent users. The CPU usage in application servers was always flat around 100%. As a first rule, when you experience the high CPU usage the first thing you should blame is the APPLICATION itself! So you have to leverage your investigation skills to find the bottlenecks in the code.

An average CPU utilization of 20–30% on a web or application server is excellent, and
50–70% is a well utilized system. An average CPU utilization above 75% indicates that a system is reaching its computational capacity; however, there's usually no need to panic if you have an average of 90–95%, as you may be able to horizontally scale out by adding an additional server.

As long as we are using Windows Communication Foundation (WCF) in the application layer we can easily leverage the WCF diagnostic option to investigate the long running requests. To turn on the tracing on WCF services you need to add the following section in web.config file.


   
      
            
            
               
            
         
      
   


Here is more information about WCF diagnostics in MSDN.

As you see we log the tracing results in Tracing.svclog file. To view the content of this file you have to use SvcTraceViewer.exe located in "C:\Program Files\Microsoft SDKs\Windows\v6.0\Bin" folder.

You can easily use this powerful tool to analyse the diagnostic traces that are generated by WCF. To get a reliable result you need to run the application for a reasonable period of time while the wcf tracing is on. In this way you could find out the number of method calls along with the process duration for each method. Choose the methods with unexpected duration time and try to modify the method to achieve a better performance.

Sometimes the service operation calls the other services or hits the database to fetch or update the data. So you will need to find out witch part of the service operation causes the bottleneck. Is it the database call or the operation process itself? You can plan different approaches based on your coding structure.

Using WCF tracing will help you to nominate the potential bottlenecks in service level. What if the bottleneck is in the other layers such as Repositories. As you may know people are using repository design patterns to separate the logic that retrieves the data and maps it to the entity model from the business logic that acts on the model. Persistence ignorance is the benefit of using this pattern. Hence we need to do tracing in repository level as well. If you are familiar with Aspect Oriented Programming, known as AOP, you can implement an Interceptor to inject a custom logger into all methods of your repositories. This logger wraps the service method and calculates the duration of the process.

As an example, let's assume that we are using Castle.Windsor as an IoC container. Here is the castle Interceptor I've implemented to do the job for us:

public class FunctionalTraceInterceptor : IInterceptor
  {
     public void Intercept(IInvocation invocation)
     {
         StringBuilder logEntry = new StringBuilder();
         Stopwatch stopWatch = new Stopwatch();
         stopWatch.Start();

         invocation.Proceed();

         stopWatch.Stop();
         double duration = stopWatch.Elapsed.TotalMilliseconds;

         logEntry.AppendLine(String.Format("{0},{1},{2}", invocation.InvocationTarget.ToString(), invocation.MethodInvocationTarget.Name, duration));
         ApplicationLogger.Inst.LogInfo("TraceAllFunctionCalls", logEntry.ToString());
    }
  }

IInterceptor is an interface located in Castle.Core.Interceptor namespace. ApplicationLogger is our singleton logger that uses log4net. You can implement your own logger instead. InvocationTarget is the service and MethodInvocationTarget is the service method.

To inject this interceptor into the services you can simply define it in castle configuration file:



  
    
        
          ${FunctionalTraceInterceptor}
        
    

    
    
  


SampleRepository is the repository that has implemented the ISampleRepository interface and we want to inject the interceptor to all of its public methods. You can use this solution for all the services you registered in IoC container.

The purpose of this post was introducing two common techniques for tracing the service calls. However the application is one of the components you need to modify to get a better performance. There are other parameters in performance improvement process that I will mention in the next posts.