This project is read-only.
The Throttling Suite has a very simple setup that requires ThrottlingSuite.Http.dll and adding related configuration either via the code or web.config (app.config in case the self-hosting is used).

Index

Standard configuration using the code
Adding ThrottlingHandler into Web API pipeline
Standard configuration using *.config file
Current status and statistics
API Client: call rate correction
LogOnly Mode
Links
Throttling algorithm considerations

Standard configuration

The Throttling Suite (since v1.3.0) enables configuration using either the .NET code or *.config file.

Configure throttling from the code

You can provide throttling configuration from within the Web API Register method.

Configuration consists of three steps:
  • create collection of IThrottlingControllerInstance(s), where each instance contains:
    • controller - defines "how" to throttle HTTP requests;
    • controller's scope - defines "what" to throttle (the scope can be the entire application, or any subset of it);
  • then create ThrottlingConfiguration object with some basic global configuration options;
  • finally, set parameters for the RequestSignatureBuilder object;
The main object that provides throttling sevice is ThrottlingControllerSuite. It implements the IThrottlingService interface, and its constructor accepts the ThrottlingConfiguration object and collection of controller instances. Once an instance of IThrottlingService is created, it can be further used to create ThrottlingHandler and add it to the Web API pipeline as a DelegatingHandler.

public static class WebApiConfig
{
    public static void Register(HttpConfiguration config)
    {
        //configure Web API routes for the application
        //...

        //create throttling controllers
        IEnumerable<IThrottlingControllerInstance> instances = 
            new List<IThrottlingControllerInstance>(new[]
            {
                ThrottlingControllerInstance
                    .Create<ShortAverageThrottlingController>("api1", 1200, 100)
                        .IncludeInScope("api/content")
                        .ExcludeFromScope("api/content/logos"),

                ThrottlingControllerInstance
                    .Create<ShortAverageThrottlingController>("api2", 1000, 50)
                        .IncludeInScope("api/users", ScopeItemCondition.HttpGet 
                                                 | ScopeItemCondition.HttpOptions)
                        .IncludeInScope("api/orders", ScopeItemCondition.HttpGet 
                                                 | ScopeItemCondition.HttpOptions),

                ThrottlingControllerInstance
                    .Create<ShortAverageThrottlingController>("api3", 1000, 10) 
                        .IncludeInScope("api/users", ScopeItemCondition.HttpPost 
                                                 | ScopeItemCondition.HttpDelete 
                                                 | ScopeItemCondition.HttpPut)
                        .IncludeInScope("api/orders", ScopeItemCondition.HttpPost 
                                                  | ScopeItemCondition.HttpDelete 
                                                  | ScopeItemCondition.HttpPut)
            });

        //set configuration
        ThrottlingConfiguration throttleConfig = new ThrottlingConfiguration()
        {
            Enabled = true,
            LogOnly = false
        };
        throttleConfig.SignatureBuilderParams.IgnoreAllQueryStringParameters = true;
        throttleConfig.SignatureBuilderParams.IgnoreClientIpAddress = true;
        throttleConfig.SignatureBuilderParams.EnableClientTracking = false;
        throttleConfig.SignatureBuilderParams.UseInstanceUrl = true;
        
        //create throttling suite providing the service
        IThrottlingService throttle = 
            new ThrottlingControllerSuite(throttleConfig, instances);

        //add throttling handler into the main pipeline
        config.MessageHandlers.Add(new Handlers.ThrottlingHandler(throttle));
    }
}
Create ThrottlingControllerInstance
You can use one of 3 available ThrottlingController types: ShortAverageThrottlingController (most general use), LongAverageThrottlingController, and LinearThrottlingController depending on a throttling algorithm you'd like to use. See Throttling algorithm considerations for more details on different throttling algorithms (methods). Provide a name, timeIntervalMsec and maxThreshold parameters to the ThrottlingControllerInstance.Create method, where:
  • name - friendly name for the instance;
    • must be unique for each instance;
  • timeIntervalMsec - time interval as a boundary to measure the number of HTTP requests within it;
    • this must be an integer number in milliseconds;
  • maxThreshold - number of HTTP requests allowed to be processed within specified timeIntervalMsec;
    • this must be an integer number.
    • To set an unlimited throttling both values should be set to -1.
    • To completely block specific section of the application, both values should be set to 0.
  • another, optional, parameter resourcesCleanupInterval can be passed in to define a timespan for internal resource clean-up process;
    • this value is measured in milliseconds;
    • default value is 300,000 = 5 minutes;
    • an acceptable range of values is between 180,000 and 600,000.
Define ThrottlingScope
Each ThrottlingControllerInstance should be given one or more of ThrottlingScope items acting as HTTP requests filter. This filter defines the part of application that specific instance should throttle. HTTP requests can be filtered by URL, or its part, HTTP method, and client tracking data.
The scope item can be added either using URL path or pathRegex:
  • The path filter always assumes startsWith condition. For example, you may use path="manager/account" to filter all requests which path begins with 'account' located under 'manager' folder, including "manager/accounting" and "manager/accounts"
    • All paths must be relative to the application root.
    • The "*" denotes "all urls".
    • The throttling scope internally would account for the application virtual path root;
  • For the pathRegex you may use any custom Regular Expression as desired.
    • The pathRegex is used to evaluate HTTP request's relative path without leading "/" character; a custom pathRegex should be aware of it and handle appropriately; consider this example: HTPP request URL is <protocol://server>/apath/api/users/12345, the application virtual path root is apath, therefore the pathRegex will try to match api/users/12345.
The scope item also includes condition parameter accepting:
  • one of HTTP method values, for example ScopeItemCondition.HttpGet, ScopeItemCondition.HttpPut, etc.; this condition allows to customize throttling for particular HTTP method(s);
  • ScopeItemCondition.HasTracking or ScopeItemCondition.HasNoTracking; this condition allows to customize throttling for either: API clients requests contining tracking data, or without such;
  • multiple conditions can be combined together; HasTracking and HasNoTracking should not be provided at the same time. If the intent is to throttle an API with tracking and without, but provide different throttling thresholds, two separate controller instances shell be used.
Multiple scope items can be combined together into a single scope, either be "included" into the scope or "excluded" from it to shape the desired filter.

NOTE: Use ThrottlingControllerInstance's .IncludeInScope and .ExcludeFromScope methods to add the scope items, do not attempt to add items directly into the ThrottlingScope.Items collection. Doing so may lead to unexpected results. You can remove items from this collection if necessary, even at a runtime.
NOTE: It is important to note that the scope is not, by any means, an equivalent or an alternative to the routing, and should not be treated or thought of this way. Whether the routing is designed to precisely identify resource (controller, operation, parameters, etc.) that HTTP request is directed to, the scope has much larger span and enables to quickly separate large parts of Web API application that require different throttling rules. The scope concept has been introduced into this project to simplify ability to set throttling rules for large parts of Web API application, and performance advantages.
NOTE: The number of throttling instances theoretically is not limited, but it is recommended to keep their number relatively low (up to 8, in my opinion) to ensure web-site performance.
Review the parts from the code excerpt above
In the example above, the first instance is being created to handle all API requests within "api/content" root, except requests for "api/content/logos". The rate for this controller is set to 100 calls per 1.2 seconds.
    ThrottlingControllerInstance
        .Create<ShortAverageThrottlingController>("api1", 1200, 100)
            .IncludeInScope("api/content")
            .ExcludeFromScope("api/content/logos"),

The second instance should handle all API requests within "api/users" and "api/orders" roots when HTTP method is either GET or OPTIONS. The rate for this controller is set to 50 calls per second.
    ThrottlingControllerInstance
        .Create<ShortAverageThrottlingController>("api2", 1000, 50)
            .IncludeInScope("api/users", ScopeItemCondition.HttpGet 
                                     | ScopeItemCondition.HttpOptions)
            .IncludeInScope("api/orders", ScopeItemCondition.HttpGet 
                                     | ScopeItemCondition.HttpOptions),

The third instance is set to handle all API requests within the same, "api/users" and "api/orders" roots, but when HTTP method is POST, DELETE, or PUT. The rate for this controller is set to 10 calls per second.
    ThrottlingControllerInstance
        .Create<ShortAverageThrottlingController>("api3", 1000, 10) 
            .IncludeInScope("api/users", ScopeItemCondition.HttpPost 
                                     | ScopeItemCondition.HttpDelete 
                                     | ScopeItemCondition.HttpPut)
            .IncludeInScope("api/orders", ScopeItemCondition.HttpPost 
                                      | ScopeItemCondition.HttpDelete 
                                      | ScopeItemCondition.HttpPut)

There are few "global" configuration settings such as:
    ThrottlingConfiguration throttleConfig = new ThrottlingConfiguration()
    {
        Enabled = true,
        LogOnly = false
    };
  • the Enabled allows to enable or disable the entire throttling service; this parameter can be changed during runtime;
    • this value is false by default;
  • the LogOnly allows to record HTTP request "blocking" events, but avoid blocking HTTP request processing itself; see LogOnly Mode below;
    • this value is false by default;.
The rest of configuration parameters are related to the component known as RequestSignatureBuilder. The throttling suite uses this component to build unique identification for an HTTP request. This identity is further used by throttling controllers to relate its statistics to.

The requestSignatureBuilder uses the following configuration parameters:
  • IgnoreClientIpAddress - a flag whether client IP address should be included into an HTTP request signature allowing to throttle requests for either: all clients together (false), or separately for each client's IP (true).
  • UseInstanceUrl - if true (recommended), allows to ignore the actual request URL substituting one with Throttling controller instance scope. It simply means that all requests identified within the Throttling Scope of specific controller instance would be treated as a single URL and counted toward the same throttling limit. Otherwise (false) controller would maintain its own limit for each URL.
  • IgnoreAllQueryStringParameters - if true, ensures that any query string parameters present within API request, including oData parameters, will be ignored. The UseInstanceUrl = true also forces this behavior.
  • EnableClientTracking - the ThrottlingSuite includes built-in API client tracking functionality. This may provide an opportunity to throttle clients separately from each other. Set EnableClientTracking = true to turn on this functionality. Please take into consideration that the tracking works well when API is utilized by a browser, but could be inefficient when non-browser clients are used. A custom implementation of RequestSignatureBuilder class may provide integration with an existing client tracking mechanism that your application might already have in place.
Below are some common scenarios examples of setting the parameters for RequestSignatureBuilder.
Enable global throttling per application (for all clients) - these are default settings:
    throttleConfig.SignatureBuilderParams.IgnoreAllQueryStringParameters = true;
    throttleConfig.SignatureBuilderParams.IgnoreClientIpAddress = true;
    throttleConfig.SignatureBuilderParams.EnableClientTracking = false;
    throttleConfig.SignatureBuilderParams.UseInstanceUrl = true;

Enable per-client throttling:
    throttleConfig.SignatureBuilderParams.IgnoreAllQueryStringParameters = true;
    throttleConfig.SignatureBuilderParams.IgnoreClientIpAddress = false;
    throttleConfig.SignatureBuilderParams.EnableClientTracking = true;
    throttleConfig.SignatureBuilderParams.UseInstanceUrl = true;

Enable per-client throttling separately for each URL path:
    throttleConfig.SignatureBuilderParams.IgnoreAllQueryStringParameters = true;
    throttleConfig.SignatureBuilderParams.IgnoreClientIpAddress = false;
    throttleConfig.SignatureBuilderParams.EnableClientTracking = true;
    throttleConfig.SignatureBuilderParams.UseInstanceUrl = false;
NOTE: This last configuration throttles separately each unique URL path for each client. It is recommended to test this scenario before using it to see the outcome. There are clearly circumstances when this is a desired behavior, but it's not the most common scenario.
Adding ThrottlingHandler into Web API pipeline and route for ThrottlingStatisticsHandler
An instance of ThrottlingSuite.Http.Handlers.ThrottlingHandler is a DelegatingHandler that should be added to the Web API application pipeline. This will ensure Throttling Suite is being invoked upon application pipeline processes an HTTP request. In case your application contains multiple DelegatingHandler items, it is recommended that you add ThrottlingHandler at the top of the list. This would cut off excessive requests earlier in the process providing better overall application performance.

    using ThrottlingSuite.Http.Handlers;
    
    public static class WebApiConfig
    {
        public static void Register(HttpConfiguration config)
        {
            //configure Web API routes for the application
            config.Routes.MapHttpRoute(
                name: "DefaultApi",
                routeTemplate: "api/{controller}/{id}",
                defaults: new { id = RouteParameter.Optional }
            );
        
            //create throttling controller instances and configuration 
            // (as shown above)
            //...        

            //create throttling suite providing the service
            IThrottlingService throttle = 
                new ThrottlingControllerSuite(throttleConfig, instances);

            //add throttling handler into the main pipeline
            config.MessageHandlers.Add(new ThrottlingHandler(throttle));

            //set throttling statistics handler (this step is optional)
            config.Routes.MapHttpRoute(
                name: "ThrottlingStatEndpoint",
                routeTemplate: "internal/throttling",
                defaults: null,
                constraints: null,
                handler: new ThrottlingStatisticsHandler(throttle)
            );
        }
    }

NOTE: there is a slight difference in the IThrottlingService initialization in case the configuring comes from the *.config file rather from the code. This example demonstrates the difference, the rest of the code would be the same:
...
    //create throttling suite providing the service 
    //(we pass no parameters, so configuration will be taken from *.config file)
    IThrottlingService throttle = new ThrottlingControllerSuite();
...

Throttling configuration section in the *.config file

This is the standard example of web.config (app.config) file that includes all necessary configuration for Throttling Suite:
<?xml version="1.0"?>
<configuration>
  <configSections>
    <section name="throttling" type="ThrottlingSuite.Core.ThrottlingConfigurationHandler, ThrottlingSuite.Http" />
  </configSections>

  <throttling enabled="true" logOnly="false" resourcesCleanupInterval="300000" enableClientTracking="true">
    <ignoreParameters>
      <add name="rsid" />
    </ignoreParameters>
    <instances>
      <!--throttle at 2 calls per sec-->
      <!--method:ShortAverage/LongAverage/Linear-->
      <instance name="controller-API" timeIntervalMsec="1000" maxThreshold="2" method="ShortAverage">
        <add path="api" />
      </instance>
      <!--throttle at 200 calls per 5 secs-->
      <instance name="controller-APIv2" timeIntervalMsec="5000" maxThreshold="200" method="ShortAverage">
        <add path="api2" condition="HttpGet,HttpPost" />
      </instance>
    </instances>
    <requestSignatureBuilder type="ThrottlingSuite.Core.DefaultRequestSignatureBuilder, ThrottlingSuite.Http" ignoreClientIp="true" useInstanceUrl="true" ignoreAllQueryStringParameters="true"  />
  </throttling>
</configuration>

<!--resourcesCleanupInterval - in ms (300000 = 5 min - default; must be between 3 and 10 mins)-->
The first requirement is configuring the section for Throttling Suite. This is achieved by adding ThrottlingSuite.Core.ThrottlingConfigurationHandler configuration section handler mapped to throttling section added into configuration sections declaration.

Next step is configuring the throttling section. This section must have at least single Throttling Controller instance declared. The number of instances theoretically is not limited, but it is recommended to keep their number relatively low (up to 8, in my opinion) to ensure web-site performance.
Each instance must provide:
  • name - unique custom name for the instance;
  • timeIntervalMsec - the time interval (must be an integer number in milliseconds) as a boundary to measure the number of requests;
  • maxThreshold - the number of requests allowed to be processed within specified time interval (must be an integer number). To set an unlimited throttling both values should be set to -1. To prevent specific section of the application from being served, both values should be set to 0.
  • method - an optional attribute accepted by instance to specify the type of throttling controller used by the instance. The ShortAverage (default), LongAverage, and Linear are acceptable values. See Throttling algorithm considerations for more details on different throttling algorithms (methods).
  • each instance should be given one or more of path elements as a description of filtered paths that specific instance should handle throttling for. You can combine add and remove nodes to form a desired filter. The filter's path works as startsWith condition. For example, you may use path="manager/account" to filter all requests which path begins with 'account' located under 'manager' folder. All paths must be relative to the application root. The "*" denotes "all urls".
The add and remove filters also support RegularExpressions. The pathRegex attribute instead of path shell be provided with valid regular expression. Both values, path and pathRegex, may not be used at the same time.

The add and remove path (or pathRegex) node can include condition accepting:
  • one of HTTP method values, for example "HttpGet", "HttpPut", etc.; this condition allows to customize throttling for particular HTTP method(s);
  • "HasTracking" or "HasNoTracking"; this condition allows to customize throttling for either: API clients requests with tracking data, or without such;
  • multiple conditions can be combined as comma-separated value: condition="HttpGet,HttpPut,HttpDelete,HasTracking". "HasTracking" and "HasNoTracking" should not be provided at the same time - use none of these instead. If the intent is to throttle an API with tracking and without, but provide different throttling thresholds, it is advised to use two alternative controllers with the same add path (or pathRegex) node, whether in one controller the node contains "HasTracking" condition, and in another "HasNoTracking".
There is an additional configuration node requestSignatureBuilder. The type attribute within this node is used as a provider for building unique request signature. A vast majority of applications would be satisfied with default implementation supplied with Throttling Suite. You may provide custom type for this functionality should you have specific requirements for handling request's information, such as, but not limited by, client- or application-specific tracking data, etc. It is important to note that all additional attributes within requestSignatureBuilder xml node will be supplied into your custom implementation as configuration parameters. As such, the default builder supplied with Throttling Suite uses ignoreClientIp as a flag whether client IP address should be included into the request signature allowing to throttle requests for either: all clients together, or separately by each client's IP.

The useInstanceUrl="true" (recommended) allows to ignore the actual request URL substituting one with Throttling controller instance scope. It simply means that all requests identified within the scope of specific controller instance would be counted toward the same throttling limit. Otherwise (false) controller would maintain its own limit for each URL.

The ignoreAllQueryStringParameters="true" ensures that query string parameters present within API request, including oData parameters, will be ignored. The useInstanceUrl="true" also forces this behavior. If this parameter is set to "false", the Throttling Suite uses full URL, including QueryString to determine the request target. You may set the list of query string parameters that should be excluded from URL used by throttling controller. This option is provided via ignoreParameters configuration node.

The ThrottlingSuite includes built-in API client tracking functionality. This may provide an opportunity to throttle clients separately from each other. Set enableClientTracking="true" to turn on this functionality. Please take into consideration that the tracking works well when API is utilized by a browser, but could be inefficient when non-browser clients are used. A custom implementation of requestSignatureBuilder class may provide integration with an existing client tracking mechanism that your application might have already in place.

NOTE: After configuring the *.config file you should add throttling handler to Web API pipeline to complete the setup. Please refer to Adding ThrottlingHandler into Web API pipeline section for additional information.

Current status and statistics

The Throttling Suite provides internal ability to query current status and statistics for all controllers. Review the Adding ThrottlingHandler into Web API pipeline code section commented with "//set throttling statistics handler". Based on this configuration, sending HTTP request to the URL <root>/internal/throttling, the server will respond with similar JSON:
{
   "enabled":true,
   "logOnly":false,
   "ignoreParamsCount":1,
   "totalInstances":2,
   "instances":[
      {
         "name":"controller-API",
         "elapsed": 23.3,
         "created":"2014-05-13T21:51:35.910-04:00",
         "maxThreshold":5.0,
         "timeIntervalMsec":1000.0,
         "dictionarySize":20,
         "totalCalls":290,
         "blockedCalls":50
      },
      {
         "name":"controller-APIv2",
         "elapsed": 23.3,
         "created":"2014-05-13T21:51:35.910-04:00",
         "maxThreshold":200.0,
         "timeIntervalMsec":2000.0,
         "dictionarySize":30,
         "totalCalls":3990,
         "blockedCalls":87
      }
   ],
   "message":"Complete.",
   "machine":"TestWin2012"
}
It is fully at your descretion whether this URL is available for application - you may opt to remove this for security reasons, or, add permission verification layer on top of it. The path you configure to call this data is customizable; you choose what fits the best for your application.

API Client: call rate correction

Throttling Suite allows for Web API client to correct the actual calling rate by providing relevant instructions. Client not necessarily knows the call rate required by the API; and the API does not necessarily want to provide such. Therefore client might send many redundant requests without knowing how to prevent the failure. A meaningful data within the "Retry-After" response header would provide enough information for the API client to set appropriate delay (in seconds) before issuing the next request. It will boost the probability of success without excessive increase of number of calls. This is an example of the blocked API response in which server instructs the client to delay next request by 1 second:
HTTP/1.1 429 Too Many Requests
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
Retry-After: 1
Date: Wed, 21 May 2014 01:54:48 GMT
Content-Length: 0

Consider the following javascript code that takes advantage of the "Retry-After" response header to repeat an API request. (This sample is over-simplified implementation provided for demonstration purposes. Great open source re-try libraries are generally available with other projects online.)
    function getContent(id) {
        $.ajax('api/content/article/' + id)
            .success(function(data) {
                handleContentData(data);
            })
            .error(function(xhr, textStatus, errorThrown) {
                if (xhr.status == 429) {
                    var delay = request.getResponseHeader('Retry-After');
                    if (delay) {
                        //re-try again in "delay" seconds
                        setTimeout('getContent("' + id + '");', delay * 1000);
                    }
                } else {
                    //handle other errors...
                }
            });
    }

LogOnly Mode

Throttling configuration allows to set LogOnly mode to record "blocking" events, but avoid blocking request processing. This configuration is very useful as an exploration mode when you need to determine the most appropriate throttling settings and filters for your web application. Such configuration requires TRACE compilation and requires tracing to be wired up to the Web API as a valid ITraceWriter. The Request: _path_ has been blocked by _controllername_. message will appear in trace log for those calls that determined to be blocked by throttling suite.

IIS configuration (when IIS hosting is used)

The Throttling Suite issues HTTP Status 429 "Too many requests" should blocking conditions occur. By default IIS lacks this status at the default error list. This HTTP Status should be added to assure proper response by IIS. Follow these steps for IIS7/7.5:
1. Open %SystemDrive%\inetpub\custerr\ folder;
2. Create text file named 429.txt at this location; insert "Too many requests" text into the file;
3. Open IIS Manager >> Open (double-click) IIS>Error Pages section;
4. Once the Error Pages section opened, click Add button on the right panel to open "Edit Custom Error Page" dialog;
5. In the dialog window enter "429" into Status Code field; make sure the "Insert content from static file into the error response" option is selected under "Response Action"; enter "%SystemDrive%\inetpub\custerr\429.txt" into "File Path" entry box; click Save button.

You may also add the following section to web.config to assure Application level error messaging coming through the IIS:
<configuration>

  <system.webServer>
    <httpErrors existingResponse="PassThrough" />
  </system.webServer>
<configuration>


Last edited Jul 4, 2014 at 8:20 PM by lennygran, version 32