Custom Throttling Policies with WSO2 API Manager

Hello Geeks!!! 😁😁😁

As you know, WSO2 API Manager is the world’s leading open-source, enterprise-grade API management platform for on-premises, cloud, and hybrid architectures.

Most of time, we need to add throttling policies for API Subscribers to manage API resources properly. We can add throttling policies for Resource level, API level, subscription level, application level etc… However, If we need to add a same policies to all the client IP addresses, we cannot directly declare the policy. (Because we don’t know all the client IP addresses) 🥴🥴🥴

Eg: If we need to add a policy for an API/API resource (5 requests per minute) for each client IP address, we cannot add all client IP addresses into the throttling policy via admin console.

So, how to achieve this??? 🤔🤔🤔

OK. We can do this in BOSS LEVEL!!! 😎😎😎 Let’s do.

🌟 For the demonstrate purpose, I will use the WSO2 API Manager 2.6.0 version WUM timestamp 1599232507417.

• Start your API Manager.
• Go to the admin console. (https://<APIM_HOST>:<APIM_PORT>/admin)

Eg: https://localhost:9443/admin

• Click on THROTTLING POLICIES -> CUSTOM POLICIES

Custom policies dashboard — WSO2 API Manager 2.6.0 Admin Console

• Click on DEFINE POLICY button.

Define custom policies dashboard — WSO2 API Manager 2.6.0 Admin Console

Mandatory Fields

1. Name — Name for the custom throttling policy.
2. Key Template — You have to provide the throttle key format. Will explain this with the Siddhi Query. 😇😇😇
3. Siddhi Query — This is the section that you have to define the custom throttling policy.

• Now you came to start the big part of this throttling policy. Let’s start.

If you click on the Show Sample button, you can see a sample Siddhi query with the relevant use-case as below.

Sample Siddhi Query — WSO2 API Manager 2.6.0 Admin Console

As you can see, the above siddhi query applied a filter to userID. Then create a throttleKey based on the above condition and add into the throttle eligible list.

str:concat(‘admin@carbon.super’,’’)

This is the throttleKey (this is the value need to configure to Key Template field above. But need to do some changes. Will explain it.) 🤓🤓🤓

• timeBatch(1 min) — this is the throttling policy time period.
• (count(userId) >= 5) as isThrottled — This is the throttling request count checker. (5 requests for each userID)

Therefore the summary of above Siddhi query is, this policy is applied for the admin user who eligible to send only 5 requests per 1 minute.

OK. Now we start to write a Siddhi query to apply throttling policy for each IP Address. (with API resource filter) 😎😎😎

FROM RequestStream
SELECT userId, (resourceKey == ‘/api/1.0.0/1.0.0/*:GET’ ) AS isEligible ,
str:concat(cast(map:get(propertiesMap,’ip’),’Long’),’:’,resourceKey) as throttleKey
 
INSERT INTO EligibilityStream;
FROM EligibilityStream[isEligible == true]#throttler:timeBatch(1 min)
SELECT throttleKey, (count(userId) >= 5) as isThrottled, expiryTimeStamp group by throttleKey
INSERT ALL EVENTS into ResultStream;
 
from ResultStream#throttler:emitOnStateChange(throttleKey, isThrottled)
select * insert into GlobalThrottleStream;

• Add this query to the Siddhi query field in the define custom policy window.

Explanation of the above Siddhi query

SELECT userId, (resourceKey == ‘/api/1.0.0/1.0.0/*:GET’ ) AS isEligible

• This throttling policy is applied for the user who is executing the API resource ‘/*’ in GET method via any subscribed application. (API context is /api and the API version is 1.0.0)

💡I will explain how to obtain this magical resourceKey later. 😉😉😉

str:concat(cast(map:get(propertiesMap,’ip’),’Long’),’:’,resourceKey) as throttleKey

• Get the client’s IP from the propertiesMap and cast it into a long value.
• Then append ‘:’ and the resourceKey and initialize the entire contact value as throttleKey.
• Therefore, the throttleKey format is,

<IP_in_LONG>:<API_Resource_Key>

The summary of the above Siddhi query is, for each subscriber who is executing the API resource ‘/*’ in GET method via any subscribed application is only capable to obtain maximum of 5 requests per 1 minute. (API context is /api and the API version is 1.0.0)

• Now we have to configure the Key Template.

Before that, you can check the supported keys for the key template by point to the question mark near to the Key Template field as below.

Supported Keys for Key Template — WSO2 API Manager 2.6.0 Admin Console

As you can see, this API manager supports for clientIp. Now you can define the key template.

• According to the above Siddhi query, the throttleKey is,

<IP_in_LONG>:<API_Resource_Key>

• Therefore, the key template should be,

$clientIp:$resourceKey

Finally, click on Apply Rule button.

Custom throttling policy with information — WSO2 API Manager 2.6.0 Admin Console

OK. Now you have applied a custom throttling policy with a resource filter. 😎😎😎

Let’s move to the publisher portal.

• Go to the publisher portal. (https://<APIM_HOST>:<APIM_PORT>/publisher)

In Design tab,

• Create a REST API with the context /api
• Create a GET resource. (path is /*)
• Click on Next:Implement button.

Create a REST API with API Resources — WSO2 API Manager 2.6.0 Publisher Portal

In Implement tab,

• Add an endpoint to the API and click on Next:Manage button.

Add an endpoint to the API — WSO2 API Manager 2.6.0 Publisher Portal

In Manage tab,

• In Throttling Settings section, put a tick to a subscription tier.
• Click on Save & Publish button.

Now your API has been published. You can check the custom throttling policy by invoking the published API resource with X-Forwarded-For header key. (X-Forwarded-For — this header key is used to manually define the client IP address.)

Request header of the Postman Hit

After 5 requests per 1 minute, you can see the below response.

<amt:fault xmlns:amt=”http://wso2.org/apimanager/throttling">
<amt:code>900806</amt:code>
<amt:message>Message throttled out</amt:message>
<amt:description>You have exceeded your quota</amt:description>
<amt:nextAccessTime>2020-Dec-17 08:34:45+0000 UTC</amt:nextAccessTime>
</amt:fault>

You can check the policy by changing the value(IP address) of the X-Forwarded-For header key.

Please go through the documentation [1] for more information about custom throttling policies.

How to obtain the API resourceKey??? 🤔🤔🤔

• Log into the management console. (https://<APIM_HOST>:<APIM_PORT>/carbon)
• Inside the Main section, click on Event -> Publishers
• Click on Add Event Publisher

In the Create a New Event Publisher window,

• Give a name to the event publisher.
• Select org.wso2.throttle.processed.request.stream:1.0.0 as the Event Source. Then you can see the stream attributes of the above event.
• Select logger as the Output Event Adapter Type.
• Click on Add Event Publisher.

Now try to invoke the published API. In the console, you can see the logger output as below.

logger event publisher output

You can see the resourceKey value in the above output.

💎 The above hack is only to obtain the API related information. After that, delete the published event. Otherwise, that will impact to the API Manager performance.

Additional Hints for Filtering

Below add some sample use case with corresponding Siddhi query and key template to implement custom throttling policies.

Eg 1 : Create a throttling policy for all the users who invoke an API resource declared with the API context /api/1.0.0 (maximum 5 requests per 1 minute for each client)

FROM RequestStream
SELECT userId, ((resourceKey == ‘/api/1.0.0/1.0.0/*:GET’ ) AND (apiContext == ‘/api/1.0.0’ )) AS isEligible, 
str:concat(cast(map:get(propertiesMap,’ip’),’Long’),’:’,resourceKey, ’:’, apiContext) as throttleKey
 
INSERT INTO EligibilityStream;
FROM EligibilityStream[isEligible == true]#throttler:timeBatch(1 min)
SELECT throttleKey, (count(userId) >= 5) as isThrottled, expiryTimeStamp group by throttleKey
INSERT ALL EVENTS into ResultStream;
 
from ResultStream#throttler:emitOnStateChange(throttleKey, isThrottled)
select * insert into GlobalThrottleStream;

The Key Template for the above Siddhi query is,

$clientIp:$resourceKey:$apiContext

Eg 2 : filter all the users who invoke any API resource declared with the API context /api/1.0.0 (maximum 5 requests per 1 minute for each client)

FROM RequestStream
SELECT userId, (apiContext == ‘/api/1.0.0’ ) AS isEligible, 
str:concat(cast(map:get(propertiesMap,’ip’),’Long’), ’:’, apiContext) as throttleKey
 
INSERT INTO EligibilityStream;
FROM EligibilityStream[isEligible == true]#throttler:timeBatch(1 min)
SELECT throttleKey, (count(userId) >= 5) as isThrottled, expiryTimeStamp group by throttleKey
INSERT ALL EVENTS into ResultStream;
 
from ResultStream#throttler:emitOnStateChange(throttleKey, isThrottled)
select * insert into GlobalThrottleStream;

The Key Template for the above Siddhi query is,

$clientIp:$apiContext

That’s it!!!

Congratulations!!! Now You can define custom throttling policies for each client with additional filters. 😇😇😇

Happy Throttling!!! 😁😁😁

[1] https://docs.wso2.com/display/AM260/Adding+New+Throttling+Policies#AddingNewThrottlingPolicies-Addingacustomthrottlingpolicy