API Design

API Throttling | What is it for?

API throttling is an essential process that API developers need to limit how often users can access their API. With API throttling, you can prevent overloads, thus ensuring a smooth API performance for all users, as well as protecting the API from denial-of-service attacks!

Steven Ang Cheong Seng

Updated on Mar 20, 2024 9 min read

APIs (Application Programming Interfaces) are the messengers that bridge together software applications. This allows two separate entities to exchange data or functionalities. However, when an application becomes so popular, APIs and servers have to work a lot harder to provide their services to more users.

💡

Although APIs are provided to users for application implementation, some malicious users may exploit your API, causing it to overload. This spells trouble for you, as other users may soon complain about the very poor performance displayed by your API.

To ensure this never happens, consider using an API tool like Apidog. With Apidog, you can build, test, mock, and document APIs all within a single app. If you're interested, start today by clicking on the button below! 👇 👇 👇

button

API throttling and API rate limiting are two terms often mixed up together. However, they do not share the same meaning. To understand more about the differences between throttling and rate limiting, check out this article first!

What is API Throttling?

API throttling is a mechanism used for controlling the rate ar which applications can access an API. In other words, API throttling limits the number of requests that can be made to an API within a specific period.

API throttling is used to protect the API from being overloaded, ensuring that everyone has a smooth performance when using the API.

How does API Throttling Work?

To implement API throttling, an application will rely on algorithms to manage the flow of incoming requests.

The Token Bucket Algorithm

The Token Bucket Algorithm revolves around two concepts:

Tokens: Tokens the "permits" to access the API
Buckets: Buckets hold a set limited number of tokens

In the Token Bucket Algorithm, there are a few concepts to familiarize yourself with:

Request Allowance: Also called rate limiting, the API defines a rate limit that determines how many tokens are added to the bucket per unit of time (for example: 10 tokens per second).
Request Processing: Whenever a request is received, the API checks the bucket for remaining tokens. If there are tokens available, n number of tokens will be removed, and the request will be processed.
Empty Bucket, Slow Down: If the bucket is found to be empty during request processing, the request is then throttled. This means that the applications will need to wait until enough tokens are added (based on the rate limit) before retrying to make another request.

Burst Capacity

The Token Bucket Model can be extended with a burst capacity; the burst capacity permits the bucket to hold an extra amount of tokens (think of it as a slightly bigger bucket with some tokens pre-filled).

With the burst capacity added, applications can send a burst of requests that exceeds the rate limit for a short period before getting throttled. It is usually implemented for handling large spikes in traffic.

API Throttling Implementation

There are various levels at where API throttling can be implemented:

IP Address Based: Limits requests coming from a specific IP address.
API Key Based: Uses unique API keys to identify and throttle individual applications.
User-Based: Throttling based on user accounts for finer control.

Error Codes and Retries

When an API request is throttled, the API will usually return an error response code (example: 429 Too Many Requests). Well-developed applications have retry logic implemented with backoff mechanisms, forcing the user to wait for a progressively longer time before being throttled.

Advantages of API Throttling

API throttling offers several significant advantages that ensure the smooth and stable operation of an API for both the provider and its users. Here's a breakdown of its key benefits:

1. Performance and Stability:

Prevents Overload: Uncontrolled request volumes can overwhelm the API server, leading to slow response times or even outages. API throttling acts as a safeguard, preventing excessive requests from crashing the system. Users experience consistent performance as the API can handle requests efficiently.

2. Fairness and Resource Management:

Equal Access: API throttling ensures that no single application or user hogs all the API resources. By limiting request rates, it distributes access fairly among all users, preventing a few from degrading the experience for others.

3. Scalability and Efficiency:

Predictable Load: API throttling helps the API handle predictable workloads. By regulating request flow, the API provider can optimize resources and infrastructure for the expected traffic patterns. This allows for smoother scaling when demand increases.

4. Security:

DoS Mitigation: Denial-of-Service (DoS) attacks aim to overwhelm a system with requests, rendering it unavailable to legitimate users. Throttling can mitigate such attacks by identifying and throttling abnormally high request rates from a single source.

5. Improved User Experience:

Consistent Response Times: By preventing overload, throttling ensures users experience consistent response times for their API requests. This translates to a faster and more reliable experience for all users relying on the API.

6. Application Health Monitoring:

Identifying Usage Patterns: Throttling data can be analyzed to understand usage patterns and identify potential issues. For instance, high throttling rates for specific applications might indicate optimization opportunities for those applications.

7. Encourages Responsible Usage:

Promotes Efficiency: Throttling incentivizes developers to design applications that make efficient use of the API. By limiting requests, developers are encouraged to optimize their code and avoid unnecessary calls.

API Throttling Server-side Coding Examples

The Python code sample below demonstrates API throttling using the Token Bucket Method.

class TokenBucket:
  def __init__(self, capacity, refill_rate):
    self.capacity = capacity  # Maximum number of tokens
    self.tokens = capacity  # Current number of tokens
    self.refill_rate = refill_rate  # Tokens added per unit time

  def get_token(self, current_time):
    # Simulate token refill based on time elapsed
    elapsed_time = current_time - self.last_refill_time
    self.tokens = min(self.capacity, self.tokens + (elapsed_time * self.refill_rate))
    self.last_refill_time = current_time

    if self.tokens > 0:
      self.tokens -= 1
      return True
    else:
      return False

# Example usage
bucket = TokenBucket(5, 1)  # 5 tokens, refill 1 per second
request_time = time.time()  # Get current time

if bucket.get_token(request_time):
  # Process request (logic omitted for brevity)
  print("Request allowed!")
else:
  print("Request throttled, retry later!")

Code Explanation:

The TokenBucket class represents the bucket with a capacity and a refill_rate.
The get_token method checks the current time and simulates refilling the bucket based on the elapsed time and refill rate.
If there's a token available (tokens is greater than 0), it removes a token and allows the request.
If the bucket is empty, the request is throttled (rejected).

Apidog - Make Unlimited Requests to APIs

Apidog An integrated platform for API design, debugging, development, mock, and testing

REAL API Design-first Development Platform. Design. Debug. Test. Document. Mock. Build APIs Faster & Together.

apidog

When you are testing out APIs to observe their response and performance, you may endlessly need to make requests. Most API tools limit their users to a certain number of requests they can make per day. However, with Apidog, you can continue to make requests to an API as long as it has not reached the API's rate limit.

button

With Apidog, an all-in-one API development tool, you can build, mock, test, and document APIs. Apidog facilitates all these vital processes of an API lifecycle without having to install or download additional API tools.

Building APIs with Apidog

With Apidog, you can create APIs by yourself. Whether niche or widely applicable in many situations, it is entirely up to you to decide what your API is capable of doing.

Begin by pressing the New API button, as shown in the image above.

Next, you can select many of the API's characteristics. On this page, you can:

Set the HTTP method (GET, POST, PUT, or DELETE)
Set the API URL (or API endpoint) for client-server interaction
Include one/multiple parameters to be passed in the API URL
Provide a description of what functionality the API aims to provide. Here, you can also describe the rate limit you plan to implement on your API.

The more details you can provide to the designing stage, the more descriptive your API documentation will be, as shown in the next section of this article.

To provide some assistance in creating APIs in case this is your first time creating one, you may consider reading these articles.

Once you have finalized all the basic necessities to make a request, you can try to make a request by clicking Send. You should then receive a response on the bottom portion of the Apidog window, as shown in the image above.

The simple and intuitive user interface allows users to easily see the response obtained from the request. It is also important to understand the structure of the response as you need to match the code on both the client and server sides.

Generate Descriptive API Documentation with Apidog

With Apidog, you can quickly create API documentation that includes everything software developers need within just a few clicks.

step by step process sharing api documentation apidog

Arrow 1 - First, press the Share button on the left side of the Apidog app window. You should then be able to see the "Shared Docs" page, which should be empty.

Arrow 2 - Press the + New button under No Data to begin creating your very first Apidog API documentation.

Select and Include Important API Documentation Properties

input api details and select api doc properties apidog

Apidog provides developers with the option of choosing the API documentation characteristics, such as who can view your API documentation and setting a file password, so only chosen individuals or organizations can view it.

open share edit api documentation apidog

Apidog compiles your API project's details into an API documentation that is viewable through a website URL. All you have to do is distribute the URL so that others can view your API documentation!

If more details are required, read this article on how to generate API documentation using Apidog:

Conclusion

API throttling plays a critical role in maintaining a healthy and functional API ecosystem. By regulating the flow of incoming requests, it safeguards the API from overload, ensuring smooth performance and consistent response times for all users. This fairness extends to resource management, preventing a single application from monopolizing resources and degrading the experience for others.

Additionally, throttling strengthens security by mitigating denial-of-service attacks that aim to overwhelm the system. In essence, API throttling creates a win-win scenario for both API providers and users, fostering a reliable and efficient environment for application development and data exchange.

Apidog can be the optimal API tool to ensure that your APIs are running smoothly. ATogether with Apidog's documentation future, you can explicitly describe what the rate limit for your API is, thus preventing users from abusing your API.