The HTTP protocol is a “stateless protocol”, that is, every time the server receives a request from the client, it is a completely new request, and the server does not know the historical request records of the client. The main purpose of Session and Cookie is to make up for the stateless nature of HTTP.

What is Session?

When the client requests the server, the server will open up a “memory space” for this request. The memory space stores the Session object, and the storage structure is ConcurrentHashMap. Session makes up for the stateless nature of HTTP. The server can use Session to store some operation records of the client during the same session.

How to determine whether it is the same session?

Because when the server receives the request for the first time, it opens up a Session space (creates a Session object), and at the same time generates a sessionId, and sends a response to the client requesting to set a Cookie through the Set-Cookie: JSESSIONID=XXX command in the response header.

After the client receives the response, it sets a Cookie information of JSESSIONID=XXX on the local client. The expiration time of this Cookie is the end of the browser session.

Next time when the client sends a request to the same website each time, the request header will carry this Cookie information (including sessionId). Then, by reading the Cookie information in the request header, the server obtains the value named JSESSIONID and gets the sessionId of this request.

Disadvantages of Session

However, the Session mechanism has a disadvantage. If your server does load balancing and stores the Session on server A during the first request. Suppose the traffic to server A surges within a period of time and requests will be forwarded to server B for access. But server B does not store the Session of server A, which will lead to the invalidation of the Session.

What is Cookie?

Photo by Pietro De Grandi on Unsplash

You should have noticed that when introducing Session, Cookie has already been mentioned. Session is implemented based on cookie. Session is stored on the server side, and sessionId will be stored in the Cookie of the client.

Cookies in the HTTP protocol include Web Cookie and browser Cookie. It is a small piece of data sent by the server to the web browser. The Cookie sent by the server to the browser will be stored by the browser and sent to the server together with the next request. Usually, it is used to determine whether two requests come from the same browser, such as when a user remains logged in.

Cookies are mainly used for the following three purposes:

1. Session management

• Cooperate with the server and identify user sessions by storing sessionid.

2. Store user information

• Login status: Remember whether the user is logged in. No need to log in again on the next visit.

• Preference settings: Such as language, theme, etc. Automatically apply on the next visit.

3. Track user behavior

• Browsing history: Record visited pages for easy recommendation and navigation.

• Analyze behavior: Understand user habits for optimization and precision marketing.

Creating Cookies

When receiving an HTTP request from a client, the server can send a response with the Set-Cookie header. Cookies are usually stored by the browser and then the Cookie is sent to the server along with the HTTP header.

Set-Cookie and Cookie headers

The Set-Cookie HTTP response header sends cookies from the server to the user agent. Here is an example of sending a Cookie.

This header tells the client to store cookies.

Now, with each new request to the server, the browser will send all previously stored cookies back to the server using the Cookie header.

There are two types of cookies. One is Session Cookies, and the other is Persistent Cookies. If a cookie does not contain an expiration date, it is regarded as a session cookie. Session cookies are stored in memory and are never written to disk. When the browser is closed, the cookie will be permanently lost thereafter. If a cookie contains an “expiration period”, it is regarded as a persistent cookie. At the specified expiration date, the cookie will be deleted from the disk.

There is also the “Secure and HttpOnly flags of cookies”. Let’s introduce them one by one.

Session Cookies

The example above creates a session cookie. Session cookies have a characteristic that when the client is closed, the cookie will be deleted because it does not specify the Expires or Max-Age directive.

However, web browsers may use session restoration, which will keep most session cookies in a permanent state as if the browser had never been closed.

Permanent Cookies

Permanent cookies do not expire when the client is closed. Instead, they expire after a “specific date (Expires)” or a “specific length of time (Max-Age)”. For example.

Set-Cookie: id=a3fWa; Expires=Sat, 21 Sep 2024 11:28:00 GMT;

Secure and HttpOnly flags of cookies

Secure cookies need to be sent to the server in an encrypted manner through the HTTPS protocol. Even if they are secure, sensitive information should not be stored in cookies because they are inherently insecure and this flag does not provide real protection.

Function of HttpOnly

• The lack of the HttpOnly property in session cookies can lead to attackers being able to obtain users’ cookie information through programs (JS scripts, Applets, etc.), resulting in the leakage of users’ cookie information and increasing the threat of cross-site scripting attacks by attackers.

• HttpOnly is an extension made by Microsoft to cookies. This value specifies whether cookies can be accessed through client scripts.

• If the HttpOnly property is not set to true in cookies, it may lead to cookie theft. Stolen cookies can contain sensitive information identifying site users, such as ASP.NET session IDs or Forms authentication tickets. Attackers can replay stolen cookies to disguise as users or obtain sensitive information and conduct cross-site scripting attacks.

Scope of cookies

The Domain and Path identifiers define the scope of cookies: that is, which URLs cookies should be sent to.

The Domain identifier specifies which hosts can accept cookies. If not specified, the current host (excluding subdomains) is the default. If Domain is specified, subdomains are generally included.

For example, if Domain=mozilla.org is set, cookies are also included in subdomains (such as developer.mozilla.org).

For example, if Path=/test is set, the following addresses will all match:

• /test

• /test/user/

• /test/user/login

Why do we need Token when we already have Session?

In modern web development, although Session can realize user authentication and state management to a certain extent, it also has some limitations.

I. Limitations of Session

Suppose you are running a large online shopping mall. When users log in to your mall, the server will create a Session to record the user’s login status. This Session is like an exclusive card prepared for users at the mall service desk, which records the user’s identity information.

However, when your mall’s business becomes more and more busy and many users are shopping online at the same time, the server needs to save this Session information for each user, which will occupy a large amount of server memory resources. Moreover, if your mall uses multiple servers to share traffic (such as through a load balancer), then a complex mechanism is needed to ensure that when users switch between different servers, their Session information can be correctly transmitted and identified. Otherwise, users may be suddenly logged out or unable to shop normally.

In addition, suppose a user is shopping in your mall using a mobile phone and suddenly has an urgent matter and needs to go out. At this time, if the user uses another device (such as a tablet) to access your mall again outside, since Session is usually bound to a specific device, the user may need to log in again, which will bring inconvenience to the user.

II. Advantages of Token

Now, let’s introduce Token. Token is like a magic pass. After a user logs in successfully, the server will generate a Token containing the user’s identity information and return this Token to the user. The user can save this Token on their own device (such as in the local storage of the browser).

When users browse products, add to the shopping cart or check out in the mall, they only need to carry this Token in each request. After the server receives the request, it can determine the user’s identity and permissions by verifying the validity of the Token, without having to look up and manage complex Session information.

For example, a user logs in to your mall on a mobile phone and obtains a Token. When the user goes out and uses a tablet to access the mall again, only this Token needs to be provided in the browser of the tablet, and the server can immediately identify the user’s identity without the user having to log in again. Moreover, no matter how many users are online at the same time in your mall, the server does not need to save a large amount of Session information for each user. It only needs to verify the Token for each request, greatly reducing the burden on the server.

In addition, Token can be easily integrated with third-party services. For example, if your mall wants to cooperate with an external payment service, only the Token needs to be passed to the payment service. The payment service can determine the user’s identity by verifying the Token without having to establish its own Session management mechanism.

Now let’s introduce Token in detail.

What is a Token?

Photo by Shubham Dhage on Unsplash

1. Access Token

Access Token is a resource credential required when accessing resource interfaces (APIs).

The composition of a token is not fixed. A simple token composition includes:

• uid (user’s unique identity identifier);

• time (timestamp of the current time);

• sign (signature, a hexadecimal string of a certain length compressed by a hash algorithm from the first few digits of the token).

import java.security.MessageDigest;
import java.util.Base64;
import java.util.Date;

public class TokenGenerator {

    public static String generateToken(int uid) {
        long time = new Date().getTime();
        String tokenContent = uid + “-” + time;
        // Generate a sign for the token content.
        String sign = generateSign(tokenContent);
        return uid + “-” + time + “-” + sign;
    }

    // Generate a sign for the given content using SHA-256 hash algorithm.
    private static String generateSign(String content) {
        try {
            MessageDigest digest = MessageDigest.getInstance(”SHA-256”);
            byte[] hash = digest.digest(content.getBytes());
            StringBuilder hexString = new StringBuilder();
            for (byte b : hash) {
                // Convert each byte to a two-digit hexadecimal string.
                String hex = Integer.toHexString(0xff & b);
                if (hex.length() == 1) hexString.append(’0’);
                hexString.append(hex);
            }
            return hexString.substring(0, 8);
        } catch (Exception e) {
            return null;
        }
    }

    public static void main(String[] args) {
        int uid = 1234;
        String token = generateToken(uid);
        System.out.println(”Generated token: “ + token);
    }
}

Output:

Generated token: 1234-1729530432169-3638dd14

It has the following characteristics:

• The server is stateless and has good scalability.

• Supports mobile devices.

• Sufficiently secure.

• Supports cross-program invocations.

The token authentication process:

From the above process, we can know that the token needs to be carried in every subsequent request, so the token needs to be placed in the HTTP Header. User authentication based on token is a server-side stateless authentication method, and the server does not need to store token data. The calculation time for parsing the token is exchanged for the storage space of the session, thereby reducing the pressure on the server and reducing frequent database query operations.

In addition, the token is completely managed by the application itself, so it can avoid the same-origin policy.

2.Refresh Token

Photo by Genessa Panainte on Unsplash

Refresh token is another kind of token that is dedicated to refreshing the access token. If there is no refresh token, the access token can also be refreshed, but every time it is refreshed, the user needs to enter the login username and password, which will be very troublesome. With the refresh token, this trouble can be reduced. The client directly uses the refresh token to update the access token without the user performing additional operations.

The validity period of the Access Token is usually relatively short. When the Access Token becomes invalid due to expiration, a new token can be obtained by using the Refresh Token. If the Refresh Token also expires, the user can only log in again.

In addition, the Refresh Token and expiration time are stored in the server’s database and are only verified when applying for a new Access Token. It will not affect the response time of business interfaces and does not need to be kept in memory all the time like Session to handle a large number of requests.

JSON Web Token

Token is a broader concept, while JSON Web Token (JWT) is a token with specific structure and characteristics. JWT has advantages in some scenarios, such as when self-contained authentication information is needed, cross-platform use, and when there are high requirements for scalability. We will provide a specific introduction to JWT in later articles.

Okay, that’s all for this time, see you next time! 🤭

Finally, if the article was helpful, please clap 👏and follow, thank you! ╰(*°▽°*)╯

I’m Dylan, looking forward to progressing with you. ❤️

Digital signature and information encryption are technologies that are often used in both front-end and back-end development. Application scenarios include:

• Users’ passwords cannot be directly stored in plaintext. Encrypted ciphertext should be stored.

• Sensitive data such as users’ bank card numbers and ID numbers need to be encrypted for transmission.

• For some important interfaces, such as payment, the client needs to generate a signature for the request, and the server needs to verify the signature to check whether this request is safe.
  -…

Different application scenarios require different signature encryption algorithms to achieve business goals. Here, I will briefly introduce several common signature encryption algorithms and their applications in some typical scenarios.

Common encryption algorithms

In general, encryption algorithms can be divided into irreversible encryption and reversible encryption. Reversible encryption can be further divided into symmetric encryption and asymmetric encryption.

Irreversible algorithms

The encryption of an irreversible encryption algorithm is irreversible, that is, the ciphertext cannot be restored to the original text.

A hash algorithm is an irreversible algorithm. In a hash algorithm, plaintext generates a hash value through a hash algorithm. The hash value is data with a fixed length and is independent of the length of the plaintext.

There are many specific implementations of hash algorithms. Common ones include MD5, SHA1, SHA-224, SHA-256, and so on.

Hash algorithms are often used in scenarios such as digital signatures, message authentication, and password storage.

Hash algorithms do not require a key. Of course, there are also some irreversible algorithms that require a key, such as the HMAC algorithm.

MD5

MD5, fully known as “Message-Digest Algorithm 5”. It can generate a fixed-length hash value by applying a hash algorithm to data of any length. The output length of the MD5 algorithm is 128 bits, usually represented by 32 hexadecimal numbers.

Let’s take a look at the Java code implementation of the MD5 algorithm:

import java.security.MessageDigest;
import java.util.Formatter;

public class MD5 {
    private static final String MD5_ALGORITHM = “MD5”;
    public static String encrypt(String data) throws Exception {
        // Obtain an instance of the MD5 algorithm.
        MessageDigest messageDigest = MessageDigest.getInstance(MD5_ALGORITHM);
        // Calculate the hash value.
        byte[] digest = messageDigest.digest(data.getBytes());
        Formatter formatter = new Formatter();
        // Pad leading zeros and format.
        for (byte b : digest) {
            formatter.format(”%02x”, b);
        }
        return formatter.toString();
    }

    public static void main(String[] args) throws Exception {
        String data = “Hello Dylan”;
        String encryptedData = encrypt(data);
        System.out.println(”Encrypted data: “ + encryptedData);
    }
}

Output:

Encrypted data: ac1fbf0692d337780a0fbffb2798ac96

MD5 has some advantages, such as fast calculation speed, fixed output length, and wide application.

However, as an encryption algorithm, it has a fatal drawback, that is, it is “unsafe”.

The MD5 algorithm has been broken. Moreover, the output length of the MD5 algorithm is limited. Attackers can find hash values identical to the original data through methods such as brute-force cracking or rainbow table attacks, thereby cracking the data.

Although the security can be increased by adding salt, that is, by adding some unfixed strings to the original text, it is completely possible to replace it with a more secure SHA series algorithm.

SHA-256

SHA (Secure Hash Algorithm) is a set of cryptographic hash functions used to map data of any length to a fixed-length hash value. The SHA series of algorithms was designed by the National Security Agency (NSA) in 1993. Currently, there are three versions: SHA-1, SHA-2, and SHA-3.

Among them, the SHA-1 series has defects and is no longer recommended for use.

The SHA-2 algorithm includes four hash functions: SHA-224, SHA-256, SHA-384, and SHA-512, which map data of any length to hash values of 224 bits, 256 bits, 384 bits, and 512 bits respectively.

Let’s take a look at the Java code implementation of the most commonly used SHA-256:

import java.security.MessageDigest;

public class SHA256 {
    private static final String SHA_256_ALGORITHM = “SHA-256”;
    public static String encrypt(String data) throws Exception {
        // Obtain an instance of the SHA-256 algorithm.
        MessageDigest messageDigest = MessageDigest.getInstance(SHA_256_ALGORITHM);
        // Compute the hash value.
        byte[] digest = messageDigest.digest(data.getBytes());
        StringBuilder stringBuilder = new StringBuilder();
        // Convert byte array to a hexadecimal string.
        for (byte b : digest) {
            stringBuilder.append(Integer.toHexString((b & 0xFF) | 0x100), 1, 3);
        }
        return stringBuilder.toString();
    }

    public static void main(String[] args) throws Exception {
        String data = “Hello Dylan”;
        String encryptedData = encrypt(data);
        System.out.println(”Encrypted data: “ + encryptedData);
    }
}

Output:

Encrypted data: 2ff61b5259b01a8ef4e6dae816d3f26b4b9cbc3957a6a9d0aceb49d11d5d950e

The reason why the SHA-2 algorithm is stronger than MD5 mainly lies in two aspects:

• Longer hash value length: For example, the hash value length of the SHA-256 algorithm is 256 bits, while that of the MD5 algorithm is 128 bits. This increases the difficulty for attackers to perform brute-force cracking or rainbow table attacks.

• Stronger collision resistance: The SHA algorithm uses a more complex operation process and more rounds, making it more difficult for attackers to find collisions through precomputation or coincidence.

Of course, SHA-2 is not absolutely safe either. Hash algorithms all have the risk of being brute-force cracked or attacked by rainbow tables. Therefore, in practical applications, adding salt is still essential.

Symmetric encryption algorithm

In symmetric encryption algorithms, the same key is used for encryption and decryption.

The encryption and decryption processes use the same key, so the security of the key is of utmost importance. If the key is leaked, attackers can easily crack the encrypted data.

Common symmetric encryption algorithms include DES, 3DES, AES, etc. Among them, the AES algorithm is currently one of the most widely used symmetric encryption algorithms and has relatively high security and encryption efficiency.

DES

DES (Data Encryption Standard) is a symmetric encryption algorithm developed by IBM in 1975 and is one of the earliest symmetric encryption algorithms widely used.

The DES algorithm uses a 56-bit key to encrypt data. In the encryption process, operations such as permutation, substitution, and exclusive OR are used, which has high security.

Let’s take a look at the Java code implementation of the DES algorithm:

import javax.crypto.Cipher;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.DESKeySpec;
import javax.crypto.spec.SecretKeySpec;
import java.security.Key;
import java.security.SecureRandom;
import java.util.Base64;

public class DES {
    private static final String DES_ALGORITHM = “DES”;

    /**
     * Encrypt data using DES algorithm.
     *
     * @param data The data to be encrypted.
     * @param key  The key for encryption, must be 8 characters long.
     * @return The encrypted data encoded in Base64.
     */
    public static String encrypt(String data, String key) throws Exception {
        // Generate key specification based on the key.
        KeySpec keySpec = new DESKeySpec(key.getBytes());
        // Generate secret key factory based on the algorithm.
        SecretKeyFactory secretKeyFactory = SecretKeyFactory.getInstance(DES_ALGORITHM);
        // Generate secret key based on the key factory and key specification.
        SecretKey secretKey = secretKeyFactory.generateSecret(keySpec);

        // Get cipher instance for the algorithm.
        Cipher cipher = Cipher.getInstance(DES_ALGORITHM);
        // Initialize the cipher for encryption mode and the secret key.
        cipher.init(Cipher.ENCRYPT_MODE, secretKey);
        // Encrypt the data.
        byte[] encryptedData = cipher.doFinal(data.getBytes());
        // Encode the encrypted data using Base64.
        return Base64.getEncoder().encodeToString(encryptedData);
    }

    /**
     * Decrypt data encrypted with DES algorithm.
     *
     * @param encryptedData The encrypted data encoded in Base64.
     * @param key           The key for decryption, must be 8 characters long.
     * @return The decrypted data.
     */
    public static String decrypt(String encryptedData, String key) throws Exception {
        // Generate key specification based on the key.
        KeySpec keySpec = new DESKeySpec(key.getBytes());
        // Generate secret key factory based on the algorithm.
        SecretKeyFactory secretKeyFactory = SecretKeyFactory.getInstance(DES_ALGORITHM);
        // Generate secret key based on the key factory and key specification.
        SecretKey secretKey = secretKeyFactory.generateSecret(keySpec);

        // Decode the encrypted data using Base64.
        byte[] decodedData = Base64.getDecoder().decode(encryptedData);
        // Get cipher instance for the algorithm.
        Cipher cipher = Cipher.getInstance(DES_ALGORITHM);
        // Initialize the cipher for decryption mode and the secret key.
        cipher.init(Cipher.DECRYPT_MODE, secretKey);
        // Decrypt the data.
        byte[] decryptedData = cipher.doFinal(decodedData);
        // Convert the decrypted data to a string.
        return new String(decryptedData);
    }

    public static void main(String[] args) throws Exception {
        String data = “Hello Dylan”;
        String key = “12345678”; 

        String encryptedData = encrypt(data, key);
        System.out.println(”Encrypted data: “ + encryptedData);

        String decryptedData = decrypt(encryptedData, key);
        System.out.println(”Decrypted data: “ + decryptedData);
    }
}

Output:

Encrypted data: 3D12G4wkjn4CESZ5YgCeKA==
Decrypted data: Hello Dylan

The DES algorithm is relatively fast, but it is not the optimal choice in terms of security. Because the key length of the DES algorithm is relatively short, there is a relatively high risk of being brute-force cracked and differentially attacked.

Generally, some more secure symmetric encryption algorithms such as 3DES and AES are recommended.

AES

AES (Advanced Encryption Standard), that is, the advanced encryption standard, is a symmetric encryption algorithm and is widely used in the field of data encryption and protection. The key length used by the AES algorithm is 128 bits, 192 bits, or 256 bits, which is longer than the key length of the DES algorithm and has higher security.

Let’s take a look at the Java code implementation of the AES algorithm:

import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.util.Base64;

public class AES {
    private static final String AES_ALGORITHM = “AES”;
    // AES encryption mode is CBC and padding method is PKCS5Padding.
    private static final String AES_TRANSFORMATION = “AES/CBC/PKCS5Padding”;
    // AES key is 16 characters.
    private static final String AES_KEY = “1234567890123456”;
    // AES initialization vector is 16 characters.
    private static final String AES_IV = “abcdefghijklmnop”;

    /**
     * Encrypt data using AES algorithm.
     *
     * @param data The data to be encrypted.
     * @return The encrypted data encoded in Base64.
     */
    public static String encrypt(String data) throws Exception {
        // Convert AES key to SecretKeySpec object.
        SecretKeySpec secretKeySpec = new SecretKeySpec(AES_KEY.getBytes(), AES_ALGORITHM);
        // Convert AES initialization vector to IvParameterSpec object.
        IvParameterSpec ivParameterSpec = new IvParameterSpec(AES_IV.getBytes());
        // Get cipher instance for the AES algorithm.
        Cipher cipher = Cipher.getInstance(AES_TRANSFORMATION);
        // Initialize the cipher for encryption mode, key and initialization vector.
        cipher.init(Cipher.ENCRYPT_MODE, secretKeySpec, ivParameterSpec);
        // Encrypt the data.
        byte[] encryptedData = cipher.doFinal(data.getBytes(java.nio.charset.StandardCharsets.UTF_8));
        // Encode the encrypted data using Base64.
        return Base64.getEncoder().encodeToString(encryptedData);
    }

    /**
     * Decrypt data encrypted with AES algorithm.
     *
     * @param encryptedData The encrypted data encoded in Base64.
     * @return The decrypted data.
     */
    public static String decrypt(String encryptedData) throws Exception {
        // Convert AES key to SecretKeySpec object.
        SecretKeySpec secretKeySpec = new SecretKeySpec(AES_KEY.getBytes(), AES_ALGORITHM);
        // Convert AES initialization vector to IvParameterSpec object.
        IvParameterSpec ivParameterSpec = new IvParameterSpec(AES_IV.getBytes());
        // Get cipher instance for the AES algorithm.
        Cipher cipher = Cipher.getInstance(AES_TRANSFORMATION);
        // Initialize the cipher for decryption mode, key and initialization vector.
        cipher.init(Cipher.DECRYPT_MODE, secretKeySpec, ivParameterSpec);
        // Decode the encrypted data using Base64.
        byte[] decodedData = Base64.getDecoder().decode(encryptedData);
        // Decrypt the data.
        byte[] decryptedData = cipher.doFinal(decodedData);
        // Return the decrypted data.
        return new String(decryptedData, java.nio.charset.StandardCharsets.UTF_8);
    }

    public static void main(String[] args) throws Exception {
        String data = “Hello World”;

        String encryptedData = encrypt(data);
        System.out.println(”Encrypted data: “ + encryptedData);

        String decryptedData = decrypt(encryptedData);
        System.out.println(”Decrypted data: “ + decryptedData);
    }
}

Output:

Encrypted data: H+v+5je7lD1T5xZZC0kdqQ==
Decrypted data: Hello Dylan

The AES algorithm uses a longer key length, a larger key space, and higher security, and can effectively resist brute-force cracking attacks.

Of course, because the key length is longer, more storage is required.

For symmetric encryption algorithms, the biggest pain point is the difficulty in key management. In contrast, asymmetric encryption does not have this concern.

Asymmetric encryption algorithm

Asymmetric encryption algorithms require two keys. These two keys are different from each other but are mutually matched. One is called the “public key” and the other is called the “private key”.

If one of them is used for encryption, then the other is used for decryption. For example, if public key encryption is used, then private key decryption is required.

RSA

The RSA algorithm is currently the most widely used asymmetric encryption algorithm. It was invented by Ron Rivest, Adi Shamir, and Leonard Adleman in 1978. The name comes from the first letters of the surnames of the three people.

Let’s take a look at the Java implementation of the RSA algorithm:

import javax.crypto.Cipher;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.NoSuchAlgorithmException;
import java.security.PublicKey;
import java.security.PrivateKey;
import java.util.Base64;

public class RSA {
    private static final String RSA_ALGORITHM = “RSA”;

    /**
     * Generate an RSA key pair.
     *
     * @return The RSA key pair.
     */
    public static KeyPair generateKeyPair() throws NoSuchAlgorithmException {
        KeyPairGenerator keyPairGenerator = KeyPairGenerator.getInstance(RSA_ALGORITHM);
        keyPairGenerator.initialize(2048); // Key size is 2048 bits.
        return keyPairGenerator.generateKeyPair();
    }

    /**
     * Encrypt data using the public key.
     *
     * @param data      The data to be encrypted.
     * @param publicKey The public key.
     * @return The encrypted data.
     */
    public static String encrypt(String data, PublicKey publicKey) throws Exception {
        Cipher cipher = Cipher.getInstance(RSA_ALGORITHM);
        cipher.init(Cipher.ENCRYPT_MODE, publicKey);
        byte[] encryptedData = cipher.doFinal(data.getBytes(java.nio.charset.StandardCharsets.UTF_8));
        return Base64.getEncoder().encodeToString(encryptedData);
    }

    /**
     * Decrypt data using the private key.
     *
     * @param encryptedData The encrypted data.
     * @param privateKey    The private key.
     * @return The decrypted data.
     */
    public static String decrypt(String encryptedData, PrivateKey privateKey) throws Exception {
        byte[] decodedData = Base64.getDecoder().decode(encryptedData);
        Cipher cipher = Cipher.getInstance(RSA_ALGORITHM);
        cipher.init(Cipher.DECRYPT_MODE, privateKey);
        byte[] decryptedData = cipher.doFinal(decodedData);
        return new String(decryptedData, java.nio.charset.StandardCharsets.UTF_8);
    }

    public static void main(String[] args) throws Exception {
        KeyPair keyPair = generateKeyPair();
        PublicKey publicKey = keyPair.getPublic();
        PrivateKey privateKey = keyPair.getPrivate();

        String data = “Hello World”;

        String encryptedData = encrypt(data, publicKey);
        System.out.println(”Encrypted data: “ + encryptedData);

        String decryptedData = decrypt(encryptedData, privateKey);
        System.out.println(”Decrypted data: “ + decryptedData);
    }
}

Output:

Encrypted data: kLKl+j90w34MwDu+k933znMxHnnajJeJCqIOfaYbiYT2IJrTmTisIGWvCJvtyprgMidObGxzhWAWs0lQKsq+dHZ70XG8m3ICMy4CxCYVdqOymEiSjD4HfEhKAZc58TDQ0J0qPdOpw+fqkmxsSG2sQmI87ffLx6ChbbuZlVrwJmm2KwuqWEwxuddilntsnZWLVn4uEXtHzHhrXm8RqOodVdLUemUxcV7N9x9v3DJyei7zZbTu2OEbija0bIHukQ9/yqZBbGSANfa8LVsQ+rsiKWd7+IGbPmsXmds8G6hgkkP1K2aHecOYgc8bo2vrY2httxadZzwqkknqn47eU+3cNg==
Decrypted data: Hello Dylan

The advantages of the RSA algorithm are high security. The public key can be made public, and the private key must be kept secret, ensuring data security. It can be used in multiple application scenarios such as digital signatures and key negotiation.

The disadvantages are that the encryption and decryption speed is relatively slow. The longer the key length, the longer the encryption and decryption time. If the key length is too short, it is easy to be brute-force cracked. If the key length is too long, it will increase the computational cost and storage space overhead.

Summary

In this issue, we have simply reviewed the five most commonly used encryption algorithms.

In fact, when it comes to the application of encryption and decryption algorithms, there is one thing that can be said to be applied to the extreme. What is it?

— HTTPS.

In conclusion, HTTPS uses symmetric encryption algorithms, asymmetric encryption algorithms, and hash algorithms to ensure the security and integrity of data, thereby ensuring the identity and data security of both communicating parties.

We will conduct a detailed exploration in a later article~😄

Okay, that’s all for this time, see you next time! 🤭

Finally, if the article was helpful, please clap 👏and follow, thank you! ╰(*°▽°*)╯

I’m Dylan, looking forward to progressing with you. ❤️

Recently, a friend encountered a very interesting question in an interview. I believe that most friends may not have encountered it.

The interviewer asked: “Why do many websites require password reset when forgetting the password instead of directly telling users the original password?” (⊙ˍ⊙)？

It’s a very interesting question, right? Many websites have the function of “Forgot password?”, but why when we click on forgot password and go through a bunch of verifications, the website will let us reset the password instead of directly telling us the original password?

So, let’s talk about this question today.

Preventing information leakage

For most people, in order to prevent forgetting too many passwords, many times everyone will use a unified password! That is to say, your multiple accounts may all have the same password. Therefore, once the password is leaked, it may affect multiple of your accounts, even bank card accounts.

Therefore, for websites (especially some large websites), protecting user data security is an extremely important matter.

So, the data you save on the website may not be so safe.

For most people, the most relevant experience (seen online) should be scam calls. They can even tell you all your personal information very clearly. So where did this information come from?

Mostly due to data leakage, all your personal information is known to others.

So how do general websites prevent data leakage?

The usual approach is encryption. And this encryption may be performed multiple times at different stages. For example, common ones are: SHA256, salting, md5, RSA, etc..

This seems to be very safe, but there is still a problem. Developers know how to decrypt them.

Or some friends will think that the correct password still exists in the database? Once there is information leakage, won’t there still be a problem of password leakage?

Yes, so to solve this problem, in fact, the website itself doesn’t know what your password is.

The website itself doesn’t know what your password is

For websites (or other applications), they should not store your original password. Instead, they save your encrypted code through a series of operations. And this encryption is already performed when transmitted from the front end to the server, and it is an irreversible encryption operation, such as: MD5 + salting.

In general, we encrypt and save passwords through hash algorithms.

Hash algorithms, also known as hash functions or digest algorithms, are used to generate a fixed-length unique identifier for data of any length, also known as a hash value, or message digest (referred to as hash value later).

Currently, a more commonly used method is to encrypt passwords through MD5 + Salt. In cryptography, salt refers to inserting a specific string at any fixed position in the password so that the result after hashing does not match the hashing result using the original password. This process is called “salting”.

However, this method is no longer recommended because the MD5 algorithm has low security and poor collision resistance. You can use a more secure encrypted hash algorithm + Salt (salt) (such as SHA2, SHA3, SM3, with higher security and stronger collision resistance) or directly use slow hashing (such as Bcrypt).

Suppose we use this method of SHA-256 + Salt here.

Here is a simple sample code:

public class HashPasswordExample {
    public static void main(String[] args) {
        // Password to be hashed
        String password = “123456”;
        // Salt for hashing
        String salt = “salt-test”;
        try {
            // Create a SHA-256 digest object
            MessageDigest messageDigest = MessageDigest.getInstance(”SHA-256”);
            // Update the digest with the combined password and salt
            messageDigest.update((password + salt).getBytes());
            // Compute the hash value
            byte[] result = messageDigest.digest();
            // Convert the hash value to a hexadecimal string
            HexBinaryAdapter hexBinaryAdapter = new HexBinaryAdapter();
            String hexString = hexBinaryAdapter.marshal(result);
            // Print the original password
            System.out.println(”Original String: “ + password);
            // Print the SHA-256 hash in lowercase
            System.out.println(”SHA-256 Hash: “ + hexString.toLowerCase());
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Output:

Original String: 123456
SHA-256 Hash: 3d5ecd90301b307c267f1c09e8a4b2433c2258fcb7441df91a32e245ea86bf96

In this example, what the server saves is the data after salting and hashing the password “123456”, that is: “3d5ecd90301b307c267f1c09e8a4b2433c2258fcb7441df91a32e245ea86bf96”.

When you enter your password to log in, the server will first take out the salt corresponding to your password, and then perform the process of obtaining the hash value again. If the finally calculated hash value is the same as the hash value saved in the database, it means that the password is correct. Otherwise, the password is incorrect.

The hash algorithm is irreversible. You cannot obtain the original value from the hashed value. In this way, the server does not know what your original password is, and naturally it cannot tell you what your original password is.

Why do many websites require that the changed password cannot be the same as the original password?

Then some friends have doubts again. Why do many websites require that the changed password cannot be the same as the original password? How is this implemented?

This process is actually the same as verifying the correctness of the password. Just calculate the hash value and compare it!

Okay, that’s all for this time, see you next time! 🤭

Finally, if the article was helpful, please clap 👏and follow, thank you! ╰(*°▽°*)╯

I’m Dylan, looking forward to progressing with you. ❤️

Let me give you an example that everyone can understand.😯

Grace is Tom’s older sister.

Lately, Tom has been obsessed with cartoons, while Grace hopes that he watches more documentaries, so she often looks for high-quality documentaries for Tom to watch. Initially, Grace would ask Tom when he was free, give him the documentary, and personally supervise him until he finished watching before leaving. Over time, both of them felt inconvenient.

After that, the way changed: Grace told Tom, “All the documentaries on the video shelf need to be watched.” Since then, whenever Grace finds a good documentary, she will put it on the video shelf, and Tom will take it down and watch it when he sees a documentary on the shelf.

The video shelf is equivalent to a message queue, and Grace is the producer, and Tom is the consumer. The benefits of this are:

1. Grace doesn’t need to ask Tom when he is free or hand him the documentary herself. She just needs to put it on the video shelf, which gives both parties more freedom of time.

2. Grace believes in Tom’s self-discipline and viewing ability, and doesn’t need to supervise his viewing process. She just needs to place the documentary, which saves a lot of time.

3. When other friends who like to watch documentaries join, Grace still only needs to put the documentary on the video shelf, and both Tom and they can pick it up from the video shelf assuming that when multiple people pick up the same documentary at the same time, they can all get a copy.

4. The documentary is placed on the video shelf, and Tom can watch it at his own pace. It doesn’t matter if he finishes it early or late. Compared to Grace’s delivery and supervision method, Tom will feel less stressed.

Here are the four benefits of message queues:👍

1.Decoupling

Each member (Grace and Tom) is not affected by other members and can be more independent. They only connect through a simple container (the video shelf). Grace may not even know who takes the documentary from the video shelf, and Tom may not know who puts the documentary on the video shelf. In their eyes, there is only the video shelf, not the other person. There is no doubt that dealing with a simple container is a hundred times easier than dealing with a complicated person. Grace and Tom can freely pursue their own preferences.

2.Speeding up

By choosing to “place the documentary on the video shelf and not ask about anything else”, Grace saves a lot of time. Although Grace is very busy, it only takes a few minutes to place the documentary on the video shelf.

3.Broadcasting

Grace only needs to make one effort to allow multiple friends to see the documentary, which saves her a lot of time and reduces the cost of new friends joining.

4.Traffic peak shaving

If Tom watches the documentary slowly, the method of Grace supervising the viewing will bring pressure to Tom and also make Grace impatient. Since Grace’s frequency of providing documentaries is unstable, if she provides five consecutive ones in the next two days and only one in the next three months, then Tom only needs to take and watch the five documentaries from the video shelf within three months, without watching them immediately, which will reduce the pressure a lot.

Of course, using message queues comes at a cost:😔

1.Introduced complexity

There is no doubt that the “video cabinet” is an extra thing that needs to be placed and needs to be protected from theft.

2.Temporary inconsistency

If Mom asks Grace, “What documentaries has Tom watched recently?” In the old way, Grace can tell Mom the answer with confidence because she personally supervised Tom watching the documentary. But in the new way, Grace will think to herself, “Tom should finish watching it soon…” There is a period of time where “Mom thinks Tom has watched a certain documentary, but Tom hasn’t actually watched it yet.” Of course, Tom’s final viewing status will be consistent with Mom’s perception, which is what is meant by “eventual consistency.”

So, under what conditions should message queues be used? 🤔

1.The producer does not need feedback from the consumer.

Before the introduction of a message queue, the return value of the interface should be empty, which makes it possible for the lower-level actions to not yet be completed, but the upper-level thinks that the actions have been completed and continues to move forward — that is, so-called asynchronous.
After Grace puts the documentary, she doesn’t ask whether Tom has watched it or not. She assumes that he has watched it, otherwise she would have to use the original method to supervise until he finishes watching it.

2.Allow temporary inconsistencies.

Mom might find that “sometimes it is said that Tom has watched a certain documentary, but in fact, he has not watched it yet.” As long as Mom is satisfied with “anyway, he watched it in the end”, asynchronous processing will be fine. If Mom cannot tolerate this situation and gets angry at Grace, Grace will not dare to use the video cabinet method.

3.Only use when it is effective and beneficial.

That said, the benefits of decoupling, speeding up, broadcasting, and peak clipping outweigh the cost of placing and monitoring the video cabinet. Otherwise, if you blindly copy, “I heard Jerry’s family bought a video cabinet, let’s get one too,” and you buy it and it doesn’t come in handy, you’ll only complicate the steps, and you might as well just give the documentary to the other side. This is not right. That is, don’t over-design.

Okay, that’s all for this time, see you next time! 🤭

Finally, if the article was helpful, please clap 👏and follow, thank you! ╰(*°▽°*)╯

I’m Dylan, looking forward to progressing with you. ❤️

Rate limiting is something every backend engineer eventually runs into. Product launches, new releases, scrapers, buggy callers. Any of them can push a system to its limits. This article skips theory for its own sake. Four algorithms are explained through a single boba shop, and the second half drops straight into Java code you can compile and run, plus the architecture diagrams.

Before writing it, I set myself four goals.

• All four algorithms explained through the same boba shop, so nothing gets introduced twice.

• Three implementations you can compile and run, one each for Jedis, Redisson, and the gateway.

• Ten Mermaid diagrams and five boba shop illustrations, so both the algorithms and the scenes have visuals.

• Production pitfalls and selection advice you can actually decide with.

Conventions. The code targets Spring Boot 2.7.x, Redisson 3.23.x, Jedis 4.3.x, and Spring Cloud Gateway 2021.0.x. Everything compiles on Java 8 and newer, and the important lines have comments.

Chapter 1: Why you need rate limiting, starting with an API overload

A popular open source project ships a new release, and everyone upgrades within hours. CI retries, issue lookups, and status polling all hit the GitHub API at once, and traffic multiplies in a short window. Endpoints start timing out, third-party integrations fail, and the alert channel fills up. Any team that runs a public API recognizes this pattern. Stripe has an engineering blog post about rate limiters. GitHub’s API docs publish the hourly quota. AWS API Gateway has token bucket throttling built in. Google’s SRE workbook devotes a chapter to client-side throttling so servers do not get overwhelmed.

I have read this postmortem too many times. Afterward, everyone says the same thing. If only we had rate limiting.

Rate limiting is not a config flag you flip and forget. The algorithm, the code, and the architecture all matter, and production will remind you if any layer is wrong.

Rate limiting is the last line of defense for a system. It does not make the system faster. It keeps overloaded requests out when pressure exceeds capacity, so the main services survive. Everything below follows from that.

1.1 How that traffic wave takes a system down

A typical API overload plays out like this.

1. Traffic exceeds the API’s capacity, and the business thread pool starts queueing.

2. Slow requests fill the database connection pool, and new requests cannot get a connection.

3. Clients retry after timeouts, and the retries pile onto the same endpoints.

4. Dependent services time out one after another, and the outage spreads from the primary API to everything around it.

5. Scaling out does not help in time. New instances get flooded too.

These incidents share a pattern. The system collapses after traffic exceeds capacity, and a code bug is usually just the trigger. Response times go from 20ms to 2s. Slow requests fill the thread pool, health checks start failing, the registry evicts instances, and traffic moves to the survivors, making things worse in a loop.

Each stage amplifies the one before it. Rate limiting stops the extra traffic at stage one.

1.2 The line between local and distributed rate limiting

Local rate limiting keeps its state inside a single JVM. Guava’s RateLimiter, a Semaphore, or an AtomicLong counter all count. With one instance it is simple, fast, and has no network cost. That breaks the moment you run several instances. Three instances each allowing 100 requests means 300 total, so the limit is meaningless.

Distributed rate limiting puts the state somewhere every instance shares, usually Redis. Each instance runs the same Lua script against the same key, so no matter how many machines are in the cluster, one threshold controls the total.

The rule of thumb is simple. Where does the state live? In JVM memory, it is local rate limiting. In shared storage like Redis, it is distributed. Neither is better, they just fit different situations.

Two conditions force you to go distributed.

• The service runs multiple instances, and the limit has to apply globally.

• The quota is counted per user, account, or device, which a single machine cannot tally.

Public API quotas, anti-abuse, and global risk control fall into the second group. Single-machine protection, local fallbacks, and local debugging can stay with local rate limiting.

1.3 What rate limiting actually protects

Everyone talks about smoothing out peaks. On a real system, rate limiting protects three things.

• Downstream capacity. Databases, third-party APIs, and message queues all have throughput ceilings. Leaky buckets and token buckets keep the input rate inside what they can handle.

• Service SLA. Under heavy traffic the main endpoints still respond within the limit, so timeout and error rates stay controlled.

• Failure boundaries. Overloaded requests stay outside the system instead of cascading into other services.

Those three map to two places. The traffic entry point caps the total, and resource-facing layers such as databases absorb slow, sustained pressure.

The first protects resources, the second protects the experience, and the third protects everything around it.

Chapter 2: Four classic rate limiting algorithms, explained through everyday scenes

2.1 Meet the boba shop

Four algorithms explained separately tend to drift apart, so I put them all in one shop. The shop is called Boba Planet, on the ground floor of an office building, with a pink sign and white lettering that lights up around noon. The queue runs from the counter to the street corner. The owner hears nothing but order calls all day, and by the time the pearls are cooked there is no time to scoop them. He sets a rule, and when it stops working, he replaces it. The four algorithms are the four rules he went through, each one fixing a weakness of the last while introducing a new problem.

The order of the rules is the order the algorithms evolved. First you solve whether there is a limit at all, then how accurate it is, then how smooth, then whether it can handle bursts.

2.2 Fixed window counter

2.2.1 The shop’s version

The first rule is the simplest. One hundred orders per hour. The barista clicks a counter with every order, and at 100 the shop puts up the full sign. The counter resets on the hour. A customer at 9:58 may be turned away, while someone at 10:00 on the dot can order immediately.

2.2.2 The mechanics

A fixed window implementation ties the count to an expiration. One Redis key per window. Each request runs INCR, the first write sets a 60 second expiry, and once the count passes the threshold, requests are rejected. When the window ends, the key disappears and a new window starts. The whole thing is O(1) and uses one counter’s worth of memory.

Only two Redis commands are involved, INCR and EXPIRE. When the first INCR returns 1, set EXPIRE 60. The count keeps growing inside the window, and the key disappears when the window ends. In Lua the two steps can be merged so the expiry setup does not race with the first request.

2.2.3 The boundary spike

The problem is at the window boundary. The 100th order lands at 9:59:59, the counter resets at 10:00:00, and another 100 orders pass at 10:00:01. Two hundred orders in two seconds, twice the limit. The algorithm only knows windows, not time, so a burst that straddles the boundary makes the limit pointless.

A fixed window only counts the current window and clears at the boundary. The 9:00 window and the 10:00 window each allow 100 orders, so the two seconds around the boundary have no window of their own. That is the boundary spike. A sliding window treats adjacent windows as one continuous span and closes the gap.

2.2.4 Trade-offs and when to use it

The upsides are simplicity, memory usage, and speed. The downside is that boundary traffic can double. It is good enough for SMS codes, login attempt limits, and basic abuse protection where a small overshoot is acceptable. For a public API that cares about the total, do not rely on it alone.

If the requirement is simply 100 orders per window with immediate rejection after that, a fixed window still works, as long as you accept the brief overshoot at the boundary.

2.3 Sliding window counter

2.3.1 The shop’s version

After seeing the boundary problem, the owner changes the rule. At most 100 orders in any continuous 60 minutes, not per clock hour. The barista splits the hour into six 10-minute slots, marks a tally in the matching slot for every order, and at any moment counts the last six slots. The window slides forward with the current time, so 9:59 and 10:01 fall inside the same window.

2.3.2 The mechanics

There are two common implementations. The first uses a sorted set. Each request’s timestamp becomes a member, ZREMRANGEBYSCORE removes everything outside the window, and ZCARD counts what is left. The second splits time into a fixed number of small counter keys, increments the slot each request lands in, and sums the last N slots when checking. Finer slots get closer to true sliding, at a higher memory and CPU cost.

The sorted set approach adds a member for every request, so the set grows with traffic. Production usually picks the second option and trades a little accuracy for a bounded cost.

2.3.3 Trade-offs and when to use it

It is accurate and gets rid of the boundary spike for good. The price is that every request leaves a trace, so memory and CPU costs are clearly higher. It suits login protection, endpoint scraping defense, and risk control rules where accuracy matters. At very large scale, teams usually layer local counters on top instead of sending every request to Redis.

A common starting point is 6 to 12 slots for a 60 second window. Going down to one second slots gets expensive fast.

2.4 Leaky bucket

2.4.1 The shop’s version

The third rule takes a different angle. The owner stops limiting how many orders are taken. A conveyor belt at the pickup counter delivers drinks at a steady pace, 20 cups per minute. Whether 20 or 200 people are waiting, the kitchen does not speed up. When the waiting area is full, new customers are turned away and told to come back.

2.4.2 The mechanics

A leaky bucket drops requests into a bucket with a fixed drain rate. An arriving request enters if there is room and is rejected if the bucket is full. Requests flow out at a constant rate and get consumed by the backend. The implementation only tracks two values, the current water level and the last drain time, and each request computes how much water leaked based on the elapsed time.

The drain is all about elapsed time. Ten seconds between two requests means ten seconds’ worth of water leaks out, and the longer the gap, the emptier the bucket. This is the mirror image of a token bucket refilling tokens.

2.4.3 Trade-offs and when to use it

The output is perfectly smooth. The downstream always sees a constant rate, with no bursts at all. The trade-off is that bursts have no chance. Even if the system built up spare capacity while idle, it cannot release it at peak. It fits databases, third-party APIs, and message consumers that demand a stable rate.

2.5 Token bucket

2.5.1 The shop’s version

The fourth rule finally handles bursts. The kitchen produces 20 bright yellow pickup tokens per minute and drops them into a bucket that holds at most 40. A customer grabs a token to pick up a drink, immediately if one is available, and waits for a fresh token otherwise. When traffic is light, tokens pile up. When the lunch rush hits, the first wave can spend the 40 accumulated tokens at once, while the kitchen refills at 20 per minute.

2.5.2 The mechanics

A token bucket has two parameters, the refill rate and the bucket capacity. Tokens are produced at a fixed rate, and the capacity caps how many can accumulate. A request consumes a token, passes if one is available, and is rejected or waits if not. Compared with a leaky bucket, the extra ability to save tokens is where bursts come from. The implementation keeps two values, the token count and the last refill time, which is a bit more work than a counter.

A common starting point for tuning is simple. Set rate to what the downstream can sustain, and set capacity to the largest burst the business allows. Capacity must be above 1, or the token bucket degrades into a leaky bucket.

Tokens refill at a fixed rate and the capacity sets the ceiling. When a request arrives, the bucket is checked first. Enough tokens means a deduction and a pass. Otherwise the request is rejected or waits. Saved tokens from idle periods are what make bursts possible, and that is the biggest difference from a leaky bucket.

2.5.3 Trade-offs and when to use it

It balances smoothness and bursts. The rate is capped, and short spikes can reach the capacity ceiling. The risk lives in the capacity parameter. Set it too high and a spike still overwhelms the downstream. Public APIs, gateway entrances, and most business rate limiting use it, which makes it the most common choice in production.

In practice, the most common token bucket implementation is Redis Lua, and all three implementations in Chapter 3 use it.

2.6 Side-by-side comparison and how to choose

My default is the token bucket. It controls both rate and bursts and covers most business scenarios. Pick the leaky bucket when smoothness is a hard requirement, the sliding window when accuracy is, and the fixed window when the only goal is cheap scraping protection.

Start with what you are protecting. Databases and abuse protection are different problems, and no single algorithm wins everywhere.

2.7 A question to think about

Question Why do public APIs like Stripe and the GitHub API almost always use a token bucket instead of a leaky bucket?

Callers of public APIs routinely burst. CI reruns, data syncs, and batch jobs can fire dozens of requests in the same second. A leaky bucket flattens that wave entirely, clients wait a long time, and timeout retries make it worse. A token bucket lets callers spend accumulated tokens at once, so part of the first peak gets through while the average rate stays capped by rate. Stripe’s engineering post on rate limiter design uses the token bucket as the main algorithm. Rate limiting keeps traffic inside what the system can handle. Short peaks are allowed, sustained overload is not. The same logic applies to thresholds. Size capacity for the peak and rate for the average, and the wider the gap between them, the more jitter the system absorbs.

Chapter 3: Distributed rate limiting in Java, production code you can compile and run

3.1 Why atomicity comes first

Rate limiting logic has three steps. Read the state, decide, deduct. When Java writes them as separate operations, races are unavoidable. Two requests both read one remaining token, both decide they can pass, and both go through. The limit fails on the spot.

Java splits the read, the decision, and the write into three Redis calls, so two instances can interleave and both see the stale value. Lua merges the three steps into one atomic execution, and a request that arrives later only ever sees the deducted result. That is what keeps distributed rate limiting correct.

A lock works on a single machine, but a JVM lock cannot stop another machine. A distributed lock is slow, heavy, and adds a new failure point. Redis solves this with Lua. Redis executes commands on a single thread, and when EVAL runs a script, the whole script completes atomically with no other command interleaved. Put the read, the refill, the decision, and the deduction in one script, and the race is gone.

There are two side benefits. One round trip runs the whole check, which beats a chain of GETs and SETs. And the logic lives on the Redis side, so the Java layer does not have to implement concurrency control.

For contrast, WATCH and MULTI can achieve atomicity with optimistic locking, but conflicting requests have to retry, and the retry rate explodes at peak. Lua has no retry cost. The script runs and the result is final. That is why mainstream rate limiting components all use Lua.

3.2 Implementation 1: plain Java, Redis, and a Lua token bucket

This option only depends on Jedis and does not tie you to a framework. It fits a self-built middleware or a project sensitive about dependencies.

3.2.1 Maven dependencies

Jedis 4.3.1, JUnit 5.9.2 for tests, and the Maven compiler targets Java 8.

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>ratelimit-jedis</artifactId>
    <version>1.0.0</version>
    <packaging>jar</packaging>

    <properties>
        <maven.compiler.source>8</maven.compiler.source>
        <maven.compiler.target>8</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <jedis.version>4.3.1</jedis.version>
        <junit.version>5.9.2</junit.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>redis.clients</groupId>
            <artifactId>jedis</artifactId>
            <version>${jedis.version}</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.36</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-simple</artifactId>
            <version>1.7.36</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.junit.jupiter</groupId>
            <artifactId>junit-jupiter</artifactId>
            <version>${junit.version}</version>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>3.0.0</version>
            </plugin>
        </plugins>
    </build>
</project>

3.2.2 The Lua script

Put the script below at src/main/resources/lua/token_bucket.lua. Refilling, deciding, and deducting all happen inside one EVAL, which is where the atomicity comes from.

-- Token bucket rate limiting script
-- KEYS[1]  bucket key, a hash that stores the token count in 'tokens' and the last refill time in 'ts'
-- ARGV[1]  bucket capacity, the maximum number of tokens that can accumulate
-- ARGV[2]  refill rate in tokens per second
-- ARGV[3]  tokens consumed by this request
-- Returns 1 to allow, 0 to reject

local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local rate = tonumber(ARGV[2])
local requested = tonumber(ARGV[3])

-- Use the Redis server time to avoid clock drift on client machines
local t = redis.call('time')
local now = tonumber(t[1])

local tokens = tonumber(redis.call('hget', key, 'tokens') or capacity)
local last = tonumber(redis.call('hget', key, 'ts') or now)

local elapsed = now - last
if elapsed > 0 then
    tokens = tokens + elapsed * rate
    if tokens > capacity then
        tokens = capacity
    end
end

-- Write back the refilled token count whether the request passes or not
redis.call('hset', key, 'tokens', tokens, 'ts', now)

if tokens >= requested then
    redis.call('hset', key, 'tokens', tokens - requested, 'ts', now)
    return 1
end

return 0

The script deliberately reads the time from the Redis server instead of accepting a client-supplied timestamp. Skewed clocks across application servers cannot affect the decision. The script uses the TIME command, and since Redis 5.0 scripts replicate by effects, the rate-limiting state stays consistent after a failover.

The argument order is fixed. The caller passes capacity, rate, and requested, and the script only computes. That keeps one script reusable across endpoints, with the key separating business dimensions.

3.2.3 The Java utility class

package com.example.ratelimit;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.nio.charset.StandardCharsets;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import redis.clients.jedis.JedisPooled;
import redis.clients.jedis.exceptions.JedisException;
import redis.clients.jedis.exceptions.JedisNoScriptException;

/**
 * Distributed token bucket rate limiter built on Jedis.
 * All state lives in Redis, so multiple instances share one bucket.
 *
 * @author Dylan
 */
public class TokenBucketRateLimiter implements AutoCloseable {

    private static final Logger log = LoggerFactory.getLogger(TokenBucketRateLimiter.class);

    private static final String LUA_PATH = "/lua/token_bucket.lua";

    private static final String LUA_SCRIPT = loadScript(LUA_PATH);

    private final JedisPooled jedis;

    /** Whether to allow requests when Redis fails. true keeps the business available, false rejects to protect the downstream. */
    private final boolean failOpen;

    /** SHA of the Lua script, loaded with SCRIPT LOAD on first use. */
    private volatile String scriptSha;

    public TokenBucketRateLimiter(JedisPooled jedis) {
        this(jedis, true);
    }

    public TokenBucketRateLimiter(JedisPooled jedis, boolean failOpen) {
        this.jedis = jedis;
        this.failOpen = failOpen;
    }

    /**
     * Try to acquire permits tokens.
     *
     * @param key      the rate limit dimension, for example order:create:9527
     * @param capacity bucket capacity
     * @param rate     tokens refilled per second
     * @param permits  tokens needed by this request
     * @return true to allow, false to reject
     */
    public boolean tryAcquire(String key, double capacity, double rate, int permits) {
        try {
            List<String> keys = Collections.singletonList(key);
            List<String> args = Arrays.asList(
                    String.valueOf(capacity),
                    String.valueOf(rate),
                    String.valueOf(permits));
            Object result = eval(keys, args);
            return Long.parseLong(result.toString()) == 1L;
        } catch (JedisException ex) {
            log.error("Rate limiting failed, applying the fallback policy, key={}", key, ex);
            if (failOpen) {
                return true;
            }
            throw new RateLimitUnavailableException("Rate limiter unavailable", ex);
        }
    }

    private Object eval(List<String> keys, List<String> args) {
        if (scriptSha == null) {
            scriptSha = jedis.scriptLoad(LUA_SCRIPT);
        }
        try {
            return jedis.evalsha(scriptSha, keys, args);
        } catch (JedisNoScriptException ex) {
            // The script cache is lost when Redis restarts, so reload and run once more
            scriptSha = jedis.scriptLoad(LUA_SCRIPT);
            return jedis.evalsha(scriptSha, keys, args);
        }
    }

    private static String loadScript(String path) {
        try (InputStream in = TokenBucketRateLimiter.class.getResourceAsStream(path)) {
            if (in == null) {
                throw new IllegalStateException("Lua script not found: " + path);
            }
            StringBuilder sb = new StringBuilder();
            try (BufferedReader reader = new BufferedReader(
                    new InputStreamReader(in, StandardCharsets.UTF_8))) {
                String line;
                while ((line = reader.readLine()) != null) {
                    sb.append(line).append('\n');
                }
            }
            return sb.toString();
        } catch (IOException ex) {
            throw new IllegalStateException("Failed to read the Lua script", ex);
        }
    }

    @Override
    public void close() {
        jedis.close();
    }
}

Two details worth knowing. The script is loaded once with SCRIPT LOAD and then executed with EVALSHA, which saves sending the script on every call. Redis can drop its script cache on restart, so JedisNoScriptException is caught and the script is reloaded once.

The companion exception class follows.

package com.example.ratelimit;

/** Thrown when the rate limiter is unavailable, used in fail-closed scenarios. */
public class RateLimitUnavailableException extends RuntimeException {

    public RateLimitUnavailableException(String message, Throwable cause) {
        super(message, cause);
    }
}

3.2.4 Unit tests

The tests need a local Redis at 127.0.0.1, port 6379. The first test fires 100 concurrent requests against a bucket with capacity 10 and expects exactly 10 to pass. The second verifies that tokens refill over time.

package com.example.ratelimit;

import java.util.UUID;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;

import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;

import redis.clients.jedis.JedisPooled;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;

class TokenBucketRateLimiterTest {

    private static final String HOST = "127.0.0.1";
    private static final int PORT = 6379;
    private static final String KEY_PREFIX = "test:token-bucket:";

    private static JedisPooled jedis;
    private static TokenBucketRateLimiter limiter;

    @BeforeAll
    static void setUp() {
        jedis = new JedisPooled(HOST, PORT);
        limiter = new TokenBucketRateLimiter(jedis, true);
    }

    @AfterAll
    static void tearDown() {
        jedis.close();
    }

    @Test
    void concurrentRequestsShouldBeLimitedByCapacity() throws Exception {
        String key = KEY_PREFIX + UUID.randomUUID();
        int total = 100;
        int capacity = 10;
        // Keep the refill rate tiny during the test so only the initial tokens are available
        double rate = 0.001;

        ExecutorService pool = Executors.newFixedThreadPool(total);
        CountDownLatch ready = new CountDownLatch(total);
        CountDownLatch start = new CountDownLatch(1);
        AtomicInteger passed = new AtomicInteger();
        AtomicInteger rejected = new AtomicInteger();

        for (int i = 0; i < total; i++) {
            pool.submit(() -> {
                ready.countDown();
                try {
                    start.await();
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                    return;
                }
                if (limiter.tryAcquire(key, capacity, rate, 1)) {
                    passed.incrementAndGet();
                } else {
                    rejected.incrementAndGet();
                }
            });
        }

        assertTrue(ready.await(10, TimeUnit.SECONDS), "threads did not become ready");
        start.countDown();
        pool.shutdown();
        assertTrue(pool.awaitTermination(30, TimeUnit.SECONDS), "tasks did not finish in time");

        assertEquals(10, passed.get(), "only 10 of 100 concurrent requests should get tokens");
        assertEquals(90, rejected.get(), "the other 90 requests should be rejected");
        jedis.del(key);
    }

    @Test
    void tokensShouldRefillOverTime() throws Exception {
        String key = KEY_PREFIX + UUID.randomUUID();
        int capacity = 2;
        double rate = 1.0;

        assertTrue(limiter.tryAcquire(key, capacity, rate, 2), "the first request should pass");
        assertFalse(limiter.tryAcquire(key, capacity, rate, 1), "should reject once the tokens run out");
        Thread.sleep(1100);
        assertTrue(limiter.tryAcquire(key, capacity, rate, 1), "should refill tokens after 1.1 seconds");
        jedis.del(key);
    }
}

3.2.5 How to run it and verify it works

How to run it.

1. Start Redis. docker run -d -p 6379:6379 --name redis7 redis:7-alpine

2. Put the pom.xml, the Lua script, and the two Java classes into a Maven project, keeping the directory layout.

3. Run mvn test.

How to verify it. Watch the test output. The concurrency test passes 10 and rejects 90, and the refill test passes again after waiting 1.1 seconds. Both assertions passing means the rate limiting works.

3.3 Implementation 2: Redisson

Redisson wraps the token bucket in RRateLimiter, which is the fastest option to wire into a production project.

3.3.1 Dependencies and configuration

Spring Boot 2.7.x pairs with Redisson 3.23.x. Matching the versions matters, because a mismatch shows up as class loading errors at startup.

<dependency>
    <groupId>org.redisson</groupId>
    <artifactId>redisson-spring-boot-starter</artifactId>
    <version>3.23.5</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

spring:
  application:
    name: ratelimit-redisson
  redis:
    host: 127.0.0.1
    port: 6379
    database: 0

The starter exposes a RedissonClient bean, and both the utility class and the aspect get their limiter from it.

Redisson’s limiter is also Lua under the hood, with the script and connection management wrapped up. rate and rateInterval combine into a speed, for example rate=20, rateInterval=1, unit=SECONDS means 20 tokens per second.

3.3.2 Calling it through a utility class

package com.example.ratelimit.redisson;

import java.util.concurrent.TimeUnit;

import javax.annotation.Resource;

import org.redisson.api.RRateLimiter;
import org.redisson.api.RateIntervalUnit;
import org.redisson.api.RateType;
import org.redisson.api.RedissonClient;
import org.springframework.stereotype.Component;

/**
 * Wraps the initialization and acquisition of RRateLimiter.
 *
 * @author Dylan
 */
@Component
public class RedissonRateLimiterUtil {

    @Resource
    private RedissonClient redissonClient;

    /** Non-blocking acquisition, returns false immediately when no token is available. */
    public boolean tryAcquire(String key, long rate, long rateInterval,
                              RateIntervalUnit unit, int permits) {
        RRateLimiter rateLimiter = redissonClient.getRateLimiter(key);
        // The config is only written on the first call, already configured keys are skipped
        rateLimiter.trySetRate(RateType.OVERALL, rate, rateInterval, unit);
        return rateLimiter.tryAcquire(permits);
    }

    /** Acquisition with a wait time, blocking the current thread while waiting. */
    public boolean tryAcquire(String key, long rate, long rateInterval,
                              RateIntervalUnit unit, int permits,
                              long timeout, TimeUnit timeUnit) {
        RRateLimiter rateLimiter = redissonClient.getRateLimiter(key);
        rateLimiter.trySetRate(RateType.OVERALL, rate, rateInterval, unit);
        return rateLimiter.tryAcquire(permits, timeout, timeUnit);
    }
}

Two things to keep in mind. trySetRate only applies when the key does not exist yet, so changing a threshold means deleting the old key and setting it again. RRateLimiter keys do not expire on their own, so watch Redis memory over the long run.

3.3.3 Calling it through an annotation

First define the annotation. The key supports SpEL placeholders such as order:create:#userId, and the annotation itself carries no logic, only the dimension and the rate.

package com.example.ratelimit.redisson;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

import org.redisson.api.RateIntervalUnit;

/** Method-level distributed rate limiting annotation. */
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface RateLimit {

    /** Rate limit key, supports #parameterName placeholders */
    String key();

    /** Rate value */
    long rate() default 10;

    /** Rate window length */
    long rateInterval() default 1;

    /** Rate window unit */
    RateIntervalUnit unit() default RateIntervalUnit.SECONDS;

    /** Tokens consumed per call */
    int permits() default 1;
}

Next comes the aspect. It resolves the key, initializes the limiter, and performs the decision. The aspect binds the annotation with @Around, grabs a token before the method runs, and throws when it cannot.

package com.example.ratelimit.redisson;

import javax.annotation.Resource;

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.reflect.MethodSignature;
import org.redisson.api.RRateLimiter;
import org.redisson.api.RateType;
import org.redisson.api.RedissonClient;
import org.springframework.expression.ExpressionParser;
import org.springframework.expression.common.TemplateParserContext;
import org.springframework.expression.spel.standard.SpelExpressionParser;
import org.springframework.expression.spel.support.StandardEvaluationContext;
import org.springframework.stereotype.Component;

/**
 * Resolves the rate limit key from the annotation and method arguments,
 * then runs the token acquisition uniformly.
 *
 * @author Dylan
 */
@Aspect
@Component
public class RateLimitAspect {

    private static final ExpressionParser PARSER = new SpelExpressionParser();
    private static final TemplateParserContext TEMPLATE = new TemplateParserContext();

    @Resource
    private RedissonClient redissonClient;

    @Around("@annotation(rateLimit)")
    public Object around(ProceedingJoinPoint joinPoint, RateLimit rateLimit) throws Throwable {
        String key = resolveKey(rateLimit.key(), joinPoint);
        RRateLimiter rateLimiter = redissonClient.getRateLimiter(key);
        rateLimiter.trySetRate(RateType.OVERALL, rateLimit.rate(), rateLimit.rateInterval(), rateLimit.unit());
        if (rateLimiter.tryAcquire(rateLimit.permits())) {
            return joinPoint.proceed();
        }
        throw new RateLimitException("Too many requests, please try again later");
    }

    private String resolveKey(String template, ProceedingJoinPoint joinPoint) {
        if (template.contains("#")) {
            MethodSignature signature = (MethodSignature) joinPoint.getSignature();
            StandardEvaluationContext context = new StandardEvaluationContext();
            String[] paramNames = signature.getParameterNames();
            Object[] args = joinPoint.getArgs();
            if (paramNames != null) {
                for (int i = 0; i < paramNames.length; i++) {
                    context.setVariable(paramNames[i], args[i]);
                }
            }
            return PARSER.parseExpression(template, TEMPLATE).getValue(context, String.class);
        }
        return template;
    }
}

The companion exception follows. It extends RuntimeException, so the business layer can catch it and return a unified error code.

package com.example.ratelimit.redisson;

public class RateLimitException extends RuntimeException {

    public RateLimitException(String message) {
        super(message);
    }
}

Here is the annotation in a business method. The key uses the #userId placeholder, and the aspect pulls the real user ID from the method arguments.

package com.example.ratelimit.service;

import org.redisson.api.RateIntervalUnit;
import org.springframework.stereotype.Service;

import com.example.ratelimit.redisson.RateLimit;

@Service
public class OrderService {

    @RateLimit(key = "order:create:#userId", rate = 20, rateInterval = 1,
            unit = RateIntervalUnit.SECONDS, permits = 1)
    public Order createOrder(Long userId, OrderCreateRequest request) {
        // normal business logic
        return new Order();
    }
}

If getParameterNames() returns null, add the -parameters flag to the compiler plugin so the aspect can resolve #userId.

3.3.4 Notes for cluster environments

Cluster mode keeps its configuration in a separate file that application.yml points to.

spring:
  redis:
    redisson:
      file: classpath:redisson-cluster.yaml

clusterServersConfig:
  nodeAddresses:
    - "redis://10.0.0.11:6379"
    - "redis://10.0.0.12:6379"
    - "redis://10.0.0.13:6379"
  scanInterval: 1000
  timeout: 3000
  retryAttempts: 2
  retryInterval: 1500

A few things deserve attention in cluster mode. Use RateType.OVERALL so the whole cluster shares one bucket, because PER_CLIENT only makes sense on a single machine. Prefix keys with the business dimension, for example rate:order:create:9527, so endpoints do not pollute each other. Keep the Redis timeout short. The rate limiter must not slow down the main flow, and timeouts should fall through to the fallback policy.

Sentinel and Cluster have different configuration layouts, and a migration should also check that the script keys land in the same slot. Single-key scripts are unaffected. Multi-key scripts must use hash tags to keep their keys together.

3.3.5 How to run it and verify it works

How to run it.

1. Start Redis, add the dependencies, and configure application.yml.

2. Inject RedissonRateLimiterUtil, or annotate a method with @RateLimit.

3. Start the application and call the endpoint repeatedly.

How to verify it. Set rate to 1 with a 1 second window. Fire requests within one second. The first passes and the rest return RateLimitException or false. Wait one second, request again, and it passes again.

3.4 Implementation 3: Spring Cloud Gateway

3.4.1 Why put rate limiting at the gateway

The gateway is the single entry point for traffic. Rate limiting there means zero changes to business code. Rejected requests never reach the services, which saves resources across the entire call path. The rules stay in one place and can combine route, IP, and user dimensions freely.

The cost is worth knowing too. Every request pays one Redis round trip, so the limiter itself becomes a cap on the gateway’s QPS. Account for it during capacity planning.

3.4.2 Dependencies and basic configuration

Spring Boot 2.7.18 with Spring Cloud 2021.0.8.

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.7.18</version>
</parent>

<properties>
    <spring-cloud.version>2021.0.8</spring-cloud.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-gateway</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-redis-reactive</artifactId>
    </dependency>
</dependencies>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-dependencies</artifactId>
            <version>${spring-cloud.version}</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Here is the route configuration. replenishRate is the number of tokens refilled per second, burstCapacity is the bucket capacity, and requestedTokens is how many tokens each request consumes.

spring:
  application:
    name: gateway
  redis:
    host: 127.0.0.1
    port: 6379
  cloud:
    gateway:
      routes:
        - id: order-service
          uri: lb://order-service
          predicates:
            - Path=/order/**
          filters:
            - name: CustomRequestRateLimiter
              args:
                key-resolver: "#{@ipKeyResolver}"
                redis-rate-limiter.replenishRate: 10
                redis-rate-limiter.burstCapacity: 20
                redis-rate-limiter.requestedTokens: 1

The three parameters map straight back to the boba shop. replenishRate is how many tokens the kitchen makes per minute, burstCapacity is the bucket size, and requestedTokens is what a customer spends to pick up one drink.

3.4.3 KeyResolver: choosing the rate limit dimension

The built-in RedisRateLimiter needs to know which dimension to limit on. The bean below limits by client IP. In production you would usually switch to a user ID or an endpoint plus user combination.

package com.example.gateway.limiter;

import java.util.Objects;

import org.springframework.cloud.gateway.filter.ratelimit.KeyResolver;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import reactor.core.publisher.Mono;

@Configuration
public class RateLimitConfig {

    /** Limits by client IP. Production usually combines a user ID or endpoint dimension. */
    @Bean
    public KeyResolver ipKeyResolver() {
        return exchange -> {
            String ip = Objects.requireNonNull(exchange.getRequest().getRemoteAddress())
                    .getAddress().getHostAddress();
            return Mono.just(ip);
        };
    }
}

3.4.4 Customizing the rate limit response

When the built-in filter rejects a request, it commits an empty 429 response and callers get no message. Returning a consistent JSON body takes two steps. First make the filter throw instead of committing the response, then let a unified exception handler render the JSON.

Start with the custom filter. It replicates the built-in decision flow and only changes the rejection path to throw a ResponseStatusException. The class name determines the filter name in the yml, so CustomRequestRateLimiterGatewayFilterFactory maps to CustomRequestRateLimiter.

package com.example.gateway.limiter;

import java.util.Map;

import org.springframework.cloud.gateway.filter.GatewayFilter;
import org.springframework.cloud.gateway.filter.factory.RequestRateLimiterGatewayFilterFactory;
import org.springframework.cloud.gateway.filter.ratelimit.KeyResolver;
import org.springframework.cloud.gateway.filter.ratelimit.RateLimiter;
import org.springframework.cloud.gateway.route.Route;
import org.springframework.cloud.gateway.support.ServerWebExchangeUtils;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ResponseStatusException;

import reactor.core.publisher.Mono;

/**
 * The built-in rate limiter commits a bare 429 response when it rejects a request,
 * so callers receive no body. This subclass changes the rejection path to throw,
 * and a unified exception handler renders the JSON.
 *
 * @author Dylan
 */
@Component
public class CustomRequestRateLimiterGatewayFilterFactory
        extends RequestRateLimiterGatewayFilterFactory {

    public CustomRequestRateLimiterGatewayFilterFactory(RateLimiter rateLimiter, KeyResolver keyResolver) {
        super(rateLimiter, keyResolver);
    }

    @Override
    public GatewayFilter apply(Config config) {
        return (exchange, chain) -> {
            KeyResolver resolver = config.getKeyResolver() != null
                    ? config.getKeyResolver() : getDefaultKeyResolver();
            RateLimiter<?> limiter = config.getRateLimiter() != null
                    ? config.getRateLimiter() : getDefaultRateLimiter();

            return resolver.resolve(exchange).defaultIfEmpty("").flatMap(key -> {
                if (key.isEmpty()) {
                    if (isDenyEmptyKey()) {
                        return Mono.error(
                                new ResponseStatusException(config.getStatusCode(), "empty key"));
                    }
                    return chain.filter(exchange);
                }
                String routeId = config.getRouteId();
                if (routeId == null) {
                    Route route = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_ROUTE_ATTR);
                    routeId = route != null ? route.getId() : "";
                }
                String finalRouteId = routeId;
                return limiter.isAllowed(finalRouteId, key).flatMap(response -> {
                    for (Map.Entry<String, String> header : response.getHeaders().entrySet()) {
                        exchange.getResponse().getHeaders().add(header.getKey(), header.getValue());
                    }
                    if (response.isAllowed()) {
                        return chain.filter(exchange);
                    }
                    return Mono.error(
                            new ResponseStatusException(config.getStatusCode(), "rate limit exceeded"));
                });
            });
        };
    }
}

Then the exception handler. It only takes over for 429, and rethrows everything else to the framework’s default handling.

package com.example.gateway.limiter;

import java.nio.charset.StandardCharsets;

import org.springframework.boot.web.reactive.error.ErrorWebExceptionHandler;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ResponseStatusException;
import org.springframework.web.server.ServerWebExchange;

import reactor.core.publisher.Mono;

/**
 * Renders rate limit exceptions as a uniform JSON body.
 *
 * @author Dylan
 */
@Component
@Order(Ordered.HIGHEST_PRECEDENCE + 1)
public class RateLimitErrorWebExceptionHandler implements ErrorWebExceptionHandler {

    @Override
    public Mono<Void> handle(ServerWebExchange exchange, Throwable ex) {
        if (ex instanceof ResponseStatusException) {
            ResponseStatusException rse = (ResponseStatusException) ex;
            if (HttpStatus.TOO_MANY_REQUESTS.equals(rse.getStatus())) {
                exchange.getResponse().setStatusCode(HttpStatus.TOO_MANY_REQUESTS);
                exchange.getResponse().getHeaders().setContentType(MediaType.APPLICATION_JSON);
                String body = "{\"code\":429,\"message\":\"Too many requests, please try again later\"}";
                DataBuffer buffer = exchange.getResponse().bufferFactory()
                        .wrap(body.getBytes(StandardCharsets.UTF_8));
                return exchange.getResponse().writeWith(Mono.just(buffer));
            }
        }
        return Mono.error(ex);
    }
}

3.4.5 How to run it and verify it works

How to run it.

1. Add the gateway and reactive Redis dependencies and configure the route.

2. Put the KeyResolver, the custom filter, and the exception handler into the project.

3. Start the gateway and hit the limited route repeatedly.

How to verify it. Run curl -i 30 times in a row. The first 20 return the business result, and from request 21 the gateway returns HTTP 429 with the JSON body. The response headers also show the token bucket state, such as X-RateLimit-Remaining.

3.5 For comparison: Guava local rate limiting

Guava’s RateLimiter is a plain local token bucket and the simplest code in this article, which makes the local versus distributed difference easy to see. The demo uses Guava 32.1.3-jre, and the class is called DylanRateLimiterDemo. Run main and watch what happens.

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>32.1.3-jre</version>
</dependency>

package com.example.ratelimit;

import com.google.common.util.concurrent.RateLimiter;

/**
 * Local token bucket demo.
 *
 * @author Dylan
 */
public class DylanRateLimiterDemo {

    public static void main(String[] args) {
        // Refills 10 tokens per second, and the bucket capacity defaults to 10
        RateLimiter rateLimiter = RateLimiter.create(10.0);

        for (int i = 1; i <= 15; i++) {
            if (rateLimiter.tryAcquire()) {
                System.out.println("Request " + i + " allowed");
            } else {
                System.out.println("Request " + i + " rejected");
            }
        }
    }
}

When you run it, the first 10 requests pass and the rest are rejected. But it only works inside this JVM. With three instances behind a gateway, the total becomes 30. When a global quota is required, local rate limiting can only act as a fallback, not the primary mechanism.

3.6 Choosing among the three distributed implementations

There is no single right answer. Here is a conservative combination I would start with.

• Use Spring Cloud Gateway at the entry for coarse, endpoint-level limiting that blocks most abnormal traffic.

• Use Redisson inside services for precise, user-level limiting, with flexible rules and low integration cost.

• Build your own with Jedis and Lua when you need a custom algorithm or want to avoid a framework, with full control over the logic.

• Keep Guava for single machine scenarios and local fallbacks.

The combination follows one pattern. The closer a layer is to the traffic, the coarser its limiting. The closer it is to the business, the finer. Coarse layers chase speed, fine layers chase accuracy.

Chapter 4: Architecture design for distributed rate limiting, diagrams included

All four diagrams are written in Mermaid and render directly in Markdown. Each one comes with a short walkthrough.

4.1 Layered rate limiting architecture

Traffic gets filtered layer by layer from the edge. Nginx applies coarse connection and IP level limits. The gateway limits by endpoint with a token bucket. Services then limit precisely by user. Each layer is configured independently, the coarse layers absorb the bulk, and the fine layers handle the details. All rate limiting state lives in the Redis cluster, so service instances stay stateless. The monitoring platform collects allowed counts, rejected counts, RT, and error rates, and pushes dynamic thresholds down to the gateway and services. The fallback module handles fail fast, queueing, and fallback data.

The point is that every layer limits, but each with a different granularity. As the granularity tightens, the room for error shrinks.

4.2 Token bucket execution flow

Inside the Lua script, the request first reads the token count and the last refill time, refills tokens by the elapsed time, and then decides. The whole flow runs atomically on Redis’s single thread, so no concurrent request can interleave and read stale state. A return of 1 allows the request, 0 rejects it, and the client does not need a second check. Time comes from the Redis server, so clock drift on application machines cannot affect the decision.

The only condition is whether the refilled token count is at least the number requested. Even a rejection writes the state back, so the next request always reads the latest value.

4.3 Rate limit fallback flow

Once a request is limited, the chosen strategy depends on the business. Fail fast returns 429 directly. Queueing suits traffic shaping. Fallback data suits read-heavy workloads. A circuit breaker fits when the downstream is already failing. Redis failure takes a separate branch, where a degradation switch decides between fail-open and fail-closed. The switch lives in the config center, so it can change without a release, and traffic returns to Redis rate limiting automatically once Redis recovers.

Policies can be set per endpoint. Checkout fails fast, exports queue, and reads fall back to cache.

4.4 High availability deployment topology

Service instances are stateless and all rate limiting state sits in Redis, so instances scale out freely. Redis runs in sentinel or cluster mode, and a master failure fails over automatically without losing the limiting state. Each instance only connects to Redis and does not depend on other instances, so one failing instance does not affect anyone else’s decisions. The monitoring platform watches Redis latency and replication state to catch capacity problems early.

Size for the worst case. Leave Redis QPS headroom for the rate limiting peak, and include replication lag in the monitoring.

Chapter 5: Production pitfalls and good practices

5.1 The pitfalls that show up most often

A single point of failure in Redis. Rate limiting depends on Redis, and when Redis goes down the limiting goes with it. Teams without a fallback end up either allowing everything or rejecting everything, and neither is controlled. Decide the fallback policy during design, not during the incident. The fix is in section 5.2.

Clock drift. If the script refills tokens using client time, a fast or slow application server shifts the decision. Use the Redis server time everywhere, and in multi region deployments watch the NTP sync on the Redis machines. When NTP drifts, the limiting clock and the business clock disagree, and log timestamps stop lining up.

Hot keys. Popular campaigns, bestsellers, and hot open source repositories all funnel requests onto a single key, and one Redis shard becomes the bottleneck. The usual fixes are layering local limiting under distributed limiting, or sharding the key by business so each shard carries its own quota. Keep the shard count modest. Eight shards are usually enough, and more than that starts costing accuracy.

Granularity that is too coarse or too fine. One global key means a single abusive user exhausts the limit for everyone. One key per user explodes memory and can be bypassed with throwaway accounts. A two-level combination works well, endpoint level for the total and user level for each user. The two keys would look like rate:api:order:create and rate:user:9527:order:create.

Killing legitimate requests. A threshold picked from thin air will hurt normal traffic the moment it fluctuates. Base the threshold on historical peak statistics, keep tuning against the rejection and false positive rates, and run load tests against the plan before a launch or campaign. Track the false positive rate itself. It is the first number I look at after a rate limiter ships.

5.2 High availability options

Redis high availability. Production should run at least sentinel mode, and cluster mode for critical traffic. All rate limiting state is read and written on the primary, with sentinel handling the failover.

Failover has a short window of unavailability, so test what the limiter does during that window too.

Kill switches. Keep three switches in the config center. A master switch turns rate limiting on or off. A degradation switch controls behavior after a Redis failure. A whitelist switch lets internal systems and load test traffic through.

Stripe’s engineering post on running rate limiters in production stresses the same two things. Fail open when Redis fails so the API does not go down with it, and keep a kill switch so the limiter can be disabled in one action. That matches the switch design above.

Multi-layer fallbacks. Nginx coarse limiting, gateway token buckets, and service local limiting stack into three layers. If one fails, the others keep working.

Local standby. Each instance carries a local token bucket and switches to it automatically when Redis fails. The per-instance value is the total divided by the instance count, so accuracy drops, but the system is never running without a limit.

The limiter first checks whether Redis is available. If it is, the distributed path runs. On failure or timeout, it switches to the local token bucket immediately. The local threshold is the total divided by the instance count, less accurate but still a limit. When Redis recovers, it switches back, and callers never notice.

5.3 Monitoring and tuning

Track four metrics. Requests allowed, requests rejected, rejection rate, and limiter latency. A sudden jump in rejection rate usually means the threshold is too low. A jump in latency means Redis or the network has a problem. Export the metrics to Prometheus and set alert rules per endpoint tier.

If the limiter takes more than 10ms, investigate. It should stay under 1ms in normal conditions.

Do not guess alert thresholds. When allowed traffic drops and rejections spike at the same time, the limiter is almost certainly hurting legitimate users. Check the whitelist and dynamic thresholds first, then Redis latency.

Dynamic thresholds live in the config center. Keep them conservative day to day, raise them per the load test results before a launch or campaign, and roll back when it ends. Log every change so reviews have a trail.

Canary verification is a fixed step before launch. Route 1% of traffic, watch the rejection and error rates, and only scale up once there is no false positive problem. Rate limit config changes go through the same process, because they affect production just like code.

5.4 Choosing by scenario

The table gives defaults. Before production, weigh the team’s operational capacity, Redis availability, and how much false positive rate the business tolerates.

Start with the goal. Anti-abuse needs accuracy, so use a window. Protecting a database needs a constant rate, so use a leaky bucket. A public API needs both an average rate cap and burst tolerance, so use a token bucket. Once the goal is clear, the design mostly follows.

Chapter 6: Summary and next steps

6.1 The key points

Here are the key points.

• The fixed window is simple but spikes at boundaries. The sliding window fixes the accuracy with time slices.

• The leaky bucket guarantees smooth output. The token bucket allows bursts, and it is the default choice in production.

• Distributed rate limiting puts the decision and the deduction in one Redis Lua script, which is what makes it atomic.

• The gateway handles coarse limiting, services handle precise limiting, and local limiting covers fallbacks. All three layers together make a complete design.

• Redis failures need a degradation switch, with fail-open or fail-closed chosen per business.

Memorizing those five only helps in interviews. To actually learn this, run the code in Chapter 3 and then load test it against the pitfalls in Chapter 5.

6.2 How rate limiting, circuit breaking, degradation, and isolation fit together

These four components get lumped together a lot, but their jobs are distinct.

• Rate limiting caps the incoming rate at the entry, so overloaded requests never reach the business layer.

• Circuit breaking watches the downstream and fails fast after repeated failures, stopping the spread.

• Degradation decides what to return, fallback data or an error.

• Isolation confines slow dependencies inside their own thread pool or semaphore, so one endpoint cannot take down the whole service.

The four pieces chain together in order. Rate limiting controls the input rate. Circuit breaking controls downstream failures. Isolation keeps slow dependencies inside their own pool. Degradation gives users a fallback result when things fail. Get the order wrong and both alerting and debugging get confusing.

A complete traffic management setup looks like this. Gateway limiting blocks most traffic. Service layer limiting protects the primary endpoints. Circuit breaking and degradation cover dependency failures. Isolation controls the blast radius. Those four components plus monitoring make the whole picture.

6.3 Questions to keep thinking about

Two questions to keep in mind, useful for interviews and for everyday design.

Question Should thresholds be dynamic or fixed? What should drive the adjustment?

Question Adaptive rate limiting can adjust thresholds from RT or error rates. What problem does that solve, and what new risks does it introduce?

I am not giving answers here. Being able to explain your own trade-offs beats memorizing ten conclusions.

References

• Redis EVAL documentation

• Redisson rate limiter wiki

• Spring Cloud Gateway rate limiter filter documentation

• Jedis project

• Stripe engineering blog, Scaling your API with rate limiters

• GitHub REST API rate limits documentation

• AWS API Gateway throttling documentation

• Google SRE workbook, Managing Load

When designing interfaces, we need to carefully consider many aspects. For example, the standardization of interfaces is like traffic rules, enabling orderly interaction between different systems.
Security is also very important. Just like guardians protecting treasures, it prevents data from being maliciously attacked or leaked.
At the same time, extensibility cannot be ignored. It can make interfaces like elastic containers continuously expand functions as requirements change.
Stability is also crucial. Just like a stable power supply, it provides a reliable guarantee for the normal operation of the system.

This article will sort out what key points need to be considered when designing good interfaces.

1. Interface Parameter Verification

The legality verification of input parameters is a prerequisite for a good interface. Through the verification of input parameters, we can filter out many invalid requests and improve the stability of the system.

We can divide the verification of input parameter legality into: routine verification and business verification.

The so-called routine verification includes: token verification, required field verification, length verification, type verification, and so on.

Business verification is the verification in specific business scenarios. For example, for the commodity order placement interface, the order amount must be greater than 0.

public String placeOrder(@RequestBody Order order) {
    if (order.getOrderAmount() > 0) {
        return “Order placed successfully.”;
    } else {
        return “Order amount must be greater than zero.”;
    }
}

2. Key Interface Log Printing

Key interfaces must have logs, including “input parameter logs”, “output parameter logs”, and “exception logs”. In this way, once a problem occurs, we can quickly locate the problem by checking the logs.

Of course, if we feel that there are too many logs printed in this way, we can set log levels and print logs of this level only in specific circumstances.

If we provide an interface for use by the client, then for each incoming request, we must record the request “input parameters”, “output parameters”, and the “unique identification ID” they give.

If the client reports a problem, we can query the complete link of this request through this unique ID. This way, it is convenient to troubleshoot and locate problems. Secondly, if it is not a problem with our interface, the logs are also evidence and there is no need to argue anymore.

@Getter
class InputData {
    private String id;
    private int param;

    public InputData(int param) {
        this.id = UUID.randomUUID().toString();
        this.param = param;
    }
}

public class InterfaceLoggerExample {

    private static final Logger logger = LoggerFactory.getLogger(InterfaceLoggerExample.class);

    public static void main(String[] args) {
        InputData inputData = new InputData(10);

        try {
            // Log the input parameter with its unique identifier
            logger.info(”Interface input parameter (ID: {}): {}”, inputData.getId(), inputData.getParam());

            // Simulate interface processing
            int result = processInput(inputData.getParam());

            // Log the output parameter with the same identifier
            logger.info(”Interface output parameter (ID: {}): {}”, inputData.getId(), result);
        } catch (Exception e) {
            // Log error message when an error occurs in the interface
            logger.error(”An error occurred in the interface (ID: {}): “, inputData.getId(), e);
        }
    }

    private static int processInput(int input) {
        if (input % 2 == 0) {
            return input * 2;
        } else {
            throw new IllegalArgumentException(”Input is not an even number”);
        }
    }
}

3. Interface Idempotency Design

The so-called idempotency means that multiple invocations of a method or interface will not change the business state and can ensure that the result of repeated invocations is consistent with the result of a single invocation.

In development, our main operations are CURD. Among them, the “read” operation and the “delete” operation are naturally idempotent. What we care about are the “create” operation and the “update” operation.

The “create operation” must be non-idempotent because it involves the generation of new data. The “update operation” may be idempotent or non-idempotent, depending on the specific business scenario.

For more detailed content on idempotency design, you can refer to another article of mine: Idempotency Control

4. Interface Throttling Control

Throttling is to better maintain the stability of the system. If we provide an interface for use by a third-party system, then throttling of this interface is very necessary.

On the one hand, throttling can prevent the interface from being brushed, causing unnecessary pressure on the service layer. On the other hand, it is to prevent the interface from being abused.

Here is a throttling case implemented by using custom annotations + AOP.

/**
 * @Target indicates that this annotation can be applied to methods.
 * @Retention indicates that this annotation is available at runtime.
 */
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface RateLimiter {
    /**
     * The default value is 1.
     */
    int value() default 1;
    /**
     * The default value is 1 second.
     */
    int durationInSeconds() default 1;
}

/**
 * Aspect class for rate limiting.
 */
@Aspect
@Component
public class RateLimiterAspect {

    /**
     * A concurrent hash map to store rate limiters for different methods.
     */
    private final ConcurrentHashMap<String, RateLimiter> rateLimiters = new ConcurrentHashMap<>();

    /**
     * Pointcut expression to target methods annotated with @RateLimiter.
     */
    @Pointcut(”@annotation(RateLimiter)”)
    public void rateLimiterPointcut(RateLimiter rateLimiterAnnotation) {
    }

    /**
     * Advice method that wraps around the targeted methods for rate limiting.
     */
    @Around(”rateLimiterPointcut(rateLimiterAnnotation)”)
    public Object around(ProceedingJoinPoint joinPoint, RateLimiter rateLimiterAnnotation) throws Throwable {
        int permits = rateLimiterAnnotation.value();
        int durationInSeconds = rateLimiterAnnotation.durationInSeconds();

        // Use the method signature as the key for the rate limiter.
        String key = joinPoint.getSignature().toLongString();
        com.google.common.util.concurrent.RateLimiter rateLimiter = rateLimiters.computeIfAbsent(key, k -> com.google.common.util.concurrent.RateLimiter.create((double) permits / durationInSeconds));

        // Try to acquire a token. If successful, execute the method. Otherwise, throw an exception.
        if (rateLimiter.tryAcquire()) {
            return joinPoint.proceed();
        } else {
            throw new RuntimeException(”Rate limit exceeded.”);
        }
    }
}

/**
 * REST controller class.
 */
@RestController
public class ApiController {

    /**
     * Endpoint with rate limiting set to 10 requests per minute.
     */
    @GetMapping(”/api/limited”)
    @RateLimiter(value = 10, durationInSeconds = 60)
    public String limitedEndpoint() {
        return “This API has a rate limit of 10 requests per minute.”;
    }

    /**
     * Endpoint without rate limiting.
     */
    @GetMapping(”/api/unlimited”)
    public String unlimitedEndpoint() {
        return “This API has no rate limit.”;
    }
} 

5. Sensitive Data Masking

During the process of interface invocation, some sensitive fields may be involved, such as “ID number”, “bank card number”, “address”, “mobile phone number”, etc. These data usually need to be masked. For example, 123–456–7890 can be masked as 123–456-XXX.

6. Prerequisite for Requesting an Interface — Token

The general process is as follows. First, after the user logs in successfully, a token is generated and returned to the front end. At the same time, the back end uses this token as the key and stores the user information as the value in the Redis cache.

Then, when accessing other interfaces later, bring this token in the request header to request the interface. The back end intercepts this interface through an interceptor and checks whether the key exists in Redis. If it does not exist, it directly returns, indicating that the user is not logged in!

Of course, some interfaces can be accessed without logging in, such as the registration interface. Then the interceptor can filter these interfaces.

7. Considering Exceptions, Timeouts, and Retries When Calling Third-Party Interfaces

Whether we provide our interface to a third party or request a third-party interface, we need to consider more. When calling a third-party interface, the first things to consider are timeouts, exceptions, and retries.

• Timeout: We cannot determine how long a third-party interface will take to return, so we should set a timeout period to ensure that our interface is not stuck.

• Exception: All interfaces may have exceptions. If an exception occurs when calling a third-party interface, there must be logs. At the same time, consider whether to retry or handle it with an alarm.

• Retry: If calling a third-party interface fails, it may be due to network reasons. Do we need to retry? How many times to retry needs to be considered.

8. Unified Response Data Format

In development, when we define an interface, we need to return a response body in a unified format. The response data generally contains three attributes: “status code” (code), “information description” (message), and “response data” (data).

The client can quickly determine whether the interface request is successful based on the status code. If it is successful, it starts processing the data. If it fails, it directly throws out the message information.

9. Single Responsibility of Interface

The single responsibility principle states that under the condition of clarifying the responsibility of the interface, try to make the interface single as much as possible, that is, one interface only does one thing, not two or more.

Doing this has many benefits. First, it can make the code easier to maintain. If an interface only does one thing, when this function needs to be modified, it is easy to find the corresponding interface for modification without affecting other unrelated functions. Second, it is convenient for expansion. If a new function needs to be added, a new interface can be created without interfering with the existing interface. Finally, it can improve the readability of the code. The name and function of the interface are clear at a glance, and other developers can know what this interface is for at a glance.

For example, in a shopping system, there can be an order interface specifically responsible for order-related operations, a commodity interface specifically handling commodity matters, and a user interface focusing on user management. In this way, the responsibilities of each interface are clear, and the system will be more stable and easy to maintain.

Many non-senior interface designers always think that the more things an interface does, the more convenient it is when designing interfaces. This is a very serious misunderstanding.

10. Does the Interface Need to Adopt Asynchronous Processing

For example, if you implement a user registration interface. When a user registers successfully, send an email or text message to notify the user.

This sending of an email or text message is more suitable for asynchronous processing. After all, a failure of a notification type cannot lead to a registration failure.

So how do we perform asynchronous operations? We can use a message queue. That is, after a user registers successfully, the producer generates a registration success message. When the consumer pulls the registration success message, it sends a notification.

11. Thread pool isolation for core interfaces

In business code, many interfaces may use thread pools. In addition to core interfaces, some ordinary interfaces may also use thread pools (to improve processing efficiency). If thread pool isolation is not implemented, if an ordinary interface has a bug and fills up the thread pool, it will cause your main business to be affected.

12. Improve the response time of interfaces

1. Try to use indexes for database queries to optimize query speed.

2. Consider whether to add caches: local caches, Redis caches, ES storage, and so on.

13. Optimize serial processing of interfaces to parallel processing

Suppose we want to develop the homepage of a website. We design an interface for querying homepage data. This interface needs to query user information, header information, news information, and so on.

The simplest way is to call the interfaces serially one by one. If you want to improve performance, then adopt a parallel calling method and query simultaneously instead of blocking.

You can use CompletableFuture (recommended) or FutureTask (not recommended).

Map<Long, List<SubjectLabelBO>> map = new HashMap<>();
List<CompletableFuture<Map<Long, List<SubjectLabelBO>>>> completableFutureList = 
categoryBOList.stream().map(category ->
        CompletableFuture.supplyAsync(() -> getLabelBOList(category), labelThreadPool)
).collect(Collectors.toList());

completableFutureList.forEach(future -> {
    try {
        Map<Long, List<SubjectLabelBO>> resultMap = future.get(); 
        map.putAll(resultMap);
    } catch (Exception e) {
        e.printStackTrace();
    }
});

14. Control lock granularity of interfaces

In high-concurrency scenarios, we will perform locking operations on shared resources to ensure thread safety. However, if the locking granularity is too large, it will affect the interface performance.

What is locking granularity? For example, you bring a love letter home but don’t want your parents to find it. Then you secretly return to your room and put it in a lockable drawer instead of locking the door of the room. This reduces the granularity of the lock.

Code example:

• Excessive lock granularity: Lock methods A and B entirely. But in fact, I only want to lock method A. This is excessive lock granularity.

void test(){
    synchronized (this) {
       B();
       A();
    }
}

Reduce lock granularity: Only lock method A when it is actually needed instead of locking both method A and method B. This reduces the lock granularity and can improve interface performance in high-concurrency scenarios.

void test(){
    B();
    synchronized (this) {
       A();
    }
}

15. Avoid long transactions

During a long transaction, CPU and memory usage may increase. In severe cases, it will lead to slow overall response of the server and make online applications unusable.

In addition to possible problems with SQL itself, the causes of long transactions are also closely related to the transaction control logic at the application layer.

How can we avoid long transaction problems as much as possible❓

1. Do not put remote interface calls inside transactions.

2. If possible, put some query-related operations outside transactions as much as possible.

3. In concurrent scenarios, try to avoid using the @Transactional annotation to operate transactions. Use orchestrated transactions of TransactionTemplate to flexibly control the scope of transactions.

When using @Transactional to manage transactions originally, it is like this.

@Transactional
public int createUser(User user){
    // Save user information
    userDao.save(user);
    passCertDao.updateFlag(user.getPassId());
    // This method is a remote interface call
    sendEmailRpc(user.getEmail());
    return user.getUserId();
}

Use TransactionTemplate for orchestrated transactions.

@Resource
private TransactionTemplate transactionTemplate;

public int createUser(User user){
    transactionTemplate.execute(transactionStatus -> {
      try {
         userDao.save(user);
         passCertDao.updateFlag(user.getPassId());
      } catch (Exception e) {
         // Exception manual setup rollback
         transactionStatus.setRollbackOnly();
      }
      return true;
    });
    // This method is a remote interface call
    sendEmailRpc(user.getEmail());
    return user.getUserId();
}

Finally, if the article was helpful, please clap 👏and follow, thank you! ╰(*°▽°*)╯

I’m Dylan, looking forward to progressing with you. ❤️

Background

Imagine a real scenario where when the order system calls the inventory deduction method, an interface timeout occurs and the call fails.

However, the order system cannot determine whether the inventory has actually been deducted successfully although the previous call returned a failure.

In this case, if a retry is initiated by default, and if there is no idempotency control for inventory deduction, then the same order may have its inventory deducted twice, ultimately leading to a situation of underselling. Therefore, it is necessary to implement idempotency control for inventory deduction.

What is idempotency?

Idempotency is a concept in mathematics and computer science.

• In mathematics, idempotency can be expressed by the function expression: f(x) = f(f(x)). For example, the function for finding the absolute value is idempotent, that is, abs(x) = abs(abs(x)).

• In computer science, idempotency means that one and multiple requests for a resource should have the same side effects. In other words, the impact of multiple requests is the same as the impact of a single request execution.

According to the definition on Wikipedia, idempotency (idempotent, idempotence) is a concept in mathematics and computer science. In mathematics, idempotency can be expressed by the function expression: f(x) = f(f(x)). For example, the function for finding the absolute value is idempotent, that is, abs(x) = abs(abs(x)).

In computer science, the characteristic of an idempotent operation is that the impact of any number of executions is the same as the impact of a single execution. An idempotent function or method refers to a function that can be executed repeatedly with the same parameters and obtain the same result. In our daily programming, read operations are naturally idempotent, and only write operations need idempotency control.

Scenarios requiring idempotency control

The just-mentioned inventory deduction scenario is just one of them. Let’s take a look at all the scenarios that I have encountered in my work that require idempotency control.

1. Interface call retry

This may be the timeout retry scenario just mentioned. If the call times out but actually has a successful impact, then a success can be returned directly after the second request. It may also not be due to timeout but due to bugs in other code. Therefore, for relatively important write operations, it is best to add idempotency control.

For cases where the call times out but is indeed a business failure, repeated execution will usually still fail, and generally there is no need for idempotency control for failures. For example, if it was insufficient inventory before the retry, and the inventory has been replenished during the retry, then it is possible to succeed. In this scenario, it is not recommended to directly use the saved idempotent result. It is better to execute the business logic again. Of course, if there are special requirements from the business side, it is an exception.

2. Duplicate consumption by MQ consumer groups

For the logic of consuming a message, since messages may be read repeatedly, message consumption needs idempotency control. The way of idempotency control can be selected according to the business scenario and will be introduced later.

3. Front-end duplicate submissions

For example, in the order placement operation here, if the submit button is clicked quickly, two orders may be created instantaneously. Of course, control may not be necessary here, depending on product requirements. After all, orders can be canceled. However, most form duplicate submissions still need to be controlled.

How is idempotency designed?

The core of idempotency is to control the impact of the same request. No matter what scheme is adopted, first of all, a unique ID is needed to identify that this request is unique.

1. Global unique ID

How do we obtain a global unique ID?

We can use UUID, but the disadvantages of UUID are relatively obvious. Its string takes up a large space, the generated ID is too random, has poor readability, and there is no incrementality.

We can also use the Snowflake algorithm to generate a unique ID.

The Snowflake algorithm is an algorithm for generating globally unique IDs in a distributed environment. The generated IDs are called Snowflake IDs. This algorithm was created by Twitter and is used for tweet IDs.

A Snowflake ID has 64 bits.

• The first bit: In Java, the highest bit of a long is the sign bit, representing positive and negative. Positive numbers are 0 and negative numbers are 1. Generally, the generated IDs are all positive, so it defaults to 0.

• The next 41 bits are timestamps, representing the number of milliseconds since a selected period.

• The next 10 bits represent the computer ID to prevent conflicts.

• The remaining 12 bits represent the sequence number for generating IDs on each machine, allowing multiple Snowflake IDs to be created within the same millisecond.

Several solutions for implementing idempotency

1. select + insert + unique index conflict

Taking the scenario of deducting inventory as an example, we have a table that records successful inventory deduction records. There are mainly three fields: order number, inventory ID, and the number of deductions.

In this scenario, the order number orderSn is our unique ID. When a request comes, first select whether there is a deduction record for this order. Then there are three situations:

• If the deduction record already exists, intercept the request and directly return success.

• If the deduction record does not exist, execute insert. If insert is successful, return success normally.

• If the deduction record does not exist, execute insert. If insert fails, catch and see if it is a DuplicateKeyException. If so, it means that the interval between multiple retry requests is too short and simultaneously bypasses the data judgment of select.

Pseudo code is as follows:

public boolean deduct(String orderSn, Long inventoryId, Integer deductCount) {
    // Select a record by order serial number.
    Record record = selectByOrderSn(orderSn);

    if (record!= null) {
        // Duplicate request. Return success.
        return true;
    }

    try {
        // Insert data.
        insert(orderSn, inventoryId, deductCount);
    } catch (DuplicateKeyException e) {
        // Unique key conflict. Duplicate request. Return success.
        return true;
    }

    // Normal processing logic... May return false if there is an error, such as insufficient inventory.

    return true;
}

Generally speaking, the insert and the subsequent normal processing logic need to be included in the same transaction. This is because if the first request successfully inserts the deduction record and then an error may occur later. If the previous insert is not rolled back, all requests after the second one will be directly intercepted and return true.

In addition, there may be a scenario where the inventory is insufficient and the deduction fails. For such a business failure, suppose the caller requires idempotency as well. That is, if the same parameters are passed next time and it also returns failure, then it also needs to be recorded in the deduction record table. At this time, the record table needs to add a status field and an extInfo to record the specific error situation, and then the error details can be directly returned.

2. insert + unique index conflict

The difference between this solution and the previous one is that there is no need for the initial select query. The rest of the logic is the same. It is used in cases where the probability of duplicate requests is relatively low.

3. Status check + update row lock

Many business scenarios have states. After a series of business logics are successfully performed, they will flow to the next state. For example, when placing an order, if a coupon is attached, the status of the coupon will be marked as “occupied state”. Generally, there are the following states.

The SQL for occupying a coupon when placing an order can be written like this (generally, it will also record which order and which sku occupies it):

update coupon_instance set status = 2 where coupon_id = ‘12315’ and status = 1;

The pseudo code implementation is as follows:

void holdCoupon(Request request) {
    String couponId = request.getCouponId();
    int rows = “update coupon_instance set status = 2 where couponId = #{couponId} and status = 1;”;
    if (rows == 0) {
        // No processing. Return directly.
        return;
    }
    if (rows > 1) {
        // Abnormal situation. Raise an alarm.
        throw new IllegalStateException();
    }
    // rows == 1 is the normal situation. Process other business logic, such as deducting and verifying inventory, disabling certain coupons after occupying the coupon, etc...
}

When the first request for couponId 12135 arrives, the status of this coupon is “unused” and needs to be updated to “occupied state”. After the update statement is executed, the number of affected rows returned is 1, and the subsequent process is executed normally.

When the second request for the same coupon comes, the status of this coupon is already “occupied state” and needs to be updated to “occupied state”. After the update statement is executed, the number of affected rows returned is 0, and it is directly returned.

In addition, the verification method here has a disadvantage. If not only this method will modify the status field, for example, after the coupon expires, there is a task that modifies the status to 3 - expired. At this time, because row == 0 is obtained after the update, so a success is returned. Generally, occupying coupons is called by the order business. If a coupon is expired and a success is still returned to it, then the order placement will continue, which is bound to cause some problems.

So in this scenario, it is more appropriate to report an error after row == 0. However, if it is changed to report an error, in fact, this method is not idempotent anymore. It only does duplicate prevention control. Because the first call returns success and the second call throws an exception. Duplicate prevention is mainly to avoid generating duplicate data. Just intercepting duplicate requests is fine. In addition to intercepting processed requests, idempotent design also requires that the same request returns the same result. For the case of repeated consumption of messages, I think an error-reporting implementation can be accepted. First, because there will not be too many duplicate messages. Second, after an error is reported, the message will enter the dead letter queue and can be discarded.

Of course, not only status can be processed in this way, other fields can also be processed. It’s just that status is more representative.

4. Idempotency control table (recommended)

In solutions 1 and 2, generally a separate record table needs to be implemented for idempotency. In fact, in a project, there are usually more than one or two points that need idempotency control. So often it is hoped that the idempotency control operation can be separated from the business. Let’s take a look at the specific design.

First, create an idempotency control table.

CREATE TABLE `idempotent` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT ‘Auto-increment primary key’,
  `biz_id` varchar(128) NOT NULL COMMENT ‘External request id, unique identifier, can be order_sn, uuid, etc.’,
  `biz_type` varchar(32) NOT NULL COMMENT ‘Business type’,
  `request` varchar(1024) NOT NULL COMMENT ‘Backup of request content’,
  `response` varchar(1024) DEFAULT NULL COMMENT ‘Backup of response content. If status is success, it represents a backup of normal return results. If it is failure, it represents a backup of abnormal content.’,
  `status` int(10) NOT NULL COMMENT ‘Status. 0 - initialization, 1 - success, 2 - failure’,
  PRIMARY KEY (`id`),
  UNIQUE KEY `biz_id_type` (`biz_id`,`biz_type`)
) ENGINE=InnoDB COMMENT=’Idempotency table’;

To enable multiple businesses to be stored in one table, add a biz_type. In this way, if biz_id is the order number, through a combined unique index, idempotency for multiple scenarios can be saved.

Taking inventory deduction as an example, the core implementation code is as follows:

public Response deduct(Request request) {
    // First, check if it has been processed before. If it has been processed successfully before, return the result directly.
    // request.getOrderSn() is bizId, and “deduct” is bizType.
    Idempotent idempotent = idempotentDAO.getIdempotent(request.getOrderSn(), “deduct”);
    if (idempotent!= null) {
        return JSON.parseObject(idempotent.getResponse(), new TypeReference<Response>() {});
    }
    idempotentDAO.insert(request.getOrderSn(), “deduct”, request);

    // Execute business code...
    Response response = new Response();

    // After the business code is executed successfully, update the record.
    idempotentDAO.setStatus(WorkOrderStatusEnum.SUCCESS.getCode());
    idempotentDAO.setResponse(JSON.toJSONString(response));
    idempotentDAO.update(idempotentDAO);
    return response;
}

Observing the above code, it is not difficult to find that this logic is actually relatively fixed and similar to template code. If each piece of business code that requires idempotency control needs to add such before and after logic, it will lead to excessive duplicate code in the project and affect code readability. Is there any way to optimize it?

In fact, it can be implemented by using aspects combined with annotations. The principle is to use dynamic proxy to dynamically generate a subclass to control access to the real object. This will not be discussed in detail here.

5. Idempotency control table + implemented by other methods

The method of using MySQL to implement an idempotency control table has a disadvantage that when the data volume reaches a certain level, it will affect the performance of the actual interface.

Currently, considering that Redis, HBase, etc. can be used for implementation, but I have not actually operated it. After exploration, I will come back and supplement…😅

Okay, that’s all for today. If this article is helpful to you, I hope you can give it a thumbs up and follow. This is very important to me.

Share The Java Knowledge Ocean

Real-time message push is ubiquitous in modern applications, from real-time sales data display on e-commerce websites, to instant message notifications on social platforms, and real-time interactions in games. Real-time message push technology permeates our digital lives.

Today, let’s dive deep into the secrets of real-time message push and explore which solution, from polling to MQTT, is your best choice.

​1. Polling — Short Polling

Short polling is like calling your friend every few minutes to ask, “Any news yet?” Whether they have something or not, you call anyway. The frontend uses a scheduled task (setInterval) to send requests to the backend at regular intervals. If the backend has data, it returns it; otherwise, it returns nothing.

Frontend Code Example

// Execute every 3 seconds
setInterval(function() {
  // Send a request to ‘/get-data’
  fetch(’/get-data’)
    .then(response => response.json()) // Parse the response as JSON
    .then(data => {
      if (data) {
        console.log(’New message received’, data); // If data exists, log it to the console
      }
    });
}, 3000); // Time interval is 3000 milliseconds (3 seconds)

Backend Code Example

@RestController
public class ShortPollingController {
    @GetMapping(”/get-data”)
    public ResponseEntity<Map<String, Object>> getData() {
        Map<String, Object> result = new HashMap<>();
        // Simulate data retrieval logic
        boolean hasData = new Random().nextBoolean();
        if (hasData) {
            result.put(”message”, “This is a new message”);
            return ResponseEntity.ok(result);
        } else {
            return ResponseEntity.noContent().build();
        }
    }
}

The issue with short polling is obvious: the frontend frequently sends requests, but only a few or even none may be valid. This wastes server resources, and with high request volumes, the server faces significant pressure.

​2. Polling — Long Polling

Long polling is smarter. After the frontend sends a request, the backend doesn’t respond immediately but holds the request until new data is available. If no data arrives within a timeout, the backend responds, and the frontend sends a new request.

Frontend Code Example

function longPolling() {
  // Send a request to ‘/long-polling’
  fetch(’/long-polling’)
    .then(response => response.json()) // Parse the response as JSON
    .then(data => {
      console.log(’Long polling received message’, data); // Log the received message
      // After receiving the message, initiate the next polling
      longPolling();
    });
}

// Start the long

Backend Code Example

@RestController
public class LongPollingController {
    private volatile Map<String, Object> data = new HashMap<>();

    @GetMapping(”/long-polling”)
    public ResponseEntity<Map<String, Object>> longPolling() {
        // Simulate waiting for new messages
        synchronized (this) {
            while (data.isEmpty()) {
                try {
                    wait();
                } catch (InterruptedException e) {
                    Thread.currentThread().interrupt();
                }
            }
        }
        ResponseEntity<Map<String, Object>> response = ResponseEntity.ok(data);
        data = new HashMap<>(); // Clear the data
        return response;
    }

    // Simulate a method to receive new messages
    @PostMapping(”/send-data”)
    public ResponseEntity<String> sendData(@RequestBody Map<String, Object> newData) {
        synchronized (this) {
            data = newData;
            this.notifyAll();
        }
        return ResponseEntity.ok(”Message sent successfully”);
    }
}

Long polling reduces the number of requests but comes with its own set of issues. If the backend has no new data, the request will time out; under high concurrency, a large number of threads accumulate, putting immense pressure on the server.

​3. SSE Data Push Solution

SSE (Server-Sent Events) is a one-way communication protocol based on HTTP. The frontend initiates a request, and the backend establishes a long-lived connection. Once the client receives it, a persistent connection is established, allowing the server to push data to the client in real-time when changes occur. This is ideal for scenarios requiring frequent data updates and low latency, such as online chat, real-time monitoring, and news feeds.

Frontend Code Example

// Create an EventSource connection to ‘/sse-connect’
const eventSource = new EventSource(’/sse-connect’);

// Handle incoming messages
eventSource.onmessage = function(event) {
  console.log(’Message received:’, event.data);
};

// Handle connection errors
eventSource.onerror = function(error) {
  console.error(’Connection error:’, error);
};

// Handle successful connection
eventSource.onopen = function() {
  console.log(’Connection established’);
};

Backend Code Example

@RestController
public class SseController {

    private final Map<String, SseEmitter> clients = new ConcurrentHashMap<>();

    @GetMapping(value = “/sse-connect”, produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public SseEmitter connect() {
        SseEmitter sseEmitter = new SseEmitter();
        String clientId = UUID.randomUUID().toString();

        // Set timeout
        sseEmitter.timeout(0L);

        // Send initial message
        try {
            sseEmitter.send(SseEmitter.event()
                    .data(”Connection successful”)
                    .comment(”Connection established”));
        } catch (IOException e) {
            e.printStackTrace();
        }

        // Completion callback
        sseEmitter.onCompletion(() -> {
            System.out.println(”Client “ + clientId + “ disconnected”);
            clients.remove(clientId);
        });

        // Timeout callback
        sseEmitter.onTimeout(() -> {
            System.out.println(”Client “ + clientId + “ connection timed out”);
            clients.remove(clientId);
        });

        // Error callback
        sseEmitter.onError(throwable -> {
            System.out.println(”Client “ + clientId + “ connection error”);
            clients.remove(clientId);
        });

        clients.put(clientId, sseEmitter);
        return sseEmitter;
    }

    // Method to push messages

SSE is a solid choice, but it has its limitations, such as being unidirectional — data can only be pushed from the server to the client, not the other way around.

​4. WebSocket Solution

WebSocket is a game-changer. It establishes a persistent, bidirectional channel, allowing both the client and server to send messages to each other at any time. This makes it ideal for scenarios requiring real-time, two-way communication, such as online chat or multiplayer gaming.

Frontend Code Example

// Create a WebSocket connection to ‘ws://localhost:8080/ws-connect’
const ws = new WebSocket(’ws://localhost:8080/ws-connect’);

// Handle connection open event
ws.onopen = function() {
    console.log(’Connection established’);
    ws.send(’Client connected’);
};

// Handle incoming messages
ws.onmessage = function(event) {
    console.log(’Message received from server:’, event.data);
};

// Handle connection errors
ws.onerror = function(error) {
    console.error(’Connection error:’, error);
};

// Handle connection close event
ws.onclose = function() {
    console.log(’Connection closed’);
};

Backend Code Example

@ServerEndpoint(”/ws-connect”)
@Component
public class WebSocketServer {

    private static final ConcurrentHashMap<String, Session> clients = new ConcurrentHashMap<>();

    @OnOpen
    public void onOpen(Session session, @PathParam(”userId”) String userId) {
        System.out.println(”Client “ + userId + “ is establishing a connection”);
        clients.put(userId, session);
    }

    @OnMessage
    public void onMessage(String message, @PathParam(”userId”) String userId) {
        System.out.println(”Client “ + userId + “ sent a message to the server: “ + message);
        // Broadcast the message to all clients
        clients.forEach((id, session) -> {
            try {
                session.getBasicRemote().sendText(”Client “ + userId + “ says: “ + message);
            } catch (IOException e) {
                e.printStackTrace();
            }
        });
    }

    @OnClose
    public void onClose(@PathParam(”userId”) String userId) {
        System.out.println(”Client “ + userId + “ closed the connection”);
        clients.remove(userId);
    }

    @OnError
    public void onError(Session session, Throwable throwable) {
        System.out.println(”An error occurred in the client connection”);
        throwable.printStackTrace();
    }
}

WebSocket performs well but has its pitfalls, such as a relatively complex connection establishment process and higher resource consumption on the server.

​5. Netty Real-Time Push Solution

Netty is a high-performance network application framework. Push solutions implemented with Netty are well-suited for scenarios demanding high concurrency and real-time performance, such as large-scale game servers and financial trading systems.

Code Example

public class NettyServer {

    public static void main(String[] args) throws InterruptedException {
        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workerGroup = new NioEventLoopGroup();
        try {
            ServerBootstrap bootstrap = new ServerBootstrap()
                    .group(bossGroup, workerGroup)
                    .channel(NioServerSocketChannel.class)
                    .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        protected void initChannel(SocketChannel ch) {
                            ch.pipeline().addLast(new NettyServerHandler());
                        }
                    });
            ChannelFuture future = bootstrap.bind(8080).sync();
            System.out.println(”Netty server started, listening on port 8080”);
            future.channel().closeFuture().sync();
        } finally {
            bossGroup.shutdownGracefully();
            workerGroup.shutdownGracefully();
        }
    }
}

// Netty server handler class
public class NettyServerHandler extends ChannelInboundHandlerAdapter {

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) {
        try {
            String message = (String) msg;
            System.out.println(”Received message from client: “ + message);
            // Broadcast the message to all clients
            ctx.channel().group().writeAndFlush(”Server received message: “ + message + “\n”);
        } finally {
            ReferenceCountUtil.release(msg);
        }
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
        cause.printStackTrace();
        ctx.close();
    }
}

Netty is powerful in performance but has a steep learning curve. To fully leverage its advantages, a deep understanding of its principles and APIs is required.

​6. MQTT for Real-Time Message Push

MQTT is a lightweight messaging protocol, ideal for resource-constrained devices and unstable network environments, such as IoT devices.

Code Example

public class MqttExample {

    public static void main(String[] args) {
        // MQTT broker URL
        String brokerUrl = “tcp://localhost:1883”;
        // Client ID
        String clientId = “JavaClient”;
        MqttClient client;
        try {
            client = new MqttClient(brokerUrl, clientId);
            // Connection options
            MqttConnectOptions options = new MqttConnectOptions();
            options.setCleanSession(true);
            // Connect to the broker
            client.connect(options);
            // Subscribe to a topic
            client.subscribe(”test/topic”);
            // Set callback
            client.setCallback(new MqttCallback() {
                @Override
                public void connectionLost(Throwable cause) {
                    System.out.println(”Connection lost”);
                }

                @Override
                public void messageArrived(String topic, MqttMessage message) {
                    System.out.println(”Message received: “ + new String(message.getPayload()));
                }

                @Override
                public void deliveryComplete(IMqttDeliveryToken token) {
                    System.out.println(”Message delivery complete”);
                }
            });
            // Publish a message
            MqttMessage msg = new MqttMessage(”Hello MQTT”.getBytes());
            msg.setQos(1);
            client.publish(”test/topic”, msg);
        } catch (MqttException e) {
            e.printStackTrace();
        }
    }
}

MQTT’s QoS mechanism ensures message reliability but also introduces some additional complexity and latency.

Summary

In summary, real-time message push solutions each have their strengths. For simple use cases with low request volumes, polling can suffice; for complex scenarios with high request loads, WebSocket and Netty are excellent choices; and for IoT applications, MQTT is the go-to option. Choose the right solution, and your application will navigate the world of real-time message push with ease!

Finally, if the article was helpful, please clap 👏and follow, thank you! ╰(*°▽°*)╯

I’m Dylan, looking forward to progressing with you. ❤️

Share The Java Knowledge Ocean

Preface

When it comes to AOP (Aspect-Oriented Programming), people often first think: “Oh, that’s used for logging, transaction management, or permission verification.”

In fact, AOP’s capabilities go far beyond that. It can also work wonders for interface self-protection in high-concurrency scenarios.

Recently, I encountered a real-world scenario in a project: it’s a scheduled batch task triggered by MQ. On normal days, everything runs smoothly, but during flash sales or when data volume surges, the backlogged messages in MQ are instantly pushed to consumers.

Although the consumer service can handle the load, the underlying core business database cannot — massive concurrent queries immediately max out the CPU (usage spikes to 100%), directly affecting the stability of online real-time services.

Considering the service is deployed as a single node, introducing Redis for distributed rate limiting would be overkill and add extra operational and maintenance costs. Eventually, I decided to use Spring AOP + Guava RateLimiter + custom annotations to implement a non-intrusive, configurable, and lightweight standalone rate limiting component.

I. Why Choose AOP + Annotations?

Before introducing the code, let’s clarify the design philosophy.

When I first started developing, I also liked to hardcode rate limiting logic directly in the Service or Controller layer, for example:

// ❌ Anti-pattern: Hardcoded, mixed logic, and poor reusability
if (!rateLimiter.tryAcquire()) {
    throw new RuntimeException(”System is busy”);
}
doBusiness();

The drawbacks of this approach are obvious:

• Mixed logic: Non-business rate limiting judgments are embedded in clear business code.

• Poor reusability: If ten interfaces need rate limiting, you have to write the same code ten times.

• Difficult maintenance: If you need to adjust the rate limiting strategy (e.g., upgrade to distributed rate limiting), there will be numerous modification points.

The core of AOP (Aspect-Oriented Programming) is “decoupling” and “reusability”.

I encapsulate the rate limiting logic into an independent “aspect” and use custom annotations as “switches”. Simply add an annotation to the target method, and the rate limiting strategy takes effect immediately. For subsequent maintenance and upgrades, you only need to focus on the aspect logic itself without touching any business code.

II. Core Principles of Guava RateLimiter

The core library I chose this time is Google Guava’s RateLimiter, which is implemented based on the Token Bucket algorithm.

1. Brief Review of Token Bucket

Unlike the rigid “leaky bucket” (constant rate outflow), its mechanism is more user-friendly:

• Token Production: The system puts tokens into the bucket at a fixed rate.

• Token Consumption: When a request arrives, it must obtain a token before execution.

• Key Feature: Supports burst traffic. If there are no requests for a period of time, tokens accumulate in the bucket (until reaching the bucket limit). When a burst of traffic arrives, the accumulated tokens can be consumed immediately without queuing.

2. Two Core Modes

Guava thoughtfully provides two implementations:

• SmoothBursty (Smooth Burst): Default mode. Suitable for most scenarios, allowing short-term traffic bursts.

• SmoothWarmingUp (Smooth Warming Up): Warming-up mode. The token issuance rate is slow initially and gradually increases to the target QPS over time. This is very friendly for resources that need “warm-up” (such as database connection pools, cache population), preventing them from being overwhelmed during cold starts.

3. Standalone Warning ⚠️

Note: Guava RateLimiter is a standalone rate limiting tool! Tokens are stored in the current JVM memory.

• If your service is deployed on a single machine, it works perfectly.

• If you deploy 10 machines, each with QPS=5, the total QPS limit of the entire cluster is 50.

4. Detailed Explanation of Common APIs

Mastering APIs proficiently is the foundation of practical application. Here are the core methods of RateLimiter:

Core Creation Methods

Method SignatureDescriptioncreate(double permitsPerSecond)Creates a SmoothBursty rate limiter, specifying the number of tokens generated per second (default: permitsPerSecond = QPS = bucket capacity).create(double permitsPerSecond, long warmupPeriod, TimeUnit unit)Creates a SmoothWarmingUp rate limiter, specifying QPS + warm-up time.

Core Acquisition Methods

Method SignatureDescriptiondouble acquire()Blocking acquisition of 1 token. If no token is available, the thread waits indefinitely until successful acquisition.double acquire(int permits)Blocking acquisition of a specified number of tokens (multiple tokens can be acquired at once).

boolean tryAcquire()Non-blocking acquisition of 1 token. Returns immediately: true for success, false for failure (no waiting).boolean tryAcquire(long timeout, TimeUnit unit)Time-limited waiting for 1 token. Returns true if acquired within the timeout period, otherwise false. This is the most recommended usage—it avoids infinite thread waiting while providing a certain buffer.

III. Code Practice: Building an Enterprise-Grade Rate Limiting Component

Next, I will implement a fully functional @RateLimit component that supports QPS configuration, blocking/non-blocking modes, timeout control, and warming-up mode.

1. Add Dependencies

<dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava</artifactId>
    <version>32.1.3-jre</version>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-aop</artifactId>
</dependency>

2. Define the @RateLimit Annotation

This annotation carries all configuration metadata for rate limiting.

import java.lang.annotation.*;
import java.util.concurrent.TimeUnit;

@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface RateLimit {
    /**
     * Rate limiting threshold (QPS), default 5 per second
     */
    double qps() default 5.0;
    /**
     * Token acquisition strategy
     * true: Blocking mode (wait until token is acquired or timeout)
     * false: Non-blocking mode (fail immediately if token is not available)
     */
    boolean block() default true;
    /**
     * Timeout for blocking wait (only effective when block=true)
     * Default 0, indicating infinite wait
     */
    long timeout() default 0;
    /**
     * Timeout unit
     */
    TimeUnit timeUnit() default TimeUnit.MILLISECONDS;
    /**
     * Warm-up time
     * Default 0 (SmoothBursty); set >0 to enable warming-up mode (SmoothWarmingUp)
     */
    long warmupPeriod() default 0;
    /**
     * Warm-up time unit
     */
    TimeUnit warmupUnit() default TimeUnit.SECONDS;
    /**
     * Rate limiting prompt message
     */
    String message() default “System is busy, please try again later”;
}

3. Define Global Exception RateLimitException

public class RateLimitException extends RuntimeException {
    public RateLimitException(String message) {
        super(message);
    }
}

4. Implement the Aspect RateLimitAop

This is the “brain” of the rate limiting component. Special attention should be paid to instance caching, thread safety, and the execution logic of different strategies.

import com.google.common.util.concurrent.RateLimiter;
import lombok.extern.slf4j.Slf4j;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.springframework.stereotype.Component;


import java.lang.reflect.Method;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
@Slf4j
@Aspect
@Component
public class RateLimitAop {
    
    // Use ConcurrentHashMap to cache RateLimiter instances for thread safety
    // Key: Method signature (ClassName.MethodName(ParamType1,ParamType2)), Value: Rate limiter instance
    private final Map<String, RateLimiter> rateLimiterCache = new ConcurrentHashMap<>();
    @Pointcut(”@annotation(com.example.annotation.RateLimit)”)
    public void rateLimitPointcut() {}
    @Around(”rateLimitPointcut()”)
    public Object around(ProceedingJoinPoint joinPoint) throws Throwable {
        MethodSignature signature = (MethodSignature) joinPoint.getSignature();
        Method method = signature.getMethod();
        RateLimit annotation = method.getAnnotation(RateLimit.class);
        // 1. Build a unique method key to prevent method overloading conflicts
        String methodKey = buildMethodKey(method);
        
        // 2. Thread-safely create or get the rate limiter
        RateLimiter rateLimiter = rateLimiterCache.computeIfAbsent(methodKey, key -> createRateLimiter(annotation));
        // 3. Execute token acquisition logic
        boolean acquireSuccess;
        if (annotation.block()) {
            // --- Blocking mode ---
            if (annotation.timeout() <= 0) {
                // Infinite wait until success
                rateLimiter.acquire();
                acquireSuccess = true;
            } else {
                // Time-limited wait
                acquireSuccess = rateLimiter.tryAcquire(annotation.timeout(), annotation.timeUnit());
            }
        } else {
            // --- Non-blocking mode ---
            // Try immediately, return on failure
            acquireSuccess = rateLimiter.tryAcquire();
        }
        // 4. Rate limiting interception
        if (!acquireSuccess) {
            log.warn(”[Rate Limiting Alert] Method {} has too high request frequency and is rejected.”, methodKey);
            throw new RateLimitException(annotation.message());
        }
        // 5. Release
        return joinPoint.proceed();
    }
    /**
     * Generate method signature: Package.Class.Method(ParamType1,ParamType2)
     */
    private String buildMethodKey(Method method) {
        StringBuilder keyBuilder = new StringBuilder();
        keyBuilder.append(method.getDeclaringClass().getName())
                .append(”.”).append(method.getName()).append(”(”);
        Class<?>[] parameterTypes = method.getParameterTypes();
        for (int i = 0; i < parameterTypes.length; i++) {
            keyBuilder.append(parameterTypes[i].getSimpleName());
            if (i < parameterTypes.length - 1) {
                keyBuilder.append(”,”);
            }
        }
        keyBuilder.append(”)”);
        return keyBuilder.toString();
    }
    /**
     * Factory method: Create a specific RateLimiter based on configuration
     */
    private RateLimiter createRateLimiter(RateLimit annotation) {
        if (annotation.warmupPeriod() > 0) {
            log.info(”Creating warming-up rate limiter: QPS={}, Warmup={}s”, annotation.qps(), annotation.warmupPeriod());
            return RateLimiter.create(annotation.qps(), annotation.warmupPeriod(), annotation.warmupUnit());
        } else {
            log.info(”Creating standard rate limiter: QPS={}”, annotation.qps());
            return RateLimiter.create(annotation.qps());
        }
    }
}

5. Business Integration Example

@Service
public class DataSyncService {

    // Scenario 1: Core data synchronization, allow queuing for 500ms to ensure execution as much as possible
    @RateLimit(qps = 10.0, block = true, timeout = 500)
    public void syncImportantData(List<Data> dataList) {
        // ... Business logic ...
    }
    // Scenario 2: Non-core interface, directly discard requests when traffic is high to protect the system
    @RateLimit(qps = 50.0, block = false, message = “Too many visitors currently”)
    public void refreshCache() {
        // ... Refresh logic ...
    }
}

IV. Advanced: Talking About the “Well-Known” Pitfall of Dynamic Proxies

When using AOP, there’s a classic interview question scenario: AOP failure caused by internal method calls within a class. As developers, we should not only know the phenomenon but also understand the reason.

Scenario Reproduction

@Service
public class TradeService {
    public void process() {
        // ... Preprocessing ...
        pay(); // ❌ Key point here: Directly call internal method
    }


    @RateLimit(qps = 5.0) 
    public void pay() { ... }
}

Why Does It Fail?

Spring AOP is implemented based on dynamic proxies.

• When the container starts, Spring generates a proxy object for TradeService.

• When process() is called externally, it goes through the proxy first.

• However, when pay() is executed inside process(), this.pay() is used. Here, this refers to the target object itself, not the proxy object.

• Since it doesn’t go through the proxy, the aspect logic is naturally bypassed.

Pitfall Avoidance Suggestions

For such issues, I recommend the following solutions:

Recommended: Split Method (Best Practice)

Split the pay() method into another independent Bean (e.g., PayService). Call it through injection, which naturally complies with the “call through proxy” rule and makes the code structure clearer.

Recommended: AopContext

Directly retrieve the current proxy object from the Spring context. (For modifying old functions)

• Enable configuration on the SpringBoot startup class: @EnableAspectJAutoProxy(exposeProxy = true)

• Modify in specific code: ((TradeService) AopContext.currentProxy()).pay();

Not Recommended: @Autowired Inject Self

Although it can solve the problem, it is prone to circular dependency exceptions and increases system startup risks.

V. Advanced Thinking: From Standalone to Distributed

Earlier, I emphasized that Guava RateLimiter is for standalone rate limiting. So, if the system scales up to 50 nodes and you need to implement global rate limiting of 1000 requests per second for a downstream API, what should you do?

At this time, the power of the AOP + annotation design pattern is reflected.

You don’t need to modify any business code or delete the @RateLimit annotation at all.

You only need to do one thing: Modify the implementation of the RateLimitAop aspect.

Replace the token acquisition logic in the aspect from Guava RateLimiter to Redis + Lua scripts, or directly integrate Redisson’s RRateLimiter.

// Pseudocode example: Seamless switch to distributed rate limiting
private RRateLimiter getRedisLimiter(String key) {
    RRateLimiter limiter = redissonClient.getRateLimiter(key);
    // ... Initialize Redis rate limiter ...
    return limiter;
}

// In the around method, replace RateLimiter.tryAcquire() with Redisson’s implementation
RRateLimiter limiter = getRedisLimiter(methodKey);
if (!limiter.tryAcquire(annotation.qps(), annotation.timeout(), annotation.timeUnit())) {
    throw new RateLimitException(”Distributed rate limiting is in effect...”);
}

This is the art of architectural design. The business side is unaware, and the underlying capabilities are upgraded smoothly.

VI. Summary and Conclusion

Overall, AOP seamlessly integrates infrastructure like rate limiting into business logic — this is the charm of an elegant architecture: converging complexity at one point to achieve simplicity elsewhere.

Finally, a frequently adapted famous quote feels particularly fitting here: “Let architecture handle architecture, and business handle business.”

May your code world be well-organized and free of bugs.

Share The Java Knowledge Ocean

Let’s Start with a Quick Rant

Ever notice how your phone buzzes nonstop all day? Order shipped, someone replied to your comment, that app you haven’t opened in three days desperately wants your attention again.

Sounds simple enough, right? Just send a message. How hard can it be?

Now try doing it for one billion users, sending billions of notifications per day, peaking at hundreds of thousands per second — and oh by the way, verification codes absolutely must arrive within three seconds. Still think it’s easy?

Today we’ll start from the most naive approach and build it up layer by layer, talking about what breaks at each stage. Useful for interviews, and even more useful for not repeating my mistakes in production.

Version 1: Database + Cron Job (Baby’s First Notification System)

Full transparency — I actually shipped something like this early in my career. A notifications table with user_id, title, content, status, and a cron job that runs every minute:

Scan table → Grab pending rows → Call push API → Update status

Brutally simple. Ten minutes to code, works great for a small team.

Here’s the Pseudocode

# Runs every minute via cron
def cron_send_notifications():
    pending = db.query(”SELECT * FROM notifications WHERE status = ‘pending’ LIMIT 1000”)
    
    for msg in pending:
        try:
            push_gateway.send(msg.user_id, msg.title, msg.content)
            db.update(msg.id, status=’sent’)
        except Exception as e:
            db.update(msg.id, status=’failed’, error=str(e))

And Then Everything Breaks

Problem #1: It’s slow. As data grows, scanning the table gets worse and worse. Finding 1,000 pending rows among 10 million records — even with good indexes, doing this every minute destroys your DB.

Problem #2: Single point of failure. One cron job on one server. If that box dies, notifications stop entirely. No redundancy, no fallback.

Problem #3: No rate control. Marketing dumps a million rows in for a campaign, cron goes brrr, and your push gateway gets DDoS’d by your own system.

Problem #4: Terrible latency. If it runs every minute, a user might wait almost 60 seconds for a verification code. By then they’ve already hit “resend” five times.

Verdict: Fine for 10k users. Beyond that? Disaster.

Version 2: Add a Message Queue (Now We’re Cooking)

Anyone with a bit of experience will say — just use Kafka!

And yeah, that’s exactly what message queues are for. Producers drop notification tasks in, consumers chew through them at their own pace. Decoupled, buffered, scalable.

Architecture Diagram

Consumer Pseudocode

def notification_worker():
    consumer = KafkaConsumer(topic=’notifications’, group_id=’push-group’)
    
    for message in consumer:
        msg = parse(message.value)
        
        try:
            push_gateway.send(msg.user_id, msg.title, msg.content)
            update_status(msg.id, ‘sent’)
            consumer.commit()  # Commit offset
        except Exception as e:
            update_status(msg.id, ‘failed’, error=str(e))
            # Failures go to retry topic
            retry_topic.produce(msg)

What This Fixes

✅ Decoupling — producers don’t care when things get sent, they just fire and forget
✅ Horizontal scaling — too slow? Add more workers. Done.
✅ Peak shaving — traffic spikes buffer in the queue, downstream stays healthy

But New Problems Emerge

Problem #1: Too many channels. Push isn’t just app push. There’s APNs for iOS, FCM for Android, SMS, email, WeChat templates, in-app messages… every single one has different APIs, rate limits, retry logic. Throwing them all in one worker pool is chaos.

Problem #2: No prioritization. Verification codes and Black Friday marketing blasts share the same queue. When a marketing flood hits, your OTPs get stuck behind for half an hour. That’s a P0 incident waiting to happen.

Problem #3: User preferences are nowhere. Some people hate being woken up at night, some opted out of marketing entirely. We have nowhere to store these rules, nowhere to enforce them.

Problem #4: Duplicate deliveries. Kafka has at-least-once delivery semantics. A worker crashes mid-processing, restarts, consumes again — user gets the same notification twice. Bad UX.

Version 3: Layered Architecture (Starting to Look Like a Real System)

Okay, let’s fix each of those problems one by one.

Upgraded Architecture Diagram

Upgrade #1: Priority Queues

Straightforward — split into three topics by importance:

Resource isolation. Low priority traffic can blow up all it wants — it won’t touch the high priority stuff. Simple but brutally effective.

Upgrade #2: Channel Adapter Pattern

Each channel handles its own rate limiting, retries, and auth — exposing one unified interface.

# Abstract base class
class ChannelAdapter:
    def send(self, user_id, title, content, extra=None):
        raise NotImplementedError

# Concrete implementation
class APNsAdapter(ChannelAdapter):
    def __init__(self):
        self.rate_limiter = TokenBucket(rate=5000, capacity=10000)
        self.retry_policy = ExponentialBackoff(max_retries=3)
    
    def send(self, user_id, title, content, extra=None):
        if not self.rate_limiter.acquire():
            raise RateLimitExceeded()
        
        token = apns_token_store.get(user_id)
        payload = build_apns_payload(title, content, extra)
        return apns_client.push(token, payload)

# Factory to get the right channel
channel_map = {
    ‘apns’: APNsAdapter(),
    ‘fcm’: FCMAdapter(),
    ‘sms’: SMSAdapter(),
    ‘email’: EmailAdapter(),
}

Upgrade #3: User Preference Center

Before sending anything, check preferences. If the rules say skip, skip.

def check_user_preferences(user_id, notification):
    prefs = preference_service.get(user_id)
    
    # 1. Is this channel even enabled?
    if notification.channel not in prefs.enabled_channels:
        return False, “channel disabled”
    
    # 2. Are we in Do-Not-Disturb hours?
    if prefs.dnd_enabled and is_in_dnd_time(prefs.dnd_start, prefs.dnd_end):
        if notification.priority != ‘P0’:  # High priority bypasses DND
            return False, “in DND period”
    
    # 3. Hit the daily cap yet?
    today_count = daily_counter.get(user_id, notification.category)
    if today_count >= prefs.daily_limit.get(notification.category, 5):
        return False, “daily limit exceeded”
    
    # 4. Subscribed to this topic?
    if notification.topic and notification.topic not in prefs.subscribed_topics:
        return False, “topic not subscribed”
    
    return True, “ok”

Upgrade #4: Idempotency / Deduplication

Redis SETNX. Dead simple, works great.

def is_duplicate(notification_id):
    key = f”notif:sent:{notification_id}”
    # If SET succeeds, it’s not a duplicate; 24h TTL
    result = redis.set(key, “1”, ex=86400, nx=True)
    return result is None  # None means key existed → duplicate

Full Notification Flow Sequence Diagram

Is This Good Enough?

For a mid-sized company with a few million DAU, this architecture will serve you well.

But for one billion users? We’re not even close.

Now comes the hard stuff.

Version 4: Billion-Scale Distributed Architecture (The Final Form)

Let’s start with the big picture, then break down each piece.

Global Architecture Overview

Key Components, One by One

① Multi-Region Deployment: Proximity Matters

A billion users aren’t all in one country. Host everything in Shanghai, and your US users get hundreds of milliseconds of latency — if the packets even arrive at all.

So you deploy a full stack in each major region:

• Users route to the nearest region via GeoDNS

• Each region has its own independent Kafka, workers, channel adapters

• Global data like user preferences syncs via Cassandra multi-region multi-master replication

Bonus benefit: disaster recovery. If APAC goes down, shift traffic to US East and business keeps running.

② Batch Push Pipeline: Don’t Flood the Queue with 100M Messages

Marketing says “send a campaign announcement to all users.” You dump 100 million messages straight into Kafka? Say goodbye to real-time notifications for hours.

The right approach is a dedicated batch dispatcher:

def batch_push_campaign(campaign_id, segment_id, template_id):
    # 1. Stream users from segmentation system, don’t load all at once
    user_stream = user_segment_service.export_stream(segment_id)
    
    # 2. Shard by region, distribute to each region’s queue
    for batch in user_stream.chunks(batch_size=1000):
        region_users = group_by_region(batch)
        
        for region, users in region_users.items():
            kafka = get_kafka_for_region(region)
            # Throttle write rate so we don’t overwhelm downstream
            rate_limiter.wait()
            kafka.produce_many([
                build_notification_msg(u, template_id, campaign_id)
                for u in users
            ])
    
    # 3. Support pause, cancel, and dynamic rate adjustment
    campaign_tracker.update(campaign_id, status=’running’)

The whole trick is streaming reads + rate-limited writes — spread 100 million messages smoothly across several hours without impacting real-time traffic.

③ Circuit Breakers + Backpressure: Don’t Die When Third Parties Die

SMS provider goes down. FCM throttles you. This stuff happens all the time. You can’t just keep retrying and fill up your entire queue.

Every channel adapter needs a circuit breaker:

class CircuitBreaker:
    CLOSED = ‘closed’       # Normal operation, let requests through
    OPEN = ‘open’           # Failing too much, reject immediately
    HALF_OPEN = ‘half_open’ # Testing if things have recovered
    
    def __init__(self, failure_threshold=0.3, recovery_timeout=60):
        self.state = self.CLOSED
        self.failure_count = 0
        self.total_count = 0
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
    
    def execute(self, func, *args, **kwargs):
        if self.state == self.OPEN:
            if time.time() - self.last_failure_time > self.recovery_timeout:
                self.state = self.HALF_OPEN
            else:
                raise CircuitBreakerOpen()
        
        try:
            result = func(*args, **kwargs)
            self.on_success()
            return result
        except Exception as e:
            self.on_failure()
            # Failed messages go to dead letter queue for later retry
            dead_letter_queue.produce({
                ‘func’: func.__name__, 
                ‘args’: args, 
                ‘error’: str(e)
            })
            raise
    
    def on_success(self):
        self.total_count += 1
        if self.state == self.HALF_OPEN:
            self.state = self.CLOSED  # Probe succeeded, back to normal
    
    def on_failure(self):
        self.failure_count += 1
        self.total_count += 1
        self.last_failure_time = time.time()
        
        if self.total_count >= 100:  # Wait for enough samples
            failure_rate = self.failure_count / self.total_count
            if failure_rate >= self.failure_threshold:
                self.state = self.OPEN

Workers also monitor their own consumer lag. If the backlog gets too big, auto-scale more workers or signal upstream to slow down. That’s backpressure.

④ Full-Link Tracing: Finding One Notification Among Billions

Product manager walks over: “Why didn’t user X get that notification yesterday?”

Without tracing, enjoy your needle-in-haystack log dive.

The right approach: every notification carries a trace_id from API entry through to delivery receipt, with structured logs at every step:

[API] received trace_id=abc123, user_id=12345
[Orchestrator] deduplicated? No, preferences check? Passed, routed to P1
[Kafka] produced to topic=notifications-p1, offset=45678
[Worker] consumed trace_id=abc123, channel=apns
[Channel] APNs response: success, apns_id=xyz789
[Delivery] receipt received, delivered_at=2024-08-15T10:23:45Z

Ship all logs to ClickHouse, queryable by user ID or trace ID. Billions of rows, sub-second answers.

⑤ Smart Scheduling & Fatigue Management

More notifications ≠ better. Send too many, and users just turn them all off. Then you have nothing.

So you need:

• Frequency capping — max N per user per day, per category

• Time optimization — send during each user’s active hours based on timezone and history

• Smart throttling — if someone ignores several in a row, they’re probably not interested — dial it back

• One-click unsubscribe — compliance (GDPR, CAN-SPAM) and basic respect

⑥ Template System: Don’t Hardcode Content

Not going deep here, but: titles, bodies, deep links all live as templates with variable substitution, i18n support, and A/B test variants. Marketing edits them in a dashboard. No deploy required.

Let’s Do the Math: What Does “Billion Scale” Actually Mean?

War Stories: The Big Ones That Bit Me

Disaster #1: A “Broadcast to All” Button Took Down the Whole System

Before we built the batch pipeline, someone in marketing clicked “send to all users.” Millions of messages flooded the queue in seconds, every worker was saturated, and real-time chat + OTPs were completely blocked for two hours.

Lesson: Bulk sends need their own pipeline, strict rate limits, and a big red cancel button.

Disaster #2: Third-Party Outage Caused a Retry Avalanche

An SMS provider had an outage. Our workers retried furiously, piling up millions of messages in the queue. When the provider came back, expired verification codes went out by the truckload. Users were very confused.

Lesson: Circuit breakers + dead letter queues + message TTLs. All three, every time.

Disaster #3: Spamming Our Way to Higher Uninstall Rates

KPI pressure was high, so we cranked up marketing notifications. Next week’s metrics? Push opt-out rate up 30%, app uninstalls up too.

Lesson: Notifications are a tool, not a harassment vector. Respect your users, play the long game.

Disaster #4: Weak Auth Got Us SMS-Farmed

Early on, our API auth was loose. Someone scraped our SMS verification endpoint and racked up a terrifying bill.

Lesson: Entry rate limiting, per-user frequency caps, fraud checks — all non-negotiable.

Wrapping Up

The evolution path is pretty straightforward:

DB + Cron → Message Queue → Layered Architecture → Distributed & Global

Every step gets forced by scale. There’s no perfect one-size-fits-all architecture you design on day one. You build it as you break it.

The real challenge isn’t just “sending fast” — it’s doing all of this at once:

• The right message

• At the right time

• Through the right channel

• To the right person

• Reliably, at scale

If you can walk through this evolution in an interview, clearly explain what breaks at each stage and how you’d fix it, and draw a couple of these diagrams? You’re already ahead of 90% of candidates.

What’s the worst notification system disaster you’ve lived through? Drop it in the comments — I love a good war story.

Subscribe now

I once watched a production Redis instance push past a million operations per second on a single box, CPU pinned at 60%, and the engineer next to me just shrugged and said, “Yeah, that’s nothing for it.” That moment stuck with me, because it forced me to ask the real question: what is it about Redis’s design that lets it scale to 100 million requests per second?

When I get this question in an interview — and I’ve been asked it more than once — I don’t start with benchmarks. I start with the thing that confuses everyone first: Redis’s core is single-threaded. How can something single-threaded beat multi-threaded systems? Let me walk through it the way I’d answer it, layer by layer.

1. Why a Single-Threaded Model Is Actually Faster

The intuitive answer is “more threads = more throughput.” In practice, for an in-memory key-value store, that intuition is wrong. Here’s why.

1.1 The concurrency tax nobody talks about

Multi-threaded designs pay three kinds of tax on every operation:

Lock contention. The moment two threads can touch the same hash table, you need a mutex. Under high load, threads spend a surprising fraction of their time waiting, not working. And it gets worse with fine-grained locking: more locks mean more complexity, more deadlock risk, and more cache-line ping-pong.

Context switching. Every thread switch costs 1–10 microseconds: saving registers, flushing the TLB, losing the CPU’s cached view of your data structures. A Redis operation on hot data takes well under a microsecond. If your scheduling overhead is bigger than your actual work, threads are a net loss.

Cache line invalidation. This is the silent killer. Modern CPUs read memory in 64-byte cache lines. When core A writes to a variable, core B’s copy of that cache line is invalidated and must be re-fetched over the interconnect. With eight threads hammering a shared hash table, you get constant false sharing — cores invalidating each other’s caches over unrelated variables that happen to sit on the same cache line. Throughput collapses not because of any bug, but because of physics.

Redis sidesteps all three taxes with a blunt instrument: one thread owns the data. No locks on the hot path. No contention. The entire dataset lives in L1/L2 cache and stays there.

1.2 But wait — doesn’t single-threaded mean you can only use one core?

That’s the follow-up I always expect, and it’s a fair one. The answer is: for the command execution path, yes — and that’s fine. Redis’s bottleneck is almost never CPU compute; it’s network I/O and memory bandwidth. And when you do need more cores, you scale out (more instances, Redis Cluster) rather than up with threads. We’ll get to that.

The mental model I use: Redis trades parallelism for the elimination of synchronization. When your critical section is the entire dataset, the cheapest lock is the one you never need.

2. The Reactor Pattern and I/O Multiplexing

So how does one thread juggle tens of thousands of client connections? This is where the answer gets interesting: Redis is a textbook Reactor pattern implementation built on I/O multiplexing — epoll on Linux, kqueue on BSD/macOS.

2.1 The key insight: don’t block, register

A naive server reads from a socket and blocks until data arrives. With 10,000 clients, that’s 10,000 blocked threads. Redis instead asks the kernel: “tell me when any of these 10,000 sockets is actually ready,” and only then processes it. The single thread never waits on I/O — it waits on the event multiplexer, which returns only sockets that have work to do.

2.2 Inside aeMain() — the heart of Redis

Every Redis instance spends its life in one function, aeMain(). If I had to whiteboard it in an interview, this is what I’d draw:

And here’s the core logic as C pseudocode — close enough to ae.c to be honest, simplified enough to read in thirty seconds:

/* Simplified version of Redis’s ae.c event loop */
void aeMain(aeEventLoop *eventLoop) {
    eventLoop->stop = 0;
    while (!eventLoop->stop) {
        aeProcessEvents(eventLoop, AE_ALL_EVENTS | AE_CALL_BEFORE_SLEEP | AE_CALL_AFTER_SLEEP);
    }
}

int aeProcessEvents(aeEventLoop *eventLoop, int flags) {
    /* 1. Find the nearest time event to bound the poll timeout */
    aeTimeEvent *shortest = aeSearchNearestTimer(eventLoop);
    struct timeval tvp = timeUntilNextTimer(shortest);
    /* 2. Ask the kernel: which sockets are ready RIGHT NOW? */
    int numevents = aeApiPoll(eventLoop, &tvp);   /* epoll_wait / kevent under the hood */
    /* 3. Handle only the sockets that actually have work */
    for (int j = 0; j < numevents; j++) {
        aeFileEvent *fe = &eventLoop->events[eventLoop->fired[j].fd];
        if (fe->mask & AE_READABLE) fe->rfileProc(eventLoop, fd, ...);  /* readQueryFromClient */
        if (fe->mask & AE_WRITABLE) fe->wfileProc(eventLoop, fd, ...);  /* sendReplyToClient */
    }
    /* 4. Run due time events: incremental rehash, expiry scans, cron */
    processTimeEvents(eventLoop);
    return numevents;
}

The details worth mentioning in an interview:

• On Linux, Redis uses epoll in edge-triggered mode with an event array, so the cost of polling is O(ready events), not O(total connections). select()/poll() are O(n) per call — that’s why epoll matters at scale.

• Time events are never late by more than one loop iteration, because the poll timeout is always bounded by the nearest timer. That’s how activeExpireCycle runs smoothly alongside client traffic.

• Even incremental rehash (covered next) gets a slice of CPU here — a few milliseconds per loop, never a big pause.

One sentence I like to say out loud at this point: “Redis isn’t fast despite being single-threaded — it’s fast because the single thread never does anything slow: it never blocks, never locks, and never context-switches.”

3. In-Memory Data Structures Built for Speed

The event loop explains how Redis handles requests. The data structures explain why each request is so cheap. Two of my favorites: SDS and the dict with progressive rehash.

3.1 SDS — Simple Dynamic String

Redis doesn’t use C strings. It rolls its own: the SDS. The definition is small, but every field is a deliberate performance choice:

/* Redis SDS (simplified; real version has sdshdr5/8/16/32/64 variants) */
struct sdshdr {
    int len;      /* used length  -> O(1) strlen, no scanning for ‘\0’ */
    int free;     /* unused capacity -> amortized O(1) append */
    char buf[];   /* flexible array holding the actual bytes */
};

Why this matters in practice:

1. O(1) length queries. STRLEN on a C string scans for the null terminator — O(n). On a 1 MB value, that’s the difference between a nanosecond and a millisecond.

2. No buffer overflows. strcat trusts you; SDS checks free and reallocates itself.

3. Space pre-allocation. Appends under 1 MB double the buffer, so repeated APPEND is amortized O(1) instead of reallocating on every write.

4. Binary-safe. len defines the string, not a null byte — you can store images, protobuf, whatever.

5. Five header sizes (sdshdr5 through sdshdr64) mean a 3-byte key pays for a 1-byte length field, not an 8-byte one. At billions of keys, that’s gigabytes saved.

3.2 The dict: hash table + incremental rehash

Redis’s core keyspace is the dict — chained-hash tables with MurmurHash/FNV and incremental rehashing. The classic problem: when a hash table doubles in size, rehashing millions of keys in one shot would freeze the single thread for hundreds of milliseconds. Unacceptable.

Redis’s answer: spread the rehash across many small steps, doing a little bit of work on every access.

/* Simplified from dict.c — the progressive rehash contract */
typedef struct dict {
    dictht ht[2];          /* two tables: source and destination */
    long rehashidx;        /* -1 = not rehashing; else next bucket to move */
} dict;

/* Called incrementally: move `n` buckets, never more */
void dictRehash(dict *d, int n) {
    while (n-- && d->ht[0].used != 0) {
        /* find next non-empty bucket */
        while (d->ht[0].table[d->rehashidx] == NULL) d->rehashidx++;
        /* move all entries in this bucket to ht[1] */
        migrateEntries(d->ht[0].table[d->rehashidx], &d->ht[1]);
        d->rehashidx++;
    }
    if (d->ht[0].used == 0) { ht[1] becomes ht[0]; rehashidx = -1; } /* done */
}

/* Every lookup/delete/insert helps a little while rehash is active */
dictEntry *dictFind(dict *d, const void *key) {
    if (dictIsRehashing(d)) dictRehash(d, 1);   /* pay one step of the toll */
    /* search ht[0], then ht[1] */
}

The beauty is in the amortization: no single operation ever pays the full rehash cost. Worst-case latency stays flat while the table grows from 1M to 100M keys. Same idea powers KEYS alternatives and why SCAN exists — Redis consistently refuses to do big things all at once.

4. Pipelining: Killing the RTT Bottleneck

Now let’s leave the server and look at the client side — because I’ve profiled plenty of “slow Redis” problems that turned out to be pure network latency.

Here’s the trap: on a 0.5 ms RTT network, sending 100,000 commands one at a time costs 50 seconds of pure round trips, even if Redis executes each in 10 microseconds. The bottleneck isn’t Redis; it’s physics.

Pipelining flips the model: send a batch of commands without waiting for replies, then read all replies at once. RTT is paid once, not N times.

4.1 Wrong way vs. right way

import redis.clients.jedis.Jedis;
import redis.clients.jedis.Pipeline;

public class PipelineDemo {
    public static void main(String[] args) {
        try (Jedis jedis = new Jedis(”localhost”, 6379)) {
            int N = 100_000;

            // ❌ Wrong: one round trip per command
            // With 0.5ms RTT this takes ~50 seconds no matter how fast Redis is.
            long start = System.nanoTime();
            for (int i = 0; i < N; i++) {
                jedis.set(”key:” + i, String.valueOf(i)); // send -> WAIT for reply -> next
            }
            System.out.printf(”naive:    %.2fs%n”, (System.nanoTime() - start) / 1e9);

            // ✅ Right: pipeline batches commands, one round trip per batch
            start = System.nanoTime();
            Pipeline pipe = jedis.pipelined();   // pipeline != MULTI/EXEC
            for (int i = 0; i < N; i++) {
                pipe.set(”key:” + i, String.valueOf(i));
            }
            pipe.sync();                         // one big send, one big read
            System.out.printf(”pipeline: %.2fs%n”, (System.nanoTime() - start) / 1e9);
        }
    }
}

On a typical cross-AZ link, I’ve seen this turn 45 seconds into 1.2 seconds — a ~40x win with zero server-side changes.

Two things I always add:

• Size your batches. A pipeline of 100k commands builds a large output buffer on the server. Batches of 1,000–10,000 are usually the sweet spot between throughput and memory.

• Pipeline ≠ transaction. A pipeline just batches I/O; commands can interleave with other clients’. Use MULTI/EXEC (e.g., Jedis’s `Transaction`) only when you need atomicity.

5. Scaling to 100M RPS: Redis Cluster

A single Redis instance tops out around 1M+ ops/s on modern hardware (more with threaded I/O, next section). To reach 100 million, we scale horizontally with Redis Cluster: 16,384 hash slots distributed across masters, each master with replicas for failover.

Rough math I’d share in an interview: 100 shards × ~1M ops/s each ≈ 100M RPS, with headroom. The cluster isn’t magic — it’s partitioning plus a gossip protocol.

5.1 Sharding topology

How keys land on slots: CRC16(key) mod 16384. Here’s the Java version:

import java.nio.charset.StandardCharsets;

public class SlotCalculator {
    /**
     * Redis Cluster slot for a key.
     *
     * Honors hash tags: ‘{user:1}.profile’ hashes only ‘user:1’,
     * letting you force related keys onto the same slot (e.g. for MULTI).
     */
    public static int keySlot(String key) {
        int s = key.indexOf(’{’), e = key.indexOf(’}’);
        if (s != -1 && e != -1 && e > s + 1) {
            key = key.substring(s + 1, e);                      // hash tag wins
        }
        return crc16(key.getBytes(StandardCharsets.UTF_8)) % 16384; // XMODEM CRC16
    }

    private static int crc16(byte[] data) {
        int crc = 0;
        for (byte b : data) {
            crc ^= (b & 0xFF) << 8;
            for (int i = 0; i < 8; i++) {
                crc = (crc & 0x8000) != 0 ? (crc << 1) ^ 0x1021 : crc << 1;
                crc &= 0xFFFF;
            }
        }
        return crc;
    }

    public static void main(String[] args) {
        System.out.println(keySlot(”user:1000”));            // e.g. 6918
        System.out.println(keySlot(”{user:1000}.profile”));  // same slot as ‘user:1000’
    }
}

And standing up a 3-master/3-replica cluster from the CLI:

# 1. Launch six nodes (ports 7000-7005), each with cluster-enabled yes
for port in $(seq 7000 7005); do
  redis-server ./nodes/$port/redis.conf &
done

# 2. Create the cluster: 3 masters, 1 replica each, slots auto-assigned
redis-cli --cluster create \
  127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 \
  127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 \
  --cluster-replicas 1
# 3. Verify slot distribution and health
redis-cli --cluster check 127.0.0.1:7000

Three operational points worth saying in an interview:

1. Clients must be cluster-aware. Smart clients (e.g., `JedisCluster`, Lettuce) cache the slot→node map and talk to the right master directly. A `MOVED` reply just means “refresh your map.”

2. No multi-key across slots. Multi-key commands spanning slots fail unless the keys share a hash tag — that’s the trade-off of partitioning.

3. Failover is automatic. If a master dies, its replicas elect a replacement via gossip-based voting, with no external sentinel required.

6. The Multi-Threaded I/O Evolution (Redis 6.0 → 7.x)

Here’s the part that surprises people, and it’s a great way to close the answer: Redis added threads — but only where they actually help.

Redis 6.0 introduced multi-threaded I/O: socket reading, protocol parsing, and response writing can be offloaded to I/O threads, while command execution stays strictly single-threaded. Redis 7.x carried this forward with refinements, and the config became genuinely practical at high connection counts.

Why this split? Profiling showed that at very high throughput, the bottleneck wasn’t executing commands — it was syscalls and string parsing of the RESP protocol. So Redis parallelized the cheap-to-parallelize part (I/O) and kept the hard-to-parallelize part (data access) lock-free.

# redis.conf — threaded I/O
io-threads 4           # I/O threads; rule of thumb: ~ half the core count
io-threads-do-reads yes

The lifecycle of one event-loop iteration with threaded I/O:

1. Main thread collects ready reads, distributes them across I/O threads.

2. I/O threads read and parse requests in parallel; main thread waits (spin) until done.

3. Main thread executes all commands sequentially — the data model stays trivially safe.

4. Replies are distributed back to I/O threads for parallel writes.

The honest numbers I’d quote: with 4–8 I/O threads, throughput roughly doubles on network-bound workloads, with no change in the consistency model. It’s not a 4x win, because execution is still serial — and that’s the design point. Redis grew threads surgically, exactly where profiling said to.

7. Key Takeaways

If I had one minute left in the interview, this is the summary I’d give:

1. Single-threaded is a feature, not a limitation. By eliminating lock contention, context switches, and cache-line invalidation, Redis’s hot path runs in microseconds with zero synchronization cost.

2. The Reactor pattern + I/O multiplexing (epoll/kqueue) let one thread serve tens of thousands of connections by processing only sockets that are actually ready — the aeMain() loop never blocks on any single client.

3. Purpose-built data structures keep every operation cheap: SDS gives O(1) length checks and amortized O(1) appends; the dict’s incremental rehash spreads table resizing across millions of operations so latency never spikes.

4. Pipelining removes the network round trip from the critical path — often a 10–50x client-side speedup with zero server changes. If your Redis feels slow, check the client first.

5. 100M RPS comes from horizontal scaling. Redis Cluster splits keyspace into 16,384 slots (CRC16(key) % 16384) across many masters with automatic replica failover; ~100 shards × ~1M ops/s gets you there.

6. Threaded I/O (6.0+, refined in 7.x) shows pragmatic evolution: parallelize socket I/O where profiling proved the bottleneck, keep command execution single-threaded to preserve the lock-free data model.

The deeper lesson, the one I’d want the interviewer to walk away with: Redis isn’t fast because of any single trick. It’s fast because every layer of the design — threading model, event loop, memory layout, wire protocol, cluster topology — was engineered to remove work rather than optimize it. The fastest instruction is the one you never execute; the fastest lock is the one that doesn’t exist.

Share Java, SpringBoot & AI Development: Coding Skills & Interview

Interviewers often open with this one: how do you check whether a username already exists among one billion users? In the previous article we gave the cheapest answer, the Bloom filter. At a 1% false positive rate each element needs about 9.6 bits, so a table of one billion usernames fits in 1.2 GB of memory. Push the target down to 0.1% and each element climbs to 14.4 bits, about 1.8 GB, still within reach of a single machine.

So what is the Cuckoo filter for? Because lists get deleted. Usernames almost never disappear, but blacklists and device lists lose entries all the time, and the Bloom filter has no delete operation.

The database stays the source of truth. The filter is just a gate in front of it. Bloom or Cuckoo comes down to one question: will this set lose entries?

Subscribe now

Understanding the Challenge

Checking a username against one billion existing users packs several problems into one path.

The set is too large to scan. On the registration hot path, a linear pass over one billion rows is out of the question.

This check runs right in front of the user. Every registration needs it, so does most login traffic, and it has to come back in single-digit milliseconds. Slower than that and the product starts to feel laggy.

Memory matters just as much. A hash set of one billion usernames, ten bytes each on average, runs to tens of gigabytes. You need a structure far smaller than the raw data to answer the membership question.

Usernames almost never disappear, but other lists do. Blacklists get unbanned, device lists expire, rate-limit lists get cleared. Once the filter cannot remove entries, the only option is a full rebuild.

Two users can race for the same username. The filter tells both “probably available”, and something downstream still has to guarantee that only one registration succeeds.

The Baseline: Database Design

No matter what filter sits in front, the database stays the source of truth. The schema from the previous article still works.

CREATE TABLE users (
    user_id BIGINT PRIMARY KEY AUTO_INCREMENT,
    username VARCHAR(255) UNIQUE NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

The unique index on username is not optional. It is the atomic gate that blocks duplicates when two requests race. A plain B-tree index handles point lookups fine, and the unique constraint doubles as the concurrency backstop.

The most direct way is to query the database once.

SELECT COUNT(*) FROM users WHERE username = 'desiredUsername';

Or as a prepared statement.

boolean exists = jdbcTemplate.queryForObject(
    "SELECT COUNT(*) FROM users WHERE username = ?",
    Boolean.class, username);

Both work, and both cost too much on the hot path. At one billion rows, every check walks the index and fetches data, burning database CPU even when the answer is obvious. The database should only see the cases the filter cannot decide.

Step One: Put a Bloom Filter in Front

A Bloom filter answers one binary question: is this element in the set? It maps the element to k positions in a bit array with k hash functions. All k positions must be 1 for the answer to be “possibly present”. If any position is 0, the element is definitely not there.

That asymmetry is the point. It never produces false negatives, it can produce false positives, and the rate is tunable. With a bit array of size m and k hash functions, the false positive rate is about

(1 - e^(-kn/m))^k

The optimal number of hash functions is k = (m/n) * ln(2). At a 1% false positive rate, m/n lands around 9.6 bits and k is 7. At 0.1%, m/n is about 14.4 bits and k is about 10. Element length does not matter: one billion entries at 1% cost a fixed 1.2 GB, whether the usernames are 6 characters or 60.

A Minimal Java Implementation

import java.util.BitSet;

public class SimpleBloomFilter {
    private final BitSet bits;
    private final int size;
    private final int hashCount;

    public SimpleBloomFilter(int size, int hashCount) {
        this.bits = new BitSet(size);
        this.size = size;
        this.hashCount = hashCount;
    }

    private int[] positions(String value) {
        int[] pos = new int[hashCount];
        int hash1 = value.hashCode();
        int hash2 = (hash1 >>> 16) | (hash1 << 16);
        for (int i = 0; i < hashCount; i++) {
            pos[i] = Math.floorMod(hash1 + i * hash2, size);
        }
        return pos;
    }

    public void add(String value) {
        for (int p : positions(value)) {
            bits.set(p);
        }
    }

    public boolean mightContain(String value) {
        for (int p : positions(value)) {
            if (!bits.get(p)) {
                return false;   // one bit is 0, definitely absent
            }
        }
        return true;            // all bits are 1, possibly present
    }
}

Don’t write this yourself in production. Guava’s BloomFilter handles sizing, hashing, and serialization.

BloomFilter<String> takenUsernames = BloomFilter.create(
        Funnels.stringFunnel(StandardCharsets.UTF_8),
        1_000_000_000L,   // expected insertions
        0.01);            // target false positive rate

takenUsernames.put("alice");
boolean taken = takenUsernames.mightContain("alice");

Redis exposes the same operations as commands.

BF.RESERVE usernames 0.01 1000000000
BF.ADD usernames alice
BF.EXISTS usernames alice

The Problem Bloom Cannot Solve: Deletion

Bits are shared. Clearing the bits of one element drags down other elements with it. A standard Bloom filter supports insert and query, nothing else.

The usual patch is a counting Bloom filter: each position holds a counter instead of a single bit, and delete decrements the counter. The cost is memory, about four times the standard filter with 4-bit counters. Counters saturate at 15, so heavy delete traffic still produces errors. It is a patch, and the underlying design still has no native delete.

For username availability this does not matter: accounts are almost never deregistered, and the set only grows. Blacklists and device lists are different, deletion is a hard requirement.

Step Two: Cuckoo Filter When Deletion Matters

The Cuckoo filter comes from Fan et al.’s 2014 paper Cuckoo Filter: Practically Better Than Bloom, built to close exactly this gap.

Instead of a bit array, it stores a short fingerprint per element. Each element maps to two candidate buckets, and lookups only check those two. When both are full, the new fingerprint kicks out an old one, and the displaced fingerprint moves to its alternate bucket. The name comes from the cuckoo’s habit of pushing other eggs out of the nest.

Kicks push the load factor to about 95%, which keeps the filter compact. Fan et al. compared it directly with the alternatives: against counting Bloom filters it saves about half the space, against a non-deletable, space-optimized Bloom filter it costs 1.5 to 2 times more. The lower the target false positive rate, the smaller the gap. A USENIX ;login article gives a concrete pair of numbers: a Cuckoo filter with 12-bit fingerprints uses about 12.53 bits per element at a 0.19% false positive rate, while a Bloom filter uses 13 bits at 0.20%. At the one-billion scale the two filters sit in the same memory class, the real difference is deletion.

Fingerprint size follows a simple rule: for a target false positive rate r and bucket size b, take f >= log2(2b/r) bits. With 4-slot buckets and a 1% target, that means 10-bit fingerprints, about 10.5 bits per element including the load factor. Slightly above the Bloom filter’s 9.6 bits. When deletion is a hard requirement, that extra cost is worth paying.

A Simplified Java Implementation with Delete

public class SimpleCuckooFilter {
    private static final int BUCKET_SIZE = 4;    // fingerprints per bucket
    private static final int MAX_KICKS = 500;    // max kick attempts
    private final long[][] buckets;
    private final int numBuckets;

    public SimpleCuckooFilter(int numBuckets) {
        this.buckets = new long[numBuckets][BUCKET_SIZE];
        this.numBuckets = numBuckets;
    }

    private long fingerprint(String value) {
        return value.hashCode() & 0xFFFFL;       // 16-bit fingerprint
    }

    private int index1(String value) {
        return Math.floorMod(value.hashCode(), numBuckets);
    }

    private int index2(int i1, long fp) {
        return Math.floorMod(i1 ^ Long.hashCode(fp), numBuckets);
    }

    private boolean insertInto(int index, long fp) {
        for (int j = 0; j < BUCKET_SIZE; j++) {
            if (buckets[index][j] == 0) {
                buckets[index][j] = fp;
                return true;
            }
        }
        return false;
    }

    public boolean insert(String value) {
        long fp = fingerprint(value);
        int i1 = index1(value);
        int i2 = index2(i1, fp);
        if (insertInto(i1, fp) || insertInto(i2, fp)) {
            return true;
        }

        int index = (Math.random() < 0.5) ? i1 : i2;
        for (int k = 0; k < MAX_KICKS; k++) {
            int slot = (int) (Math.random() * BUCKET_SIZE);
            long kicked = buckets[index][slot];
            buckets[index][slot] = fp;
            fp = kicked;
            index = index2(index, fp);
            if (insertInto(index, fp)) {
                return true;
            }
        }
        return false;                            // too full, expand or rebuild
    }

    public boolean contains(String value) {
        long fp = fingerprint(value);
        int i1 = index1(value);
        int i2 = index2(i1, fp);
        for (int j = 0; j < BUCKET_SIZE; j++) {
            if (buckets[i1][j] == fp || buckets[i2][j] == fp) {
                return true;
            }
        }
        return false;
    }

    public boolean delete(String value) {
        long fp = fingerprint(value);
        int i1 = index1(value);
        int i2 = index2(i1, fp);
        for (int j = 0; j < BUCKET_SIZE; j++) {
            if (buckets[i1][j] == fp) {
                buckets[i1][j] = 0;
                return true;
            }
        }
        for (int j = 0; j < BUCKET_SIZE; j++) {
            if (buckets[i2][j] == fp) {
                buckets[i2][j] = 0;
                return true;
            }
        }
        return false;
    }
}

This is a teaching version: empty slots are marked with 0 and fingerprints are only 16 bits. A production implementation also has to handle fingerprint collisions, delete edge cases, and expansion. One warning: deleting a fingerprint that was never inserted, a false positive, removes someone else’s fingerprint and creates a false negative. Only delete what you actually added.

Redis provides the same operations, and delete is the one command Bloom cannot offer.

CF.RESERVE usernames 1000000000 BUCKETSIZE 4
CF.ADD usernames alice
CF.EXISTS usernames alice
CF.DEL usernames alice

Rewriting the Original Example: A Filter-First Service

The previous article cached query results with short TTLs. This version moves the filter in front of the database, which stays the source of truth.

@Service
public class UsernameAvailabilityService {

    private final BloomFilter<String> takenUsernames = BloomFilter.create(
            Funnels.stringFunnel(StandardCharsets.UTF_8),
            1_000_000_000L,
            0.01);

    private final UserRepository userRepository;

    public UsernameAvailabilityService(UserRepository userRepository) {
        this.userRepository = userRepository;
    }

    public boolean isAvailable(String username) {
        if (!takenUsernames.mightContain(username)) {
            return true; // definitely absent, no database call
        }
        // filter says "possibly taken", verify against the source of truth
        return !userRepository.existsByUsername(username);
    }

    @Transactional
    public void register(String username) {
        if (!isAvailable(username)) {
            throw new UsernameTakenException(username);
        }
        try {
            userRepository.insert(username); // unique index is the final gate
            takenUsernames.put(username);    // keep the filter warm
        } catch (DuplicateKeyException e) {
            throw new UsernameTakenException(username);
        }
    }
}

The flow matches the original architecture diagram: the filter absorbs the vast majority of requests, and the database only sees the “possibly taken” cases plus the final insert.

Lists that need deletion use the same pattern, just with a Cuckoo filter. The service below keeps a blacklist in memory and supports unbanning, something the Bloom version cannot do without a rebuild.

public class BlacklistService {
    private final SimpleCuckooFilter blockedTokens = new SimpleCuckooFilter(1 << 20);

    public boolean isBlocked(String token) {
        return blockedTokens.contains(token);
    }

    public void block(String token) {
        blockedTokens.insert(token);
    }

    public void unblock(String token) {
        blockedTokens.delete(token);
    }
}

Handling Concurrency

The filter answers the membership question, it is not a transaction. Two users can both see the same username as “available”, and the unique index turns the race into a single winner.

The cleanest approach is atomic check-and-insert: try the insert, let the database enforce the constraint, and translate DuplicateKeyException into a friendly error. The optimistic and pessimistic locking from the previous article still applies to rows with extra state. For the username itself, the unique index is simpler and faster than any application-level lock.

There is a second concurrency concern the previous article never had to face: not every Bloom filter implementation is thread-safe for concurrent writes. If the filter lives in application memory, writes need synchronization or a thread-safe variant. If it lives in Redis, commands execute atomically on the server, one more reason to take the Redis path.

Performance Considerations

You can shard the database by user id or username hash. The filter stays whole, it only answers existence, and 1.2 GB is small enough for every application node to keep a copy.

The Bloom filter has no delete, so a shrinking list eventually needs a rebuild. A rebuild is not a config change: you replay the current set, load the new filter, switch traffic, and watch the false positive rate for a while.

When insert still fails after the maximum number of kicks, the Cuckoo filter is full. Redis’s CF.RESERVE supports expansion parameters, and production implementations grow the table instead of failing the request.

The availability check must be synchronous. Downstream work like confirmation emails or analytics can go into a queue.

Which Filter Do You Pick

Answer one question first: will the list delete entries?

No, use a Bloom filter. At typical false positive rates it is simpler and smaller, with a mature ecosystem: Guava and Redis ship it, most databases do too. One-billion-username checks are its standard use case.

Yes, then look at how often and how much. If deletions are sparse and the list can be rebuilt, stay with Bloom plus periodic rebuilds. With frequent deletions at scale, the rebuild cost becomes unbearable, use the Cuckoo filter there.

DimensionBloom FilterCuckoo FilterBits per element, 1% FPR~9.6~10.5Bits per element, ~0.2% FPR~13~12.5DeletionnoyesInsert can failnoyes, needs expansionLoad factorn/a~95%Java ecosystemGuava, matureno Guava-level defaultRedis supportBF.* commandsCF.* commands

The Cuckoo filter costs more to implement and adds an insert-failure path. That cost is only justified when deletion is a real requirement. Blacklists and device lists qualify. Username availability, where accounts almost never disappear, does not.

References

• Fan et al., Cuckoo Filter: Practically Better Than Bloom, ACM CoNEXT 2014

• USENIX ;login article with the 12.53 vs 13 bits comparison, https://www.usenix.org/system/files/login/issues/1308_login_online.pdf

• ScyllaDB glossary entry on Bloom filters, 9.6 bits per element at 1% FPR, https://www.scylladb.com/glossary/bloom-filter/

• Redis Stack documentation for the CF.ADD command, https://redis-stack.io/commands/cf.add/

• Guava BloomFilter API documentation, https://guava.dev/releases/31.0-jre/api/docs/com/google/common/hash/BloomFilter.html

Share Java, SpringBoot & AI Development: Coding Skills & Interview

Preface

I wonder if everyone has noticed that when registering with some apps, it prompts you that the username has been occupied and you need to change one.

Subscribe now

There are many ways to implement this function. Now let’s take a look at the advantages and disadvantages of different design schemes one by one.

Database Scheme

This method is the simplest to implement, but it will bring the following problems:

1. There are performance problems with relatively high latency. If the data volume is huge, the query speed will become slow. Moreover, database queries involve network communication between the application server and the database server. The time required for establishing connections, sending queries, and receiving responses also causes latency.

2. The database load is excessive. Frequent execution of SELECT queries to check the uniqueness of usernames, and each query consumes database resources, including CPU and I/O resources.

3. Poor scalability. Databases have limitations on concurrent connections and resources. If the registration rate continues to rise, the database server may have difficulty handling the increasing number of incoming requests. Vertical expansion of the database (adding more resources to a single server) may be costly and may have limitations.

Cache Solution

To address the performance issue of database calls for checking the uniqueness of usernames, an efficient Redis cache is introduced.

import org.redisson.Redisson;  import org.redisson.api.RedissonClient;  import org.redisson.config.Config;  import org.redisson.api.RMap;  
  public class UserExistenceChecker {  
  
    // Redis hash map name to store user information  
    private static final String USER_HASH_NAME = “users”;  
  
    public static void main(String[] args) {  
        // Create a Redisson client  
        RedissonClient redisson = createRedissonClient();  
  
        // Retrieve the hash map to store user information  
        RMap<String, String> users = redisson.getMap(USER_HASH_NAME);  
  
        // Add a user to the hash map  
        users.put(”user123”, “someUserInfo”); // Here “someUserInfo” could be a JSON string, UUID, etc.  
  
        // Check if a user exists  
        boolean exists = users.containsKey(”user123”);  
        System.out.println(”User ‘user123’ exists? “ + exists);  
  
        // Check for a non-existent user  
        exists = users.containsKey(”user456”);  
        System.out.println(”User ‘user456’ exists? “ + exists);  
  
        // Shutdown the Redisson client  
        redisson.shutdown();  
    }  
  
    // Helper method to create a Redisson client  
    private static RedissonClient createRedissonClient() {  
        Config config = new Config();  
        config.useSingleServer()  
                .setAddress(”redis://127.0.0.1:6379”) // Adjust to your Redis address  
                .setPassword(”yourpassword”); // Provide your Redis password if any  
  
        return Redisson.create(config);  
    }  
}

The biggest problem with this solution is the excessive memory usage. Suppose each username requires approximately 15 bytes of memory. If you want to store one billion usernames, you would need 15GB of memory.

Total memory = Memory usage per record * Number of records = 15 bytes/record * 1,000,000,000 records = 15,000,000,000 bytes ≈ 15,000,000 KB ≈ 15,000 MB ≈ 15 GB

Bloom Filter Scheme

If direct caching for judgment results in excessive memory usage, is there a better way? The Bloom Filter is a very good choice.

What is a Bloom Filter?

The Bloom Filter is a highly space-efficient random data structure. It uses a bit array to concisely represent a set and can determine whether an element belongs to this set.

This efficiency of the Bloom Filter comes at a certain cost: When determining whether an element belongs to a certain set, it is possible to mistakenly consider an element that does not belong to the set as belonging to the set (false positive).

Therefore, the Bloom Filter is not suitable for application scenarios with “zero errors”. However, in application scenarios that can tolerate a low error rate, the Bloom Filter achieves significant savings in storage space through very few errors.

Structure

It can be known from the above analysis that the core idea of the Bloom filter is to use a bit array (bit array) and a set of hash functions.

The bit array, each bit is 0 initially

When inserting the value x, use k hash functions (3 in the figure) to hash the value of x respectively, and take the remainder of the hash value and the capacity(bit array length) of the Bloom filter, and set the value of the corresponding bit represented by the result to 1.

A search process is similar to an insertion process. Similarly, k hash functions are used to hash the value to be searched. Only when the value of each bit obtained by hashing is 1 does it indicate that the value “possibly” truly exists; conversely, if the value of any bit is 0, it indicates that the value must not exist. For example, y1 must not exist; while y2 may exist.

Redis itself supports the data structure of the Bloom filter. Let’s simply implement the code using the Redisson client:

import org.redisson.Redisson;
import org.redisson.api.RBloomFilter;
import org.redisson.api.RedissonClient;
import org.redisson.config.Config;

public class UserExistenceChecker {

    // Name of the Bloom Filter in Redis
    private static final String BLOOM_FILTER_NAME = “user_existence_filter”;

    public static void main(String[] args) {
        // Create a Redisson client
        RedissonClient redisson = createRedissonClient();

        // Retrieve or create a Bloom Filter instance
        // Expected number of elements and false positive probability are parameters
        RBloomFilter<String> bloomFilter = redisson.getBloomFilter(BLOOM_FILTER_NAME);
        bloomFilter.tryInit(100000L, 0.001); // Initialize the Bloom Filter with expected elements and false positive rate

        // Add a user to the Bloom Filter
        bloomFilter.add(”user123”);

        // Check if a user exists
        boolean exists = bloomFilter.contains(”user123”); // Should return true
        System.out.println(”User ‘user123’ exists? “ + exists);

        // Check for a non-existent user (might falsely report as true due to Bloom Filter’s nature)
        exists = bloomFilter.contains(”user456”); // Assuming not added, should ideally return false, but could be a false positive
        System.out.println(”User ‘user456’ exists? “ + exists);

        // Shutdown the Redisson client
        redisson.shutdown();
    }

    // Helper method to create a Redisson client
    private static RedissonClient createRedissonClient() {
        Config config = new Config();
        config.useSingleServer()
                .setAddress(”redis://127.0.0.1:6379”); // Adjust to your Redis address
//                .setPassword(”yourpassword”); // Provide your Redis password if any

        return Redisson.create(config);
    }
}

Advantages:

• Memory space saving: Compared to using data structures such as hash tables, the Bloom filter usually requires less memory space because it does not store actual elements but only the hash values of the elements. If storing 1 billion records with an error probability of 0.001, only 1.67 GB of memory is needed. Compared to the original 15G, it has been greatly reduced.

• Efficient lookup: The Bloom filter can quickly determine whether an element exists in the set in constant time (O(1)), without traversing the entire set.

Disadvantages:

• False positive rate exists: When the Bloom filter determines whether an element exists, there is a certain false positive rate. This means that in some cases, it may wrongly report that an element exists, but will not wrongly report that an element does not exist.However, this generally has little impact.

• Cannot delete elements: The Bloom filter usually does not support deleting elements from the set because deleting an element will affect the hash values of other elements, increasing the false positive rate.

How to ensure that the Bloom filter scheme has no false positives?

Here, the scheme of combining the Bloom filter with the database can be considered.

When the Bloom filter is used to determine whether an element exists, there is a certain probability that it may wrongly report that the element exists, but it will not wrongly report that the element does not exist.

So, when using the Bloom filter to determine that an element does not exist, you can directly trust this result and return. If it is determined that an element exists, don’t completely trust its judgment at this time. Instead, query the real result in the database.

Because the probability of determining that an element exists is already very low, the actual magnitude of accessing the database will also be very small, and the overall pressure is not great.

Can elements be deleted from a Bloom filter?

Why can’t elements be deleted from a Bloom Filter? We can use an example to illustrate.

For example, if we want to delete the member “Jerry” from the set, then we will first use k(2 in the figure) hash functions to calculate it. Since “Jerry” is already a member of the set, the corresponding positions in the bit array must be 1. If we want to delete this member “Jerry”, we need to set all the 1s at the calculated positions to 0. In the following figure, just set the values at index positions 2 and 5 to 0.

Here comes the problem! Now, let’s assume that “Tom” is also already an element of the set. If we need to query whether “Tom” is in the set, after calculation by the hash function, we will judge whether the third and fifth positions are 1. At this time, we get the result that the fifth position is 0, that is, “Tom” does not belong to the set. Obviously, this is a false positive here.

So, the original Bloom Filter does not support deleting elements, but the Counting Bloom Filter can.

The emergence of the Counting Bloom Filter solves the above problem. It extends each bit of the bit array of the standard Bloom Filter to a small counter. When inserting an element, the values of the corresponding k (k is the number of hash functions) counters are incremented by 1 respectively. When deleting an element, the values of the corresponding k counters are decremented by 1 respectively.

From this, we can see that the Counting Bloom Filter adds a deletion operation to the Bloom Filter at the cost of occupying several times more storage space. Isn’t the basic principle very simple? Looking at the following picture, you can understand where the difference between it and the Bloom Filter lies.

Selection of Counter Size

A major difference between the Counting Bloom Filter and the Bloom Filter is that the Counting Bloom Filter replaces one bit in the Bloom Filter with a Counter.

So, how large should the Counter be? Here, the issue of space utilization needs to be considered. From a usage perspective, of course, the larger the better, because a larger Counter can represent more information. But a larger Counter means more resource occupation and often causes a great waste of space.

Therefore, when choosing a Counter, we should try to meet the requirements as much as possible. The specific calculation of the Counter is relatively complex and involves a series of mathematical formulas. It will not be discussed here. Interested friends can refer to pages 6 and 7 of the paper: “Summary Cache: A Scalable Wide-Area Web Cache Sharing Protocol”, which specifically elaborates on it.

Summary

The Redis Bloom filter scheme provides an efficient memory-based solution for uniqueness verification under large data volumes. It needs to strike a balance between memory consumption and error rate. Of course, the Bloom filter has more application scenarios, such as preventing cache penetration and preventing malicious access.

Share Java, SpringBoot & AI Development: Coding Skills & Interview