python how to implement cache with expiration time, full text search, frequency limit


In the previous article, I introduced you to using Python's own LRU cache to implement a cache with expiration time: One Day One Technique: Implementing an LRU Cache with Expiration Time. I also talked about inverted indexing: using inverted indexing to improve string search efficiency extremely fast. However, these codes are difficult for beginners and can be written with errors.

In fact, all of these functions can actually be implemented using Redis, and they can be done in just one minute each. The full-text search feature can even intelligently identify spelling errors when searching for English.

To achieve these functions, only two things need to be done:

  1. Installing Redis

  2. Python installation of third-party libraries: walrus

After the installation is complete, let's see how easy it is:

Cache decorator with expiration time

We want to implement a decorator which decorates a function. Let me use the cached data when accessing the function multiple times within 1 minute; re-execute the function's internal code only after 1 minute:

import time
import datetime
from walrus import Database

db = Database()
cache = db.cache()

def test():
    now =
    return now

now = test()
print('The data returned by the function is:', now)
time.sleep(10) # Wait 10 seconds, at this point the cache will be used
print('The data returned by the function is:', test())
time.sleep(5) # Wait 5 seconds, at this point the cache is still in use
print('The data returned by the function is:', test())

time.sleep(50)  # Let the time exceed the cache time
print('The data returned by the function is:', test())


Full text search

Let's take a look at the full-text search function, which is also very simple to implement:

from walrus import Database

db = Database()
search = db.Index('xxx')  # The name is arbitrary
poem1 = 'Early in the day it was whispered that we should sail in a boat, only thou and I, and never a soul in the world would know of this our pilgrimage to no country and to no end.'
poem2 = 'Had I the heavens’ embroidered cloths,Enwrought with golden and silver light'
poem3 = 'to be or not to be, that is a question.'

search.add('docid1', poem1) # The first parameter cannot be repeated
search.add('docid2', poem2)
search.add('docid3', poem3)

for doc in'end'):

The running effect is shown in the following figure:


If you want him to be compatible with spelling errors, then you can change search = db.Index('xxx') to search = db.Index('xxx', metaphone=True) and run the effect as shown below:


Unfortunately, however, this full-text search function only supports English.

Frequency limitation

Sometimes we want to limit the frequency of calls to a certain function, or we want to limit the frequency of IP access to a certain interface of a website. This can be easily done with walrus:

import time
from walrus import Database

db = Database()
rate = db.rate_limit('xxx', limit=5, per=60) # Only 5 calls per minute

for _ in range(35):
    if rate.limit('xxx'):
        print('Too frequent visits!')
        print('No access frequency limit triggered yet')


Where the parameter limit indicates how many times it can appear and per indicates over how long.

rate.limit will start checking how often this parameter appears in the set time as long as the same parameter is passed in.

You may think that this example doesn't tell us much, so let's combine it with FastAPI and use it to limit the frequency of IP access to the interface. Write the following code

from walrus import Database, RateLimitException
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse

db = Database()
rate = db.rate_limit('xxx', limit=5, per=60) # Only 5 calls per minute

app = FastAPI()

def parse_rate_litmit_exception(request: Request, exc: RateLimitException):
    msg = {'success': False, 'msg': f'Your ip access is too fast!'}
    return JSONResponse(status_code=429, content=msg)

def index():
    return {'success': True}

@rate.rate_limited(lambda request:
def query_important_data(request: Request):
    data = 'Important Data'
    return {'success': True, 'data': data}

The above code defines a global exception interceptor:

def parse_rate_litmit_exception(request: Request, exc: RateLimitException):
    msg = {'success': False, 'msg': f'Your ip access is too fast!'}
    return JSONResponse(status_code=429, content=msg)

Related articles

Three ways to configure logging for FastAPI

Recently, when I was using FastAPI, I found that the official documentation of FastAPI did not have instructions related to configuring logs. Today, I will share three methods of configuring logs for FastAPI.

FastAPI interface flow limitation

Without interface flow limiting, it may lead to server load imbalance, brute force password cracking, malicious requests, extra server charges, denial of service attacks, etc. Therefore it is necessary to do a good job of interface flow limiting.

Replacing excel with Mito

Mito is a spreadsheet library in Python. If you can edit Excel files, you can write code. That's because for every action we perform in a table, Mito will automatically generate the corresponding Python code. Say goodbye to repetitive and boring operation

Python face recognition algorithm

Today we give you a summary of a few simple and good face recognition algorithms. Face recognition is a relatively common technology in computer vision. In life, the face recognition scenario that we have the most contact with is face attendance.

Three ways to parse parameters in Python

Like the diagram above, we have a standard structure to organize our small projects: The folder named data that contains our dataset file The file for specifying hyperparameters

Seaborn draws 11 bar charts

This article is about how to use seaborn to draw various bar charts Base Bar Chart Horizontal Bar Chart Title Settings DataFrame based drawing hue parameter setting Color processing Multi-dimensional processing

Tips for speeding up pandas

When people talk about data analysis, the most mentioned languages are Python and SQL. python is suitable for data analysis because it has many powerful third-party libraries to assist, pandas being one of them. pandas is described in the documentation as

Speed up pandas with Modin

Modin exists for a reason: to change one line of code to speed up the pandas workflow.Pandas needs no introduction in the field of data science, providing high-performance, easy-to-use data structures and data analysis tools. However, when dealing

How to open a txt file in python

Two ways to open a file f = open("data.txt","r") #Setting the file object f.close() #Close file #For convenience, and to avoid forgetting to close the file object, you can use the following instead