Cache

At some point when building you application, you might need to retrieve data that is expensive to compute or fetch. When this happens, you can use a cache to store the data for a certain amount of time to improve the performance of your application.

Expanse provides a unified interface for managing caches in your application, and offer features that go beyond the basic caching capabilities, like multi-tier caching and cache stampede protection.

Expanse provides a cache system with powerful features at your disposal:

• Support for multiple cache stores.
• Multi-tier caching that uses an in-memory cache as a first layer and a persistent cache as a second layer.
• Cache stampede protection.

Configuration

The cache system is configured based on two concepts: drivers and stores. Drivers are the underlying implementations of the cache system, like Redis or a database, while stores represent a caching layer supported by one of the supported drivers. You can have multiple stores using the same driver, and each store can be configured independently.

To start using the cache system, you need to configure at least one store and its corresponding driver and reference it in the CACHE_STORE environment variable.

CACHE_STORE=default

Each store is configured via environment variables following the pattern CACHE_STORES__STORE_NAME__PARAMETER=VALUE, where STORE_NAME is the name of the store and PARAMETER is a parameter supported by the driver used by the store.

Available drivers

Redis

The Redis driver uses Redis as the underlying cache store. To use the Redis driver, you need to have at least one Redis connection configured.

CACHE_STORES__DEFAULT__DRIVER=redis
CACHE_STORES__DEFAULT__CONNECTION=default

If you want to use a separate Redis connection for locks, you can specify it via the LOCK_CONNECTION parameter:

CACHE_STORES__DEFAULT__LOCK_CONNECTION=locks

Database

The Database driver uses a database table as the underlying cache store. To use the Database driver, you need to have at least one database connection configured.

CACHE_STORES__DEFAULT__DRIVER=database
CACHE_STORES__DEFAULT__CONNECTION=default

By default, the Database driver uses a table named cache to store the cache items. You can specify a custom table name via the TABLE parameter:

CACHE_STORES__DEFAULT__TABLE=custom_cache_table

To create the cache table, you can use the make cache table command provided by Expanse:

python ./beam make cache table

It will create a migration file in the migrations directory of your application. You can then run the migration to create the cache table in your database:

python ./beam db migrate

You can specify the name of the database table when executing the make cache table by using the --table-name option:

python ./beam make cache table --table-name=custom_cache_table

If you want locks to be stored in the database as well, you can use the --with-locks-table option:

python ./beam make cache table --with-locks-table

By default, the locks will be stored in a table named cache_locks, but you can specify a custom name by passing a value to the --with-locks-table option:

The lock table name can then be references in the cache store configuration:

CACHE_STORES__DEFAULT__LOCKS_TABLE=database

Memory

The memory driver uses an LRU (Least Recently Used) in-memory cache as the underlying cache store. This driver is best suited for L1 caches in a multi-tier caching system.

CACHE_STORES__DEFAULT__DRIVER=memory
CACHE_STORES__DEFAULT__MAX_SIZE=100mb
CACHE_STORES__DEFAULT__MAXE_ITEMS=1000

Using the cache

To start using the cache in your application, you can inject a Cache instance into your routes, controllers, or any other part of your application.

from expanse.contracts.cache.asynchronous.cache import Cache


async def my_route(cache: Cache) -> Response:
    await cache.set("key", "value", 3600)

    return Response()

from expanse.contracts.cache.synchronous.cache import Cache


def my_route(cache: Cache) -> Response:
    cache.set("key", "value", 3600)

    return Response()

If you have multiple cache stores configured, you can specify which store to use by injecting an annotated Cache instance:

from typing import Annotated

from expanse.contracts.cache.asynchronous.cache import Cache


async def my_route(cache: Annotated[Cache, "redis"]) -> Response:
    await cache.set("key", "value", 3600)

from typing import Annotated

from expanse.contracts.cache.synchronous.cache import Cache


def my_route(cache: Annotated[Cache, "redis"]) -> Response:
    cache.set("key", "value", 3600)

Getting and setting items in the cache

The preferred way to retrieve and store items in the cache is by using the remember method. It tries to retrieve the item from the cache, and if it does not exist or has expired, it computes the value using a callback function, stores it in the cache, and returns it.

from typing import Sequence
from functools import partial
from sqlalchemy import select

from expanse.contracts.cache.asynchronous.cache import Cache
from expanse.database.asynchronous.session import AsyncSession
from expanse.http.response import Response
from expanse.http.helpers import json

from app.models.user import User


async def get_users(session: AsyncSession) -> Sequence[User]:
    return (await session.execute(select(User))).scalars().all()


async def retrieve_users(cache: Cache, session: AsyncSession) -> Response:
    users = await cache.remember("key", partial(get_users, session), 600)

    return json([{"id": user.id, "name": user.name} for user in users])

from typing import Sequence
from functools import partial
from sqlalchemy import select

from expanse.contracts.cache.synchronous.cache import Cache
from expanse.database.synchronous.session import Session
from expanse.http.response import Response
from expanse.http.helpers import json

from app.models.user import User


def get_users(session: Session) -> Sequence[User]:
    return session.execute(select(User)).scalars().all()


def retrieve_users(cache: Cache, session: Session) -> Response:
    users = cache.remember("key", partial(get_users, session), 600)

    return json([{"id": user.id, "name": user.name} for user in users])

The remember method accepts three parameters: the cache key, a callback function to compute the value if it does not exist or has expired, and the time-to-live (TTL) of the cache item in seconds. The TTL can be omitted if you want the item to never expire.

The main advantage of using the remember method is that it provides stampede protection out of the box.

Alternativelly, you can use the get and set methods to retrieve and store items in the cache separately.

await cache.set("key", "value", 3600)

value = await cache.get("key")

cache.set("key", "value", 3600)

value = cache.get("key")

Deleting items from the cache

To delete an item from the cache, you can use the delete method of the Cache instance.

await cache.delete("key")

cache.delete("key")

Checking if an item exists in the cache

To check if an item exists in the cache, you can use the has method of the Cache instance.

exists = await cache.has("key")

exists = cache.has("key")

Multi-tier caching

Multi-tier caching is a caching strategy that uses multiple cache stores to improve the performance of your application. The most common use case for multi-tier caching is to use an in-memory cache as a first layer (L1 cache) and a persistent cache as a second layer (L2 cache).

When using multi-tier caching, the cache system will first try to retrieve the item from the fast, in-memory L1 cache. If it does not exist or has expired, it will try to retrieve it from the L2 (Redis for instance) cache. If the item exists in the L2 cache and is still valid, it will be stored in the L1 cache and returned.

When items are stored or deleted, both cache layers are updated and a bus notifies other running application instances to evict the items from their L1 cache.

Configuration

To activate multi-tier caching, you need to configure two things: a store for the L1 cache (typically a memory store) and a bus to send cache invalidation messages between different application instances. If you have only on instance of your application running, you can skip the bus configuration.

Configuring the L1 cache

To configure the L1 cache, you need to configure a store using the memory driver and reference it in the CACHE_L1_STORE environment variable.

CACHE__STORES__DEFAULT__L1_CACHE__DRIVER=memory

You can technically use any driver for the L1 cache, but the memory driver is the best choice for this layer due to its speed.

Configuring a bus

To configure a cache invalidation bus, you need to specify the driver to use.

CACHE_STORES__DEFAULT__L1_CACHE__BUS__DRIVER=redis
CACHE_STORES__DEFAULT__L1_CACHE__BUS__CONNECTION=cache_bus

For now only two drivers are supported for the cache invalidation bus: redis and memory. The redis driver uses Redis Pub/Sub to send cache invalidation messages between different application instances, while the memory driver uses an in-memory message bus that only works within the same application instance. The memory driver is useful for development and testing purposes, but it should not be used in production.

Stampede protection

Cache stampede (also known as cache thundering) is a situation that occurs when multiple requests try to access a cache item that does not exist or has expired at the same time. This can lead to a sudden spike in traffic to the underlying data source, which can cause performance issues or even downtime.

Thankfully, Expanse provides cache stampede protection out of the box when using the remember method to retrieve items from the cache. When the first request tries to retrieve an item that does not exist or has expired, it will acquire a lock for that cache key and compute the value using the provided callback function. Meanwhile, any other request that tries to retrieve the same item will wait for the lock to be released and then try to retrieve the item from the cache again. This way, only one request will compute the value for a given cache key at a time, regardless of how many requests are processed at the same time.

Using locks

Sometimes, you might want to ensure that only one instance of your application is performing a certain action at a given time. For this purpose you can use get a lock from a Cache instance.

from expanse.contracts.cache.asynchronous.cache import Cache


cache: Cache

async def my_action(cache: Cache) -> None:
    async with cache.lock("lock-name", 30):
        # Lock acquired for 30 seconds, or until the context manager is exited
        ...

from expanse.contracts.cache.synchronous.cache import Cache


cache: Cache

def my_action(cache: Cache) -> None:
    with cache.lock("lock-name", 30):
        # Lock acquired for 30 seconds, or until the context manager is exited
        ...

The lock is acquired for a given name and a specified TTL in seconds. If the lock is already acquired by another instance of your application, the code will wait until the lock is released before acquiring it.

Available commands

A few commands are available to help you manage the cache system in your application.

cache clear

Remove all entries from the cache.

# Clear the default cache store
python ./beam cache clear

# Clear a specific cache store
python ./beam cache clear redis

cache delete

Remove specific items from the cache.

# Delete a single item from the default cache store
python ./beam cache delete users:list

# Delete multiple items from a specific cache store
python ./beam cache delete users:list posts:list comments:list --store redis