Matrix memory cache

This notebook demonstrates the usage of the in-memory precomputed interpolation weights cache. Please note this is only avilable for the precomputed backend (for both the high-level and and array-level regrid() methods). By default this cache is enabled, for details see In-memory weights caching.

import earthkit.regrid as ekr
from earthkit.regrid.array import regrid
import numpy as np

# create input data array for an O1280 grid
in_data = np.ones(6599680)

# helper method for regrid
def _run(n=10):
    for _ in range(n):
        regrid(in_data,
            {"grid": "O1280"},
            {"grid": [0.1,0.1]}, backend="precomputed")

In the examples below we will change the configuration multiple times. First we ensure all the changes are temporary and no options are saved into the configuration file. We also reset the configuration to the defaults.

ekr.config.autosave = False
ekr.config.reset()

The “off” cache policy

We can control the in-memory weights cache via the Configuration. In this example we turn off the in-memory cache.

ekr.config.set(weights_memory_cache_policy="off")

%time _run()

CPU times: user 9.18 s, sys: 1.54 s, total: 10.7 s
Wall time: 7.29 s

ekr.memory_cache_info()

CacheInfo(hits=0, misses=0, maxsize=0, currsize=0, count=0, policy='off')

The “largest” (default) cache policy

Now we will use the default memory cache policy, which is “largest”. For details see: Largest cache policy. Notice the roughly x9 speed-up we achieved with the caching.

ekr.config.set(weights_memory_cache_policy="largest")

%time _run()

CPU times: user 728 ms, sys: 77.5 ms, total: 805 ms
Wall time: 806 ms

ekr.memory_cache_info()

CacheInfo(hits=9, misses=1, maxsize=524288000, currsize=259170724, count=1, policy='largest')

Clearing the cache

We can clear the cache with clear_memory_cache().

ekr.clear_memory_cache()
ekr.memory_cache_info()

CacheInfo(hits=0, misses=0, maxsize=524288000, currsize=0, count=0, policy='largest')