_pickle.UnpicklingError: invalid load key, '9'

[fili]

using blueprints, FileSystemCache on Google App Engine Standard Python3.7 and having the cache separate:

cache.py:

from flask_caching import Cache
cache = Cache()

config:

CACHE_CONFIG = {
    'CACHE_TYPE': 'filesystem',
    'CACHE_DIR': '/tmp',
    'CACHE_THRESHOLD': 1000,
    'CACHE_DEFAULT_TIMEOUT': 3600,
}

app.py:

app = Flask(__name__)
cache.init_app(app, config.CACHE_CONFIG)
app.config.from_object(flask_config)

blueprint:

from cache import cache 

@public.route('/<lang>/', methods=['GET'])
@cache.cached()
def page(lang):
    ...

Then getting this error:

ERROR:root:Traceback (most recent call last):
  File "/tmp/tmphwgdsy/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/tmp/tmphwgdsy/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/tmp/tmphwgdsy/lib/python3.6/site-packages/flask_caching/__init__.py", line 400, in decorated_function
    timeout=decorated_function.cache_timeout,
  File "/tmp/tmphwgdsy/lib/python3.6/site-packages/flask_caching/backends/filesystem.py", line 160, in set
    self._prune()
  File "/tmp/tmphwgdsy/lib/python3.6/site-packages/flask_caching/backends/filesystem.py", line 109, in _prune
    expires = pickle.load(f)
_pickle.UnpicklingError: invalid load key, '9'.

[sh4nks]

Can you reproduce this error without Google App Engine?

[sh4nks]

Can't reproduce on my local machine.

[dzpt]

@sh4nks
same issue, it occurs randomly, it's hard to reproduce

_pickle.UnpicklingError
_pickle.UnpicklingError: invalid load key, '\x00'.

Traceback (most recent call last)
File "//**/python//venv/lib/python3.8/site-packages/flask/app.py", line 2463, in call
return self.wsgi_app(environ, start_response)
File "//**/python//venv/lib/python3.8/site-packages/flask/app.py", line 2449, in wsgi_app
response = self.handle_exception(e)
File "//**/python//venv/lib/python3.8/site-packages/flask/app.py", line 1866, in handle_exception
reraise(exc_type, exc_value, tb)
File "//**/python//venv/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "//**/python//venv/lib/python3.8/site-packages/flask/app.py", line 2446, in wsgi_app
response = self.full_dispatch_request()
File "//**/python//venv/lib/python3.8/site-packages/flask/app.py", line 1951, in full_dispatch_request
rv = self.handle_user_exception(e)
File "//**/python//venv/lib/python3.8/site-packages/flask/app.py", line 1820, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "//**/python//venv/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "//**/python//venv/lib/python3.8/site-packages/flask/app.py", line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "//**/python//venv/lib/python3.8/site-packages/flask/app.py", line 1935, in dispatch_request
return self.view_functionsrule.endpoint
File "//**/python//venv/lib/python3.8/site-packages/flask_login/utils.py", line 261, in decorated_view
return func(args, **kwargs)
File "///python//base/admin/dashboard.py", line 22, in dashboard
loadavg, boot_time, boot_time_days = check_loadavg()
File "///python//venv/lib/python3.8/site-packages/flask_caching/init.py", line 838, in decorated_function
self.cache.set(
File "///python//venv/lib/python3.8/site-packages/flask_caching/backends/filesystem.py", line 177, in set
self._prune()
File "///python/*/venv/lib/python3.8/site-packages/flask_caching/backends/filesystem.py", line 126, in _prune
expires = pickle.load(f)
_pickle.UnpicklingError: invalid load key, '\x00'.

here is code from load_avg()

@cache.memoize(timeout=300)
def check_loadavg() -> Tuple[List[str], str, int]:
    try:
        loadavg = []
        for lavg in psutil.getloadavg():
            loadavg.append("{0:.2f}".format(lavg))
        boot_time = util.readable_unixtime(psutil.boot_time())
        boot_time_days = math.ceil(
            (util.unixtime() - psutil.boot_time()) / 86400)
        return loadavg, boot_time, boot_time_days
    except Exception:
        return [], 'NAN', 0

[gergelypolonkai]

@dzpt are you on GAE, too?

[dzpt]

@gergelypolonkai no, i was running it on localhost

[gergelypolonkai]

@dzpt is there a chance you can share with us the pickled value that made the pickle library throw up? Like reading it directly from from the cache or something. Iʼm trying to devise a failing test to this bug.

[anubhav-nd]

Hi,
Is there any update on this error? I too have been facing this but at random frequencies and unable to reproduce it consistently.