Parse bytes stored using Flask-Caching

Question

I used Flask-Caching to cache the response of a Flask view in Redis. Getting the cached data directly from Redis returns some bytes. How can I parse this in Python to examine the cached value?

b'!\x80\x03cflask.wrappers\nResponse\nq\x00)\x81q\x01}q\x02(X\x07\x00\x00\x00headersq\x03cwerkzeug.datastructures\nHeaders\nq\x04)\x81q\x05}q\x06X\x05\x00\x00\x00_listq\x07]q\x08X\x0c\x00\x00\x00Content-Typeq\tX\x10\x00\x00\x00application/jsonq\n\x86q\x0basbX\x0c\x00\x00\x00_status_codeq\x0cK\xc8X\x07\x00\x00\x00_statusq\rX\x06\x00\x00\x00200 OKq\x0eX\x12\x00\x00\x00direct_passthroughq\x0f\x89X\t\x00\x00\x00_on_closeq\x10]q\x11X\x08\x00\x00\x00responseq\x12Xn\x00\x00\x00[\n  {\n    "desc": "pronoun object", \n    "tag": "CLO"\n  }, \n  {\n    "desc": "pronoun", \n    "tag": "CLS"\n  }\n]q\x13X\x01\x00\x00\x00\nq\x14\x86q\x15ub.'

Answer 1

Flask-Cache uses the RedisCache backend provided by Werkzeug, which serializes values using pickle.dumps. It also prepends a ! for help when deserializing. You should typically not mess with these values directly, and let Flask-Caching handle it.

You can use pickletools.dis to safely examine the representation, then pickle.loads to deserialize it. Security note: pickle.loads can execute arbitrary code, so make sure you understand untrusted data first using pickletools.dis.

>>> data = b'!\x80\x03cflask.wrappers\nResponse\nq\x00)\x81q\x01}q\x02(X\x07\x00\x00\x00headersq\x03cwerkzeug.datastructures\nHeaders\nq\x04)\x81q\x05}q\x06X\x05\x00\x00\x00_listq\x07]q\x08X\x0c\x00\x00\x00Content-Typeq\tX\x10\x00\x00\x00application/jsonq\n\x86q\x0basbX\x0c\x00\x00\x00_status_codeq\x0cK\xc8X\x07\x00\x00\x00_statusq\rX\x06\x00\x00\x00200 OKq\x0eX\x12\x00\x00\x00direct_passthroughq\x0f\x89X\t\x00\x00\x00_on_closeq\x10]q\x11X\x08\x00\x00\x00responseq\x12Xn\x00\x00\x00[\n  {\n    "desc": "pronoun object", \n    "tag": "CLO"\n  }, \n  {\n    "desc": "pronoun", \n    "tag": "CLS"\n  }\n]q\x13X\x01\x00\x00\x00\nq\x14\x86q\x15ub.'
>>> value = value[1:]  # strip leading !

>>> import pickletools
>>> pickletools.dis(data)
        0: \x80 PROTO      3
    2: c    GLOBAL     'flask.wrappers Response'
   27: q    BINPUT     0
   29: )    EMPTY_TUPLE
   30: \x81 NEWOBJ
   31: q    BINPUT     1
   33: }    EMPTY_DICT
   34: q    BINPUT     2
   36: (    MARK
   37: X        BINUNICODE 'headers'
   49: q        BINPUT     3
   51: c        GLOBAL     'werkzeug.datastructures Headers'
   84: q        BINPUT     4
   86: )        EMPTY_TUPLE
   87: \x81     NEWOBJ
   88: q        BINPUT     5
   90: }        EMPTY_DICT
   91: q        BINPUT     6
   93: X        BINUNICODE '_list'
  103: q        BINPUT     7
  105: ]        EMPTY_LIST
  106: q        BINPUT     8
  108: X        BINUNICODE 'Content-Type'
  125: q        BINPUT     9
  127: X        BINUNICODE 'application/json'
  148: q        BINPUT     10
  150: \x86     TUPLE2
  151: q        BINPUT     11
  153: a        APPEND
  154: s        SETITEM
  155: b        BUILD
  156: X        BINUNICODE '_status_code'
  173: q        BINPUT     12
  175: K        BININT1    200
  177: X        BINUNICODE '_status'
  189: q        BINPUT     13
  191: X        BINUNICODE '200 OK'
  202: q        BINPUT     14
  204: X        BINUNICODE 'direct_passthrough'
  227: q        BINPUT     15
  229: \x89     NEWFALSE
  230: X        BINUNICODE '_on_close'
  244: q        BINPUT     16
  246: ]        EMPTY_LIST
  247: q        BINPUT     17
  249: X        BINUNICODE 'response'
  262: q        BINPUT     18
  264: X        BINUNICODE '[\n  {\n    "desc": "pronoun object", \n    "tag": "CLO"\n  }, \n  {\n    "desc": "pronoun", \n    "tag": "CLS"\n  }\n]'
  379: q        BINPUT     19
  381: X        BINUNICODE '\n'
  387: q        BINPUT     20
  389: \x86     TUPLE2
  390: q        BINPUT     21
  392: u        SETITEMS   (MARK at 36)
  393: b    BUILD
  394: .    STOP
highest protocol among opcodes = 2

>>> import pickle
>>> response = pickle.loads(data)
>>> response.content_type
'application/json'
>>> response.json
[{'desc': 'pronoun object', 'tag': 'CLO'}, {'desc': 'pronoun', 'tag': 'CLS'}]