Debugging Web Authentication for Apache Airflow

Question

I am trying to enable web authentication for my gcp vm running an airflow server. I used the official docs to update the config file:

[webserver]
authenticate = True
auth_backend = airflow.contrib.auth.backends.password_auth

as well as create an user in the meta db.

$ cd ~/airflow
$ python
Python 2.7.9 (default, Feb 10 2015, 03:28:08)
Type "help", "copyright", "credits" or "license" for more information.
>>> import airflow
>>> from airflow import models, settings
>>> from airflow.contrib.auth.backends.password_auth import PasswordUser
>>> user = PasswordUser(models.User())
>>> user.username = 'new_user_name'
>>> user.email = '[email protected]'
>>> user.password = 'set_the_password'
>>> session = settings.Session()
>>> session.add(user)
>>> session.commit()
>>> session.close()
>>> exit()

The the above password section gave an error, so I ran the following command as proposed here:

user._set_password = ‘………….’.encode('utf8') 

There were no errors in the python terminal thereafter, and I assumed the user was successfully created in the meta DB. Thereafter I initialized the db , which resulted in an error:

$ airflow initdb
....
....
  File "/usr/lib/python2.7/base64.py", line 119, in urlsafe_b64decode
    return b64decode(s.translate(_urlsafe_decode_translation))
  File "/usr/lib/python2.7/base64.py", line 78, in b64decode
    raise TypeError(msg)
TypeError: Incorrect padding

I don't know why, but setting up the Fernet Key helped and I could run the initdb without any errors:

~$ airflow initdb
[2018-06-07 20:58:58,301] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/Grammar.txt
[2018-06-07 20:58:58,322] {driver.py:120} INFO - Generating grammar tables from /usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2018-06-07 20:58:58,447] {__init__.py:45} INFO - Using executor LocalExecutor
[2018-06-07 20:58:58,546] {db.py:312} INFO - Creating tables
INFO  [alembic.runtime.migration] Context impl MySQLImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
[2018-06-07 20:58:59,982] {models.py:189} INFO - Filling up the DagBag from /home/username/airflow/dags
Done.

But...when I wanted to run the webserver, it did not work:

$ airflow webserver -p 8080
....
....
Traceback (most recent call last):
  File "/usr/local/bin/airflow", line 27, in <module>
    args.func(args)
  File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 678, in webserver
    app = cached_app(conf)
  File "/usr/local/lib/python2.7/dist-packages/airflow/www/app.py", line 161, in cached_app
    app = create_app(config)
  File "/usr/local/lib/python2.7/dist-packages/airflow/www/app.py", line 52, in create_app
    app=app, config={'CACHE_TYPE': 'filesystem', 'CACHE_DIR': '/tmp'})
  File "/usr/local/lib/python2.7/dist-packages/flask_cache/__init__.py", line 121, in __init__
    self.init_app(app, config)
  File "/usr/local/lib/python2.7/dist-packages/flask_cache/__init__.py", line 156, in init_app
    from .jinja2ext import CacheExtension, JINJA_CACHE_ATTR_NAME
  File "/usr/local/lib/python2.7/dist-packages/flask_cache/jinja2ext.py", line 33, in <module>
    from flask.ext.cache import make_template_fragment_key
ImportError: No module named ext.cache

Someone pointed me to this issue:

• https://github.com/thadeusb/flask-cache/issues/142

I installed both these packages, but the issue persists:

• https://pythonhosted.org/Flask-Cache/
• https://pythonhosted.org/Flask-Caching/

Does anyone have suggestions to further troubleshoot this?