How do I force dev_appserver to reference local data?

Question

I've been using dev_appserver for years for local Python 2.7 development. The datastore is created locally and just works as expected. I'm now migrating to Cloud Datastore and the default behavior appears to be that it accesses my production datastore. Not at all what I expected, and I cannot get dev_appserver to look locally rather than to my cloud data.

The docs: https://cloud.google.com/datastore/docs/tools/datastore-emulator

... tell me to set my env vars as follows:

DATASTORE_DATASET=<my dataset>
DATASTORE_PROJECT_ID=<my project id>
DATASTORE_EMULATOR_HOST_PATH=localhost:8081/datastore
DATASTORE_HOST=http://localhost:8081
DATASTORE_EMULATOR_HOST=localhost:8081

I'm invoking dev_appserver as follows:

    python3 /home/ml/google-cloud-sdk/bin/dev_appserver.py \
    ~/appengine/apps/$1 \
    --host=dev6 \
    --enable_host_checking=false \
    --port=8082 \
    --admin_host=dev6 \
    --admin_port=8002 \
    --application=<my app> \
    --support_datastore_emulator=true \
    --enable_sendmail \
    --smtp_host=dev6 \
    --smtp_port=10025 \
    --api_port=37063

... and I've tried this also without '--support_datastore_emulator=true'.

... yet my app is still accessing my production datastore.

Can anyone tell me how this is supposed to be configured in order to use a local datastore?

Answer 1

I found this very good article on Medium, which describes specifically how to get dev_appserver.py to play nice with the Cloud Datastore Emulator. The relevant part follows:

Running the Development Sever: dev_appserver.py

You’ll need two terminals for this. In the first do:

10. Initialize the Datastore emulator.

$ gcloud beta emulators datastore start

When you see that the start up has finished, you’ll see this message.

[datastore] 
[datastore] Dev App Server is now running.
[datastore]

11. Open another terminal and run the following command to see the emulator’s and active project’s environment variables.

$ gcloud beta emulators datastore env-init

The command above will give you an output as seen below. Save these for the next steps.

export DATASTORE_DATASET=<your-project-name>
export DATASTORE_EMULATOR_HOST=localhost:8081
export DATASTORE_EMULATOR_HOST_PATH=localhost:8081/datastore
export DATASTORE_HOST=http://localhost:8081
export DATASTORE_PROJECT_ID=<your-project-name>

12. Apply the variables to this terminal:

$ $(gcloud beta emulators datastore env-init)

This need to be done in any console you want to run dev_appserver.py. Without it the Datastore GUI will not connect to the Datastore emulator instance you've just opened; it will start another.

13. Pass the same variables to the Python virtual environment inside dev_appserver.py with the following call structure:

$ dev_appserver.py \
--application=<your-project-name> \
--env_var DATASTORE_DATASET=<your-project-name> \
--env_var DATASTORE_EMULATOR_HOST=localhost:8081 \
--env_var DATASTORE_EMULATOR_HOST_PATH=localhost:8081/datastore \
--env_var DATASTORE_HOST=http://localhost:8081 \
--env_var DATASTORE_EMULATOR_HOST_PATH=localhost:8081/datastore \
app.yaml

This is also very important. Without it the libraries inside the virtual environment will not point to your emulator; they will point to the real Datastore instead.

14. Celebrate! The Datastore GUI on localhost:8000 and your app libraries should both be pointing to the emulator you opened in step 9.