How can I use my App Engine API to run an executable on Compute Engine?

Question

My project structure is roughly as follows:

• Frontend - built with Svelte, SvelteKit, and hosted on App Engine
• API - built with FastAPI, also hosted on App Engine (*1)
• Backend - proprietary code/executable stored on a Compute Engine VM

I will use a simplified example, but it is basically analogous to what is happening under the hood.

The frontend is a simple web page with a form and a button. The form is two inputs long and accepts two numbers, and the button sends these two numbers as a JSON to the API. (call this numbers.json)

The API checks that everything is fine (correct formatting, etc), and saves numbers.json to the VM. (*2) Then, the API runs our in house executable, addition, which reads numbers.json, adds the two numbers together, and produces an output file called sum.txt.

*1 : In development, the frontend server and API server were being hosted locally and simultaneous with distinct ports. Does this set up translate to App Engine applications/APIs?

*2 : In this stage in development, saving the data on the VM is fine, but the pipeline will eventually be set up to save the data to Cloud Storage

What is common practice for hosting APIs for App Engine applications and having an App Engine API communicate with Compute Engine? In the given example, how can I use my FastAPI API to send a JSON to a Compute Engine VM and run an executable?

I am thinking that my current solution is to use VPC Peering to connect App Engine to Compute Engine. This has not been implemented yet, since I am still exploring my options. Can Endpoints communicate with Compute Engine in any way?

Answer 1

In development, the frontend server and API server were being hosted locally and simultaneous with distinct ports. Does this set up translate to App Engine applications/APIs?

In GAE, you can have multiple services (essentially multiple Apps) under a single project and then you use a dispatch.yaml file to route traffic to them i.e. your dispatch.yaml will say

a) If url is example.com, route to the front end service

b) If url is api.example.com, route to the API service

What is common practice for hosting APIs for App Engine applications and having an App Engine API communicate with Compute Engine?

One way of doing this is - you have a web based service on the VM which when triggered invokes the executable i.e. there's a URL that you load and the code handling visits to this URL then triggers the executable. If you have this, then your GAE App simply invokes the URL using the Request package or Urllib.Request or however you want to invoke URLs programmatically.

In addition to the above, you might then want to deal with things like making sure your VM only allows traffic from your App Engine App (i.e. the public can't access your VM). You can look into Serverless VPC Access