Google homegraph API returns 404 entity not found on sync request - initial sync intent not received

Question

I am having an issue with the initial account linking and sync with google home and syncing with the homegraph API. I cannot get an initial sync intent on retrieving a valid token (which I can use to query other API). HomeGraph API scope is not valid for users so I have a service account token creator set up to make requests to the homegraph API; this service account can make requests to home graph but since I never receive the initial sync intent, a request sync just returns { "error": { "code": 404, "message": "Requested entity was not found.", "status": "NOT_FOUND" } }

My application is written in python utilizing flask and ngrok (for development) to tunnel my local host.

As mentioned, I have completed the OAuth2 flow for the user (using scopes 'openid', 'https ://www.googleapis.com/auth/userinfo.profile', 'https ://www.googleapis.com/auth/userinfo.email'). I cannot use https: //www.googleapis.com/auth/homegraph here as this scope is restricted from users. (spaces to avoid looking like spam)

Instead, when handling intents or requests to google HomeGraph, I use a service account.

I complete the OAuth2 flow, then can use the token to access the defined user scopes. I complete the service account auth flow and can query the HomeGraph API (I can see the API in my project receiving requests and returning errors in the cloud console) but I have not achieved linking the user account to the service.

My service does not link at the end of this flow in GHA using works with google to connect, no initial sync intent is sent to my fulfillment endpoint, and so my devices linked through this service do not sync initially and any request sync with an agent_user_id will return 404 entity not found. No initial sync intent means no agent_user_id associated with the service and no devices can be associated with a non existent agent_user_id

function for requesting sync:

def request_sync():
        creds = service_account.Credentials.from_service_account_file(
            'filepath',
            scopes=['https://www.googleapis.com/auth/homegraph']
        )

        auth_request = google.auth.transport.requests.Request()
        creds.refresh(auth_request)
        # Define the URL for the HomeGraph API endpoint
        url = "https://homegraph.googleapis.com/v1/devices:requestSync"

        # Define the headers for the request
        headers = {
            "Authorization": "Bearer " + creds.token,
            "Content-Type": "application/json"
        }
         # Generate a unique requestId
        request_id = str(uuid.uuid4())

        agent_user_id = session.get('agent_user_id')
        # Define the body of the request
        body = {
            "agentUserId": agent_user_id,
            "async": True  # Set this to False if you want a synchronous request
        }
            # Print the body of the request for debugging
        print("Request Body:")
        print(json.dumps(body, indent=4))

        # Send the POST request
        response = requests.post(url, headers=headers, json=body)

        # Check if the request was successful
        if response.status_code == 200:
            print("Sync request was successful.")
        else:
            print("Failed to send sync request: {}".format(response.content))
            
            # Print more detailed error information for debugging
            print("Response Status Code: {}".format(response.status_code))
            print("Response Text: {}".format(response.text))
            try:
                print("Response JSON: {}".format(json.dumps(response.json(), indent=4)))
            except ValueError:
                print("Response could not be parsed as JSON.")
            return response.text

intent handling from fulfillment url (not fully implemented, looking at sync first before straightening out):

 @app.route('/', methods=['POST'])
    def handle_request():
        global current_scene  # Declare the variable as global
        global agent_user_id
        data = request.get_json()
        intent = data['inputs'][0]['intent']

        if intent == 'action.devices.SYNC':
            print(intent)
            redirect(url_for('sync'))

      

        return json.dumps({"status": "success"})

OAuth 2 flow for user:

@app.route('/callback')
    def callback():
        print('Callback route was called')
        global session
        # Specify the state when creating the flow in the callback so that it can
        # verified in the authorization server response.
        #Many flask functions as imported at top
        state = session['state']

        flow = google_auth_oauthlib.flow.Flow.from_client_secrets_file(
            CLIENT_SECRETS_FILE, scopes=SCOPES, state=state)
        flow.redirect_uri = "https://903e-2607-fea8-ff01-7fe9-5c8f-a50d-dd11-2c8.ngrok-free.app/callback"

        # Use the authorization server's response to fetch the OAuth 2.0 tokens. (this url is the response from google in our authrisation function where it redirects to google.  Google then responds to this oauth2 callback so we are passing that authorisation response exactly to get the token for making authenticated requests)
        authorization_response = request.url
        print(state)
        flow.fetch_token(authorization_response=authorization_response)

        # Store credentials in the session.
        credentials = flow.credentials
        # Convert credentials to a dictionary and store it back in credentials
        credentials = credentials_to_dict(credentials)

        session['credentials'] = credentials
        # Set agent_user_id in session after OAuth 2.0 flow is complete.
       
        user_info = get_user_info(credentials['token'])
        session['agent_user_id'] = user_info['sub']  # unique id for signed in account
        print(session['agent_user_id'])
        return request_sync()



    @app.route('/authorize')
    def authorize():
        print('Authorize route was called')
        # Create flow instance to manage the OAuth 2.0 Authorization Grant Flow steps.
        flow = google_auth_oauthlib.flow.Flow.from_client_secrets_file(
            CLIENT_SECRETS_FILE, scopes=SCOPES)

        
        flow.redirect_uri = "https://903e-2607-fea8-ff01-7fe9-5c8f-a50d-dd11-2c8.ngrok-free.app/callback"
        #creates auth url with link to google and returns it as well as the included state to variables
        authorization_url, state = flow.authorization_url(
        
        access_type='offline',
       
        include_granted_scopes='true',
        prompt='consent')

        global session
        session['state'] = state
        print(state)
       
        return redirect(authorization_url)


    def credentials_to_dict(credentials):
        return {
            'token': credentials.token,
            'refresh_token': credentials.refresh_token,
            'token_uri': credentials.token_uri,
            'client_id': credentials.client_id,
            'client_secret': credentials.client_secret,
            'scopes': credentials.scopes
        }

Sync intent handling snippet:

#for handling google requests
    app = Flask(__name__)
    app.secret_key= secrets.token_hex()

    @app.route('/', methods=['POST'])
    def handle_request():
        global current_scene  # Declare the variable as global
        global agent_user_id
        data = request.get_json()
        intent = data['inputs'][0]['intent']

        if intent == 'action.devices.SYNC':
            print(intent)
            redirect(url_for('sync'))

and sync intent response were it to be called (it is not triggered currently)

 @app.route('/sync')
    def sync():
        creds = service_account.Credentials.from_service_account_file(
            'filepath',
            scopes=['https://www.googleapis.com/auth/homegraph']
        )

        auth_request = google.auth.transport.requests.Request()
        creds.refresh(auth_request)
        # Define the URL for the HomeGraph API endpoint
        url = "https://homegraph.googleapis.com/v1/devices:sync"

        scenes = []
        with open('Scenes.txt', 'r') as file:
            for line in file:
                scene = json.loads(line.strip())  
                scenes.append(scene)

        # Generate a unique requestId
        request_id = str(uuid.uuid4())
        # Define the headers for the request
        headers = {
            "Authorization": "Bearer " + creds.token,
            "Content-Type": "application/json"
        }

        agent_user_id = session.get('agent_user_id')
        # Define the body of the response
        body = {
            "requestId": request_id,
            "payload": {
                "agentUserId": agent_user_id,
                "devices": scenes
            }
        }
        # Return the response
        requests.post(url, headers=headers, json=body) # Send the POST request
        print (body)
        return body

and finally, the error:

Failed to send sync request: b'{\n  "error": {\n    "code": 404,\n    "message": "Requested entity was not found.",\n    "status": "NOT_FOUND"\n  }\n}\n'
Response Status Code: 404
Response Text: {
  "error": {
    "code": 404,
    "message": "Requested entity was not found.",
    "status": "NOT_FOUND"
  }
}

Response JSON: {
    "error": {
        "code": 404,
        "message": "Requested entity was not found.",
        "status": "NOT_FOUND"
    }
}

If anybody can help me understand what I am missing I'd be a very happy dev! Thanks :)

Answer 1

The error is in my approach to authenticating with google. Authentication with and requests to homegraph api work in a different manner than user-scoped apis. I did not understand the meaning of this on first approach and ended up mixing google user account auth access flow (for querying personal google account information) with use of my authorized service account to hit the homegraph api separately.

For anyone else in my shoes trying to wrap their head around this flow:

You need to implement user accounts specific to your application (creation and login). Homegraph sees the service/application you're building as its own entity with its own accounts and homegraph keeps record itself of what google account is associated with what user account under your service (assuming you properly complete the oauth flow for registering a new 'works with google' service/app through the google home app).

The Oauth flow to link your google account to your in service/app account through your service account requires logging into your own created account on your service/app (nothing to do with google- your own user file/db etc) through the works with google section in google home. This will start the flow at your authorize endpoint with the proper params, you can redirect with the code and state, then (IMPORTANT) assuming the auth code and state match at your token endpoint, generate a token and return for google to use to authenticate requests with your app instead of signing in manually again every time.

To summarize, homegraph api will interact with all users through your service account. Assuming login is successful and a token is properly generated/provided to google, the google home app registers that your application user is associated with your google account (the one logged into google home app) in the homegraph via your service account (each service account is only scoped to access its own data so all services, their associated accounts/devices, etc remain separate and singular to their account owner regardless of how many users a service has- or how many services a user has linked to google home).

As a note, This requires previously setting up your cloud project and actions project with the appropriate apis/service accounts in the cloud console and configuring the actions project with account linking pertaining to your service account, and urls to your auth, token, valid redirect uri if in use.

Answer 2

You cannot issue request sync or other Homegraph calls before completing account linking for user and handling initial sync request. The error received for Request sync is indeed expected at this point. You should focus on implementing account linking and handling initial sync intent(to complete the account linking process)