403 error in Google Indexing API (after trying all solutions from other SO questions)

Question

I'm trying to use the Google Indexing API using the PHP client library.

This is my code:

    $client = new Google_Client();

    //use the private key that we created for our service account.
    $client->setAuthConfig(storage_path('google_auth_config.json'));    //this works
    $client->addScope('https://www.googleapis.com/auth/indexing');

    // Get a Guzzle HTTP Client
    $httpClient = $client->authorize();
    $endpoint = 'https://indexing.googleapis.com/v3/urlNotifications:publish';

    $content = '{
      "url": "https://myverifieddomain.com/url",
      "type": "URL_UPDATED"
    }';

    $response = $httpClient->post($endpoint, [ 'body' => $content ]);
    $status_code = $response->getStatusCode();

The auth part works.

However, the $status_code I get is 403. Here's the full $response:

object(GuzzleHttp\Psr7\Response)#2233 (6) {
  ["reasonPhrase":"GuzzleHttp\Psr7\Response":private]=>
  string(9) "Forbidden"
  ["statusCode":"GuzzleHttp\Psr7\Response":private]=>
  int(403)
  ["headers":"GuzzleHttp\Psr7\Response":private]=>
  array(11) {
    ["Vary"]=>
    array(3) {
      [0]=>
      string(8) "X-Origin"
      [1]=>
      string(7) "Referer"
      [2]=>
      string(22) "Origin,Accept-Encoding"
    }
    ["Content-Type"]=>
    array(1) {
      [0]=>
      string(31) "application/json; charset=UTF-8"
    }
    ["Date"]=>
    array(1) {
      [0]=>
      string(29) "Fri, 24 Jun 2022 10:26:02 GMT"
    }
    ["Server"]=>
    array(1) {
      [0]=>
      string(3) "ESF"
    }
    ["Cache-Control"]=>
    array(1) {
      [0]=>
      string(7) "private"
    }
    ["X-XSS-Protection"]=>
    array(1) {
      [0]=>
      string(1) "0"
    }
    ["X-Frame-Options"]=>
    array(1) {
      [0]=>
      string(10) "SAMEORIGIN"
    }
    ["X-Content-Type-Options"]=>
    array(1) {
      [0]=>
      string(7) "nosniff"
    }
    ["Alt-Svc"]=>
    array(1) {
      [0]=>
      string(162) "h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43""
    }
    ["Accept-Ranges"]=>
    array(1) {
      [0]=>
      string(4) "none"
    }
    ["Transfer-Encoding"]=>
    array(1) {
      [0]=>
      string(7) "chunked"
    }
  }
  ["headerNames":"GuzzleHttp\Psr7\Response":private]=>
  array(11) {
    ["vary"]=>
    string(4) "Vary"
    ["content-type"]=>
    string(12) "Content-Type"
    ["date"]=>
    string(4) "Date"
    ["server"]=>
    string(6) "Server"
    ["cache-control"]=>
    string(13) "Cache-Control"
    ["x-xss-protection"]=>
    string(16) "X-XSS-Protection"
    ["x-frame-options"]=>
    string(15) "X-Frame-Options"
    ["x-content-type-options"]=>
    string(22) "X-Content-Type-Options"
    ["alt-svc"]=>
    string(7) "Alt-Svc"
    ["accept-ranges"]=>
    string(13) "Accept-Ranges"
    ["transfer-encoding"]=>
    string(17) "Transfer-Encoding"
  }
  ["protocol":"GuzzleHttp\Psr7\Response":private]=>
  string(3) "1.1"
  ["stream":"GuzzleHttp\Psr7\Response":private]=>
  object(GuzzleHttp\Psr7\Stream)#2221 (7) {
    ["stream":"GuzzleHttp\Psr7\Stream":private]=>
    resource(767) of type (stream)
    ["size":"GuzzleHttp\Psr7\Stream":private]=>
    NULL
    ["seekable":"GuzzleHttp\Psr7\Stream":private]=>
    bool(true)
    ["readable":"GuzzleHttp\Psr7\Stream":private]=>
    bool(true)
    ["writable":"GuzzleHttp\Psr7\Stream":private]=>
    bool(true)
    ["uri":"GuzzleHttp\Psr7\Stream":private]=>
    string(10) "php://temp"
    ["customMetadata":"GuzzleHttp\Psr7\Stream":private]=>
    array(0) {
    }
  }
}

I've enabled the API as instructed here.

The service account is added as an owner as instructed here.

I have no idea what else I can do. The error message doesn't appear to have any additional information.

Answer 1

Fist thing to do is to figure out which Forbiden error you are getting. I would like for you to try to get the full error message back.

forbiden - Indexing API has not been used in project

I just ran this. Using a service account the full error i got back was

The service indexing has thrown an exception.
HttpStatusCode is Forbidden.
Google.Apis.Requests.RequestError
Indexing API has not been used in project 1015451674269 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/indexing.googleapis.com/overview?project=1015451674269 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry. [403]
Errors [
    Message[Indexing API has not been used in project 101574269 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/indexing.googleapis.com/overview?project=101574269 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.] Location[ - ] Reason[accessNotConfigured] Domain[usageLimits]
]

This error means that you have not enabled the indexing api under library in google cloud console. Have you done that.

Forbidden Failed to verify the URL ownership.

If i then make a request and ask for data on a url i have no access to I get.

The service indexing has thrown an exception.
HttpStatusCode is Forbidden.
Google.Apis.Requests.RequestError
Permission denied. Failed to verify the URL ownership. [403]
Errors [
    Message[Permission denied. Failed to verify the URL ownership.] Location[ - ] Reason[forbidden] Domain[global]
]

Which means the service account has no access to the url i am trying to access.

After a lot of digging I finally found the following link Grant owner status to your service account

By going to [Webmasters Verification}(https://www.google.com/webmasters/verification) I was able to add the service account and grant it access via delegation to my account.

Answer 2

See Indexing API-specific errors:

Permission denied. Failed to verify the URL ownership.
User did not complete the Ownership Verification process
or is trying to update a URL that they do not own.

However, the provided response does not state this error message -
which suggests, that this is a general HTTP 403 and not a specific one.

In order to obtain more information about the cause, impersonate the service account with roles/iam.serviceAccountUser - or filter the Cloud Logs Explorer for the service account.

What if Google hasn't crawled that URL at all, so that it cannot update it?
I'm not certain, but I would try an URL which had been indexed already.
Try to get the notification status as described here: Get notification status.

Answer 3

HTTP 403 is an HTTP status code meaning access to the requested resource is forbidden. The server understood the request, but will not fulfill it.

This is the definition of error 403, so you have to check some corners!

First of all you might check your Private and Public keys are correct and belong to your desired domain(s).

If all above were ok, check your host (or server) is not banned in GOOGLE DNS servers. You can use IP tracker / tracers do find it.

At the end, check if IPs from your country are not banned! If so, you have to use some special protocols like L2TP!