GoogleAPI service account OAuth2 authentication fails with 'Syntax error'

Question

CONTEXT: Sending mail SMTP via Google API php ‘Client’ interface and smtp.gmail.com gateway using a service account (i.e. with 2-legged OAuth2 flow).

PROBLEM: Authentication with an access token bounces with an SMTP “555 5.5.2 Syntax error”. Reports of this error on Stack Overflow relate to either the To: and/or From: email addresses not being encapsulated in < >, or the From: or user principal domain name being generic @gmail and not a private (Google workspace) domain name. Neither apply here, as will be seen.

Authentication and sending with authorization_code grant works fine.

Common to both authorization_code and service_account grant flows are:

$this->provider  = new Client();
$this->provider -> setScopes([Gmail::MAIL_GOOGLE_COM]); 
$this->provider -> setAuthConfig('gmail-xoauth2-credentials.json');
$this->provider -> useApplicationDefaultCredentials();
$this->provider -> setApplicationName(self::GOOGLEAPI_APPLICATION_NAME);
$this->provider -> addScope(['https://mail.google.com']);  

The gmail-xoauth2-credentials.json files are unaltered downloads from https://console.cloud.google.com/apis/credentials for authorization_code grant and https://console.cloud.google.com/iam-admin/serviceaccounts / Keys for service_account grant. Both pass RFC8259 verification.

Previously, for authorization_code flow only, to solicit user authentication and get a refresh token:

$this->provider -> setAccessType('offline');
$this->provider -> setApprovalPrompt('force');
$this->provider -> setPrompt('consent');

Access tokens are obtained thus:

Authorization_code grant

$tokenArray = $this->provider->fetchAccessTokenWithRefreshToken($refreshToken);
$accessToken =  $tokenArray['access_token']; 

Service_account grant

$tokenArray = $this->provider->fetchAccessTokenWithAssertion();
$accessToken =  $tokenArray['access_token'];

each of which are then base64 encoded with the sender prefix:

base64_encode (
'user=' . $SMTPEnvelopeFromAddress . "\001auth=Bearer " . $accessToken . "\001\001"
);

The service account clientID and scope of https://mail.google.com were added via domain-wide delegation to a Google Workspace account, whose primary (admin) email address is the 'own-domain' sender address used for SMTP ($SMTPEnvelopeFromAddress above).

DIAGNOSTICS: If an access token obtained through authorization_code grant is shoehorned (within its expiry time) into the service_account code, authentication (and subsequent mail sending) work. If an access token obtained through service_account grant is inserted into the authorization_code code, authentication fails with ‘Syntax error’. This would appear to exonerate the surrounding code from syntax errors.

The access tokens obtained with the two grants are very similar (decoded using https://www.googleapis.com/oauth2/v3/tokeninfo?accessToken=$accessToken). For security in what follows, sensitive digit strings have been redacted to 123435…

Authorization code

{ "azp": "123451234512-ouhovpkol8adsm3rh94273nv5kqp23pn.apps.googleusercontent.com", "aud": "123451234512-ouhovpkol8adsm3rh94273nv5kqp23pn.apps.googleusercontent.com", "scope": "https://mail.google.com/", "exp": "1704576433", "expires_in": "3458", "access_type": "offline" }

Service_account

{"azp": "123451234512345123451", "aud": "123451234512345123451", "scope": "https://mail.google.com/", "exp": "1704464536", "expires_in": "3444", "access_type": "online"}

The authorization_code access token returned from Google (i.e. before decoding) expanded by 25% when decoded as above. This makes sense: the coded access token could contain just the information specific to the transaction (essentially scope, audience and expiry time) with the remainder fleshed out from the static data held by the token server (fed in turn from Google Cloud API and IAM).

The service account access token, however, is always 1024 (!) characters, whereas after decoding it shrinks hugely to 171 (above). This ‘fat token’ oddity has been noted elsewhere on Stack Overflow (e.g. MailKit /Google OAUTH of STMP Server throws Exception "5.5.2 Syntax error, goodbye"). The long token could be overflowing a Google buffer, but since the token decodes with no problem, this seems unlikely.

The extra could be the private key that, for security, is not shown when decoded, but since it has already been used to obtain the access token, there is no point in repeating it. (Unlike Microsoft access token JWTs, Google’s are not independently decodable but need the token server to do it.)

I’m running out of ideas…
(and with apologies for a long post)