SPFX use aadHttpClientFactory to access Blob directly?

Question

I have managed to use an Azure managed identity to access a function securely using aadHTTPClient, and managed identity with the "user_impersonation" api exposed.

But can you use a managed identity to access a Blob API point directly, without using a SAAS token? Can you pass credentials directly onto Blob storage from spfx?

Here is what I have done:

1. Created the app registration, and added the Azure Storage/user_impersonation permission
2. Exposed the api with "User_impersonation"
3. On the storage account, granted the app registration "storage blob data contributer" role
4. Added the resource and scope "user_Impersonation" to the package-solution.json
5. Used the aadHttpClientFactory in spfx to get the client of the api I exposed in (2)
6. Built and uploaded to app catalogue, then approve request in sharepointadmin "api"

These are broadly the steps to call the Azure function which works - but doesn't work when calling the Blob directly over http.

The closest I get is an error about the scope not matching. The scope embedded in the token is to the api://GUID (which is the same when calling function http endpoints).

Is this even possible? Can you use AADCLientFactory to access blob endpoints directly using the browser?

Thanks