I set up an Azure key Vault with a private endpoint and no public access. I also enabled the option to "Allow trusted Microsoft services to bypass this firewall". However, I can't disable this option from the Azure portal now by keeping remain no public access. The event log shows a Bad Request error.
I have configured this key vault to use role-based access control and assigned myself four roles, as you can see in the screenshot below.
Where did I make a mistake?
I have finally discovered the cause of the firewall bypass issue. It is related to a dependency that key vault firewall bypass has on the ARM Template deployment option. When this option is enabled, the service tries to access secrets from the Azure Devops task ARM Template Deployment.