How to publish a message to google pub/sub without the need of service account credentials json required at the start of the application?

Question

I built a gradle application which publishes a message to google pub/sub. I have created a service account on GCP and downloaded the credentials json file. And I have created Topic and subscription on GCP and in subcription, I have added my service account name as the principal and gave pub/sub editor role. Here are my gradle application details.

Gradle dependencies: implementation group: 'org.springframework.cloud', name: 'spring-cloud-gcp-starter-pubsub',version: '1.2.8.RELEASE'

implementation 'org.springframework.cloud:spring-cloud-gcp-starter:1.2.0.RELEASE'

application.yaml: spring: profiles: active: myEnv application: name: pubsub cloud: gcp: project-id: project_id credentials: location: classpath:credentials.json

I have placed this credentials.json in src/main/resources.

Here is my Service class.

public String publishMessageToPubSub() throws IOException, InterruptedException, ExecutionException {
        String response = "Success";
        Publisher publisher = null;
        boolean isShutdown = false;
        try {           
                    
            //pubsubProperties.getGcp_config() : This piece of code is getting the credJsonString from AWS paramstore
            InputStream inputStream = new ByteArrayInputStream(pubsubProperties.getGcp_config().getBytes());

            CredentialsProvider credentialsProvider = FixedCredentialsProvider
                    .create(GoogleCredentials.fromStream(inputStream));

            ProjectTopicName topic = ProjectTopicName.of("project_id","topic_id");

            publisher = Publisher.newBuilder(topic).setCredentialsProvider(credentialsProvider).build();

            String messageToBePublished = "Hello World";

            byte[] encodedBytes = Base64.getEncoder().encode(messageToBePublished.getBytes());
            String encodedMessage = new String(encodedBytes);

            ByteString data = ByteString.copyFromUtf8(encodedMessage);
            PubsubMessage pubsubMessage = PubsubMessage.newBuilder().setData(data).build();

            ApiFuture<String> messageIdFuture = publisher.publish(pubsubMessage);
            String messageId = null;

            try {
                messageId = messageIdFuture.get();
                log.info("Message Published Successfully with message Id : " + messageId);
            } catch (Exception e) {
                
                throw e;
            }

            publisher.shutdown();
            isShutdown = true;
        } catch (HttpStatusCodeException ioe) {
            throw ioe;
        } catch (Exception e) {
            throw e;
        } finally {
            if (publisher != null && !isShutdown) {
                // When finished with the publisher, shutdown to free up resources.
                publisher.shutdown();
                publisher.awaitTermination(1, TimeUnit.MINUTES);
            }
        }
    

        return response;
    }

This code is working fine and the message is getting published in the GCP while the credentials.json file is in the src/main/resources.

As a security concern, we cannot put this file in the github and this application is to be deployed through Jenkins pipeline taking the git branch to the ECS cluster. But,the application is not starting and is throwing the below error right after the run method is executed if the file is not present.

java.io.IOException: The Application Default Credentials are not available. They are available if running in Google Compute Engine. Otherwise, the environment variable GOOGLE_APPLICATION_CREDENTIALS must be defined pointing to a file defining the credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.

It is expecting the spring.cloud.gcp.project-id and spring.cloud.gcp.credentials.location properties and is trying to find the location of the credentials.json and is using the content to instantiate some PubSub classes.

And this java application is deployed to ECS cluster.

Please suggest me a way that doesn't need the credentials.json path to start the application as I can manage to get the content of the json to create the GoogleCredentials in the code. Or is there any other way of authentication that doesn't need credentials.json file.

Answer 1

You need to add service account ( application which will read message from pubsub)in ur project. And add gcp publisher access to that service account.

So you can publish on the tipic without credentials