Setup:
- Springboot & Kotlin app hosted in GCP GKE
- Above app receives data from AMQP/rabbitMQ service
- Above app receives AMQP data and publishes into Google cloud PubSub , GCP GKE and PubSub are in different GCP projects , data published using service account
- PubSub Push subscription - Cloud function . Memory 512MB, FirstGen, Maximum 3000 instances
Problem: Sometimes when there is a spike in AMQP messages seeing delays in publish around 20-60 seconds. For instance after receiving the message from AMQP , time taken to publish is around 20-60 seconds.
Questions:
- Is my way of logging correct, does publish time means publishing to push subscription or invoked cloud function completion.
- What do i need to do to increase the throughput? These messages are time sensitive means they need to be delivered as soon as its created.
- Is there something i need to consider to scale when there is a spike . like horizontal scaling , multiple publish clients ,etc.,
fun publishCode(receivedTime:LocalDateTime){
PubsubMessage.newBuilder()
.setData(ByteString.copyFrom(message.body))
.build()?.let {
val apiFuture: ApiFuture<String> = publisher.publish(it) //publisher created using bean
ApiFutures.addCallback(apiFuture, object : ApiFutureCallback<String> {
override fun onSuccess(messageID: String?) {
val publishedTime = getCurrentTime()
log.info("Published success : Received:$receivedTime Published:$publishedTime"
}
})
}
}
@Bean
fun publisher(): Publisher {
return Publisher.newBuilder("$topicToPublish").setRetrySettings(retrySettings)
.build()
}