Micrometer with Prometheus Pushgateway - Add TLS Support

Question

I have a Spring boot application with Prometheus Pushgateway using Micrometer, mainly based on this tutorial: https://luramarchanjo.tech/2020/01/05/spring-boot-2.2-and-prometheus-pushgateway-with-micrometer.html

pom.xml has following related dependencies:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-core</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
    <groupId>io.prometheus</groupId>
    <artifactId>simpleclient_pushgateway</artifactId>
    <version>0.16.0</version>
</dependency>

And application.properties file has:

management.metrics.export.prometheus.pushgateway.enabled=true
management.metrics.export.prometheus.pushgateway.shutdown-operation=PUSH
management.metrics.export.prometheus.pushgateway.baseUrl=localhost:9091

It is working fine locally in Dev environment while connecting to Pushgateway without any TLS. In our CI environment, Prometheus Pushgateway has TLS enabled. How do I configure TLS support and configure certs in this Spring boot application?

Answer 1

Due to the usage of TLS, you will need to customize a few Spring classes:

• HttpConnectionFactory -> PushGateway -> PrometheusPushGatewayManager

A HttpConnectionFactory, is used by prometheus' PushGateway to create a secure connection, and then, create a PrometheusPushGatewayManager which uses the previous pushgateway.

You will need to implement the prometheus’ interface HttpConnectionFactory, I’m assuming you are able to create a valid javax.net.ssl.SSLContext object (if not, more details in the end¹).

HttpConnectionFactory example:

public class MyTlsConnectionFactory implements io.prometheus.client.exporter.HttpConnectionFactory {
    @Override
    public HttpURLConnection create(String hostUrl) {
        // considering you can get javax.net.ssl.SSLContext or javax.net.ssl.SSLSocketFactory
        URL url = new URL(hostUrl);
        HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
        connection.setSSLSocketFactory(sslContext.getSocketFactory());
        return connection;
    }
}

PushGateway and PrometheusPushGatewayManager:

@Bean 
public HttpConnectionFactory tlsConnectionFactory() {
    return new MyTlsConnectionFactory();
}

@Bean 
public PushGateway pushGateway(HttpConnectionFactory connectionFactory) throws MalformedURLException {
    String url = "https://localhost:9091"; // replace by your props
    PushGateway pushGateway = new PushGateway(new URL(url));
    pushGateway.setConnectionFactory(connectionFactory); 
    return pushGateway;
}

@Bean 
public PrometheusPushGatewayManager tlsPrometheusPushGatewayManager(PushGateway pushGateway,
                                                                    CollectorRegistry registry) {
    // fill the others params accordingly (the important is pushGateway!)
    return new PrometheusPushGatewayManager(
            pushGateway,
            registry,
            Duration.of(15, ChronoUnit.SECONDS),
            "some-job-id",
            null,
            PrometheusPushGatewayManager.ShutdownOperation.PUSH
    );
}

---

¹If you face difficulty retrieving the SSLContext from java code, I recommend studying the library https://github.com/Hakky54/sslcontext-kickstart and https://github.com/Hakky54/mutual-tls-ssl (which shows how to apply it with different client libs).

Then, will be possible to generate SSLContext in java code in a clean way, e.g.:

String keyStorePath = "client.jks";
char[] keyStorePassword = "password".toCharArray();
SSLFactory sslFactory = SSLFactory.builder()
        .withIdentityMaterial(keyStorePath, keyStorePassword)
        .build();
javax.net.ssl.SSLContext sslContext = sslFactory.getSslContext();

Finally, if you need setup a local Prometheus + TLS environment for testing purposes, I recommend following the post: https://smallstep.com/hello-mtls/doc/client/prometheus