Determining if threads are actually doing any work in Java

Question

We have an application which uses Lmax Disruptor. We originally configured the Disruptor by

        this.disruptor = new Disruptor<>(
                GenericEvent::new,
                BUFFER_SIZE,
                createThreadFactory("consumer-thread"),
                ProducerType.SINGLE,
                new BlockingWaitStrategy());

        this.ringBuffer = disruptor.getRingBuffer();

        ConsumerPool consumers = eventHandlerFactory.createPool(POOL_SIZE);
        this.disruptor.handleEventsWithWorkerPool(consumers);

But from what I understand, the pool sent to handleEventsWithWorkerPool does not distribute the work to all the consumers in the pool. We saw (through JFR), that only the last thread was doing all the work.

My question is, what sort of metric could I have used to see that only one thread was doing all the work? Something that doesn't require running JFR after the application has gotten too slow?

I know if my threads are in a ThreadPool Executor, I can get lots of metrics on usage, etc.

Thanks

Answer 1

Assuming createThreadFactory("consumer-thread") is a custom method which creates threads.You should be able to give uniquename to each thread and task could log threadname when loglevel=DEBUG

class CustomThreadFactory1 implements ThreadFactory{

    @Override
    public Thread newThread(Runnable r) {
        Thread thread = new Thread(r);
        thread.setName(RandomStringUtils.random(100));//any random value
        return thread;
    }
}

Answer 2

Here's one way of monitoring and reporting how much CPU each thread is using.

package org.example;

import java.lang.management.ManagementFactory;
import java.lang.management.ThreadInfo;
import java.lang.management.ThreadMXBean;
import java.util.HashMap;
import java.util.Map;
import java.util.Random;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

public class ThreadMonitor {
    public static void main(String[] args) {
        final ThreadMXBean threadMXBean = ManagementFactory.getThreadMXBean();
        if (!threadMXBean.isThreadCpuTimeSupported()) {
            throw new RuntimeException("CPU time not supported");
        }
        threadMXBean.setThreadCpuTimeEnabled(true);
        final Map<Long,Long> cpuTimeByThread = new HashMap<>();
        ScheduledExecutorService monitor = Executors.newScheduledThreadPool(1);
        monitor.scheduleAtFixedRate(() -> {
            for (long id : threadMXBean.getAllThreadIds()) {
                long cpuTime = threadMXBean.getThreadCpuTime(id);
                Long prevCpuTime = cpuTimeByThread.put(id, cpuTime);
                if (prevCpuTime != null && cpuTime - prevCpuTime > 0) {
                    ThreadInfo info = threadMXBean.getThreadInfo(id);
                    System.out.printf("Thread %s ran %f ms%n", info.getThreadName(), (cpuTime - prevCpuTime)/1_000_000.0);
                }
            }
        }, 0, 10, TimeUnit.SECONDS);
        Executor executor = Executors.newFixedThreadPool(5);
        for (int i = 0; i <= 4; i++) {
            final int sleepTime = i * 25;
            executor.execute(() -> {
                Random r = new Random();
                long l = 0;
                while (true) {
                    try {
                        if (sleepTime > 0) {
                            Thread.sleep(sleepTime);
                        }
                        long now = System.currentTimeMillis();
                        while (System.currentTimeMillis() - now < 100) {
                            l += r.nextLong();
                        }
                    } catch (InterruptedException e) {
                        throw new RuntimeException(e);
                    }
                }
            });
        }
    }
}

I don't actually get the result I expect -- I expect the first thread in the pool to use a CPU for 100% of the time, but it reports less than 50% utilisation. Still, this does tell you which threads are working.