I have a use-case of a Spring app where in multiple points in the code I split the work into disjoint workers taken from a fixed thread pool. The parent thread passes any shared variables to the workers, including a CountDownLatch which they count down when done, and waits on the CountDownLatch until termination of the workers.
Here is an example from the code:
ExecutorService workers = Executors.newFixedThreadPool(numWorkers);
CountDownLatch latch = new CountDownLatch(numWorkers);
for (File csv : csvs) {
workers.submit(new CSVFileDBPersister(db, csv, csvReader, latch));
}
try {
latch.await();
} catch (InterruptedException ignored) {}
CSVFileDBPersister is a Runnable whose run() method reads a particular .csv file, persists its contents into a DB, and then counts down on the latch. In the above use-case, I split the work across numWorkers workers to make things a bit more efficient with the parsing and persistence of the files. I use the exact same pattern in other positions of the code, with different Runnables that do different things.
What I'm worried about is that in the Spring console, every time I have a use-case that splits work between threads from a fixed thread pool, I'm seeing that the index of the thread pool increases by 1.
.
.
.
2023-09-06T01:21:36.331+03:00 INFO 92410 --- [pool-2-thread-1] c.x.c.persistence.CryptoPricePersister : Inserted all prices for crypto: ETH
2023-09-06T01:21:36.385+03:00 INFO 92410 --- [pool-2-thread-5] c.x.c.util.logger.ComponentLogger : Completed the call DatabaseConnectionImpl.insertAllCryptoPrices(..) with arguments: BTC, [CryptoPrice(timestamp=2022-01-01 06:00:00.0, price=46813.21), CryptoPrice(timestamp=2022-01-01 09:00:00.0, price=46979.61), CryptoPrice(timestamp=2022-01-01 12:00:00.0, price=47143.98), CryptoPrice(timestamp=2022-01-01 13:00:00.0, price=46871.09), CryptoPrice(timestamp=2022-01-01 16:00:00.0, price=47023.24), CryptoPrice(timestamp=2022-01-02 02:00:00.0, price=47722.66), CryptoPrice(timestamp=2022-01-03 00:00:00.0, price=47017.98), CryptoPrice(timestamp=2022-01-03 04:00:00.0, price=47116.22), CryptoPrice(timestamp=2022-01-03 23:00:00.0, price=45922.01), CryptoPrice(timestamp=2022-01-04 17:00:00.0, price=47336.98), CryptoPrice(timestamp=2022-01-05 06:00:00.0, price=46422.56), CryptoPrice(timestamp=2022-01-05 10:00:00.0, price=46407.5), CryptoPrice(timestamp=2022-01-05 12:00:00.0, price=46858.93), CryptoPrice(timestamp=2022-01-05 17:00:00.0, price=46689.52), CryptoPrice(timestamp=2022-01-05 20:00:00.0, price=46010.72), CryptoPrice(timestamp=2022-01-06 20:00:00.0, price=43005.3), CryptoPrice(timestamp=2022-01-06 21:00:00.0, price=43369.04), CryptoPrice(timestamp=2022-01-07 00:00:00.0, price=43120.63), CryptoPrice(timestamp=2022-01-07 11:00:00.0, price=42088.61), CryptoPrice(timestamp=2022-01-07 13:00:00.0, price=42172.84), CryptoPrice(timestamp=2022-01-08 03:00:00.0, price=41991.89), CryptoPrice(timestamp=2022-01-08 06:00:00.0, price=41810.92), CryptoPrice(timestamp=2022-01-09 09:00:00.0, price=41873.62), CryptoPrice(timestamp=2022-01-09 21:00:00.0, price=41966.99), CryptoPrice(timestamp=2022-01-10 03:00:00.0, price=41655.18), CryptoPrice(timestamp=2022-01-10 04:00:00.0, price=41815.32), CryptoPrice(timestamp=2022-01-10 05:00:00.0, price=41927.7), CryptoPrice(timestamp=2022-01-10 12:00:00.0, price=41796.01), CryptoPrice(timestamp=2022-01-10 13:00:00.0, price=41778.88), CryptoPrice(timestamp=2022-01-10 16:00:00.0, price=40774.01), CryptoPrice(timestamp=2022-01-11 06:00:00.0, price=42277.99), CryptoPrice(timestamp=2022-01-11 08:00:00.0, price=42110), CryptoPrice(timestamp=2022-01-11 18:00:00.0, price=41733.92), CryptoPrice(timestamp=2022-01-12 02:00:00.0, price=42729.29), CryptoPrice(timestamp=2022-01-12 15:00:00.0, price=43285.52), CryptoPrice(timestamp=2022-01-12 18:00:00.0, price=43560.29), CryptoPrice(timestamp=2022-01-13 07:00:00.0, price=43623.01), CryptoPrice(timestamp=2022-01-13 10:00:00.0, price=43699.99), CryptoPrice(timestamp=2022-01-13 17:00:00.0, price=44154.52), CryptoPrice(timestamp=2022-01-14 03:00:00.0, price=42750.66), CryptoPrice(timestamp=2022-01-14 11:00:00.0, price=42519.99), CryptoPrice(timestamp=2022-01-14 18:00:00.0, price=43109), CryptoPrice(timestamp=2022-01-15 00:00:00.0, price=43296.29), CryptoPrice(timestamp=2022-01-15 01:00:00.0, price=43250.56), CryptoPrice(timestamp=2022-01-15 16:00:00.0, price=42906.51), CryptoPrice(timestamp=2022-01-16 03:00:00.0, price=43069.69), CryptoPrice(timestamp=2022-01-16 04:00:00.0, price=42996.99), CryptoPrice(timestamp=2022-01-16 07:00:00.0, price=43023.57), CryptoPrice(timestamp=2022-01-16 14:00:00.0, price=42968.3), CryptoPrice(timestamp=2022-01-16 17:00:00.0, price=43300.01), CryptoPrice(timestamp=2022-01-17 11:00:00.0, price=42790.35), CryptoPrice(timestamp=2022-01-17 14:00:00.0, price=42671.3), CryptoPrice(timestamp=2022-01-17 18:00:00.0, price=42477.67), CryptoPrice(timestamp=2022-01-17 20:00:00.0, price=42110.84), CryptoPrice(timestamp=2022-01-18 13:00:00.0, price=41924.73), CryptoPrice(timestamp=2022-01-18 17:00:00.0, price=41449.34), CryptoPrice(timestamp=2022-01-18 18:00:00.0, price=41587.26), CryptoPrice(timestamp=2022-01-18 20:00:00.0, price=41489.04), CryptoPrice(timestamp=2022-01-19 02:00:00.0, price=42352.12), CryptoPrice(timestamp=2022-01-19 04:00:00.0, price=42420.17), CryptoPrice(timestamp=2022-01-19 09:00:00.0, price=41710.59), CryptoPrice(timestamp=2022-01-19 19:00:00.0, price=41912.77), CryptoPrice(timestamp=2022-01-20 04:00:00.0, price=41770), CryptoPrice(timestamp=2022-01-20 19:00:00.0, price=43274.77), CryptoPrice(timestamp=2022-01-21 07:00:00.0, price=38842.03), CryptoPrice(timestamp=2022-01-22 10:00:00.0, price=35400.01), CryptoPrice(timestamp=2022-01-22 13:00:00.0, price=35488.54), CryptoPrice(timestamp=2022-01-22 17:00:00.0, price=35009.27), CryptoPrice(timestamp=2022-01-22 22:00:00.0, price=34528.61), CryptoPrice(timestamp=2022-01-23 01:00:00.0, price=34875), CryptoPrice(timestamp=2022-01-23 15:00:00.0, price=35803.21), CryptoPrice(timestamp=2022-01-23 17:00:00.0, price=35323.85), CryptoPrice(timestamp=2022-01-23 20:00:00.0, price=35332.59), CryptoPrice(timestamp=2022-01-24 03:00:00.0, price=35840.3), CryptoPrice(timestamp=2022-01-24 05:00:00.0, price=35201.17), CryptoPrice(timestamp=2022-01-24 07:00:00.0, price=35123.57), CryptoPrice(timestamp=2022-01-24 08:00:00.0, price=35184.06), CryptoPrice(timestamp=2022-01-24 13:00:00.0, price=33276.59), CryptoPrice(timestamp=2022-01-24 17:00:00.0, price=33622.7), CryptoPrice(timestamp=2022-01-25 00:00:00.0, price=36807.92), CryptoPrice(timestamp=2022-01-25 09:00:00.0, price=35843.89), CryptoPrice(timestamp=2022-01-26 13:00:00.0, price=37940.46), CryptoPrice(timestamp=2022-01-26 20:00:00.0, price=38040.9), CryptoPrice(timestamp=2022-01-27 03:00:00.0, price=37004.43), CryptoPrice(timestamp=2022-01-27 22:00:00.0, price=36209.19), CryptoPrice(timestamp=2022-01-28 00:00:00.0, price=36187.01), CryptoPrice(timestamp=2022-01-28 01:00:00.0, price=36704.68), CryptoPrice(timestamp=2022-01-28 09:00:00.0, price=36907.14), CryptoPrice(timestamp=2022-01-28 10:00:00.0, price=36799.82), CryptoPrice(timestamp=2022-01-28 12:00:00.0, price=36486.1), CryptoPrice(timestamp=2022-01-28 17:00:00.0, price=36759.65), CryptoPrice(timestamp=2022-01-29 15:00:00.0, price=37847.57), CryptoPrice(timestamp=2022-01-29 16:00:00.0, price=38161.69), CryptoPrice(timestamp=2022-01-29 22:00:00.0, price=37853), CryptoPrice(timestamp=2022-01-30 05:00:00.0, price=37951.79), CryptoPrice(timestamp=2022-01-30 06:00:00.0, price=37917.13), CryptoPrice(timestamp=2022-01-30 19:00:00.0, price=37832.45), CryptoPrice(timestamp=2022-01-31 04:00:00.0, price=36823.19), CryptoPrice(timestamp=2022-01-31 13:00:00.0, price=37300.31), CryptoPrice(timestamp=2022-01-31 22:00:00.0, price=38415.79)]
2023-09-06T01:21:36.385+03:00 INFO 92410 --- [pool-2-thread-5] c.x.c.persistence.CryptoPricePersister : Inserted all prices for crypto: BTC
2023-09-06T01:21:36.386+03:00 INFO 92410 --- [ main] c.x.c.util.logger.ComponentLogger : Making the call DatabaseConnectionImpl.createTableOfCryptoNames(..) with arguments: [ETH, DOGE, LTC, XRP, BTC]
2023-09-06T01:21:36.534+03:00 INFO 92410 --- [ main] c.x.c.util.logger.ComponentLogger : Completed the call DatabaseConnectionImpl.createTableOfCryptoNames(..) with arguments: [ETH, DOGE, LTC, XRP, BTC]
2023-09-06T01:21:36.534+03:00 INFO 92410 --- [ main] c.x.c.util.logger.ComponentLogger : Completed the call CryptoDirectoryParser.persistAllCSVsInDirectory(..) with arguments: ./task/prices
2023-09-06T01:21:36.534+03:00 INFO 92410 --- [ main] c.x.c.util.logger.ComponentLogger : Making the call StatsCalculationService.computeAndLoadAllStats(..) with arguments: [ETH, DOGE, LTC, XRP, BTC]
2023-09-06T01:21:36.535+03:00 INFO 92410 --- [pool-3-thread-1] c.x.c.util.logger.ComponentLogger : Making the call DatabaseConnectionImpl.getCryptoPriceStats(..) with arguments: ETH
2023-09-06T01:21:36.535+03:00 INFO 92410 --- [pool-3-thread-2] c.x.c.util.logger.ComponentLogger : Making the call DatabaseConnectionImpl.getCryptoPriceStats(..) with arguments: DOGE
2023-09-06T01:21:36.535+03:00 INFO 92410 --- [pool-3-thread-3] c.x.c.util.logger.ComponentLogger : Making the call DatabaseConnectionImpl.getCryptoPriceStats(..) with arguments: LTC
2023-09-06T01:21:36.536+03:00 INFO 92410 --- [pool-3-thread-4] c.x.c.util.logger.ComponentLogger : Making the call DatabaseConnectionImpl.getCryptoPriceStats(..) with arguments: XRP
.
.
.
I guess I could just have one global thread pool that all my use-cases pull from, but I'm wondering if there is a deeper issue here with how I'm managing the shutting down of my ExecutorService. I have also tried enhancing the instances of latch.await() as follows:
try {
latch.await();
workers.shutdownNow();
workers.awaitTermination(2, TimeUnit.SECONDS);
} catch(InterruptedException ignored){}
but the thread pool index keeps increasing. Is this a cause for concern, or does Spring manage the names of the thread pools in this sequential way by default?
// EDIT:
I also tried a global ExecutorService that is used by all of my workers, and confirmed that the index stays the same. It seems as if even with just the latch I can re-use the existing worker pool. So why does the index increase? Is it a convention?