Extract all Product database in a single List<Product> with CommerceTools SDK v2

Question

I'm looking for a way to migrate certain functionality from CommerceTools Java SDK-v1 to SDK-v2. This functionality allows me to extract the entire product database (around 30k products) into a single list of products (List<Product>).

This is the code we currently use with SDK v1:

            final QueryPredicate qPred = ProductQuery.of().productType().isInIds(ids);
            final ProductQuery seedQuery = ProductQuery.of()
                .withSort(m -> m.id().sort().asc())
                .withLimit(ProjectConstants.PAGE_SIZE)
                .withPredicates(qPred)
                .withFetchTotal(false);
            final CompletionStage<List<Product>> resultStage = findNext(seedQuery, seedQuery, new LinkedList<>(), connCT);
            final List<Product> actualProducts = resultStage.toCompletableFuture().join();
            final List<Product> javaSortedActual = actualProducts.stream().sorted(Comparator.comparing(p -> p.getId())).collect(toList());
            return javaSortedActual;

As you can see, ProductQuery is used to create the seedQuery that is then used with CompletionStage.findNext which is the command that retrieves all the product pages one by one and enters them into the complete list.

The problem is that with the SDK v2 the ProductQuery method no longer exists and I no longer know how I should recover all the products from our database in a single list.

Does anyone know how I can get the same functionality with SDK v2?

Thanks in advance.

Answer 1

You can use the QueryUtils class with it's methods. They provide similar functionality.

    public void queryAll() {
    ProjectApiRoot apiRoot = createProjectClient();
    List<String> ids = QueryUtils.queryAll(apiRoot.productProjections().get(), (productProjections) -> {
        return productProjections.stream().map(ProductProjection::getId).collect(Collectors.toList());
    }, 100)
            .thenApply(lists -> lists.stream().flatMap(List::stream).collect(Collectors.toList()))
            .toCompletableFuture()
            .join();
    }

Please see https://commercetools.github.io/commercetools-sdk-java-v2/javadoc/com/commercetools/docs/meta/Querying.html

Answer 2

Thank you very much @jenschude, but I finally decided to create my own method because I needed to add the complete Product objects in a List to process them later. It may not be the best solution, but it works for me, which is the important thing :-)

    public List<Product> getAllProductsByPredicate(final String predicate) throws InterruptedException {
        List<Product> tmpProducts = new java.util.ArrayList<>(Collections.emptyList());
        boolean lastPage = false;
        int pagina = 0;

        try {
            // Instead of using offset to get a page, ask for elements being greater than the id of the first
            // product in your project
            String lastId = getApiRoot()
                .products()
                .get()
                .withWhere(predicate)
                .withSort("id asc")
                .withLimit(1)
                .execute()
                .toCompletableFuture().get()
                .getBody().getResults().get(0).getId();
            lastId = lastId.substring(0, lastId.length() - 1) + "0"; // Starting last id less than the first one

            while (!lastPage) {
                pagina++;
                ProductPagedQueryResponse productPagedQueryResponse =
                    getApiRoot()
                        .products()
                        .get()
                        // Important, internally we use id > $lastId, it will not work without this line
                        .withSort("id asc")
                        .withWhere(predicate)
                        .addWhere("id > :lastId")
                        .addPredicateVar("lastId", lastId)
                        // Limit the size per page
                        .withLimit(MAXIMUM_AMOUNT_OF_RESULTS)
                        // always use this
                        .withWithTotal(false)
                        .execute()
                        .toCompletableFuture()
                        .get()
                        .getBody();

                // Add all the results to temporary list os products
                tmpProducts.addAll(productPagedQueryResponse.getResults());
                // Evaluate the size of current results
                int size = productPagedQueryResponse.getResults().size();
                if (size != 0) {
                    LOGGER.debug("Recuperada la página #{} con {} productos", pagina, size);
                    lastId = productPagedQueryResponse.getResults().get(size - 1).getId();
                }
                // If (size < MAXIMUM_AMOUNT_OF_RESULTS) then lastPage = true to exit the loop
                if (size != MAXIMUM_AMOUNT_OF_RESULTS) {
                    LOGGER.debug("Ultima página de productos (total {} páginas)", pagina);
                    lastPage = true;
                }
            }
            return tmpProducts;
        } catch (ExecutionException e1) {
            LOGGER.error("Failed to execute. ExecutionException:\n{}", e1.getMessage());
            return tmpProducts;
        }
    }

Thanks again