How to scan redis db without index?

Question

I am working on a legacy redis database recently. I want delete some unused data with specific pattern. But the table size is too big, 200 millions keys. unfortunatlly, there is no index so the scan is almost impossible. Below is a sample of the records:

PREFIX:64_BIT_HASH@INDEX_A PREFIX:128_BIT_HASH@INDEX_B PREFIX:n_BIT_HASH@INDEX_C ...... PREFIX:x_BIT_HASH@INDEX_N

There are around 300 type of INDEX defined in the database. I need delete pattern PREFIX:64_BIT_HASH@INDEX_A

I am using reactive redis data in springboot to have a scan, then do the filter, to first list all keys matching PREFIX:64_BIT_HASH@INDEX_A, but since there are 200 million records, the scan took 10 hours still not be able to return. Is there any way to scan by limit the result number, e.g. scan every 20K keys, instead of all keys?

try to use following code to scan, but never return because the data is too big.

public Flux<String> findGidsByDomain(String domain){
    String wildCastKey = "PREFIX:*@INDEX_A";
    ScanOptions opts = ScanOptions.scanOptions().match(wildCastKey).build();
    return redisTemplate.scan(opts).map(key -> keyBuilder.fetchKey(key)).filter(key -> key.length() == 64);
}

Answer 1

Use cursor build from connectionFactory

LettuceConnectionFactory connectionFactory;

public Flux findxxxsByCode(String code, int limit){

String wildCastKey = keyBuilder.wildCastKey(code);
Cursor cursor = connectionFactory.getConnection()
  .scan(ScanOptions.scanOptions().match(wildCastKey).build());

List<String> result = new ArrayList<>();
long count = 0;
while(count < limit){
  String k = new String((byte[]) cursor.next());
  log.debug("find key={}", k);
  String key = keyBuilder.fetchKey(k);
  if(key.length() == 64) {
    result.add(key);
    count++;
  }
}
cursor.close();
return Flux.fromIterable(result);

}