How can you harvest CKAN datsets using filter

426 Views Asked by At

I'm using CKAN 2.9 version with the Remote harvesting extension.

I don't want to harvest all datasets, I want to harvest select datasets of name or of the ID within packages on a website.

Can I use a filter e.g. with CQL filter in the config or another way possibility (another ext?)? Do you have an idea of how I can harvest the datasets with a filter and a simple example if I have to use the config?

1

There are 1 best solutions below

0
Zoran Pandovski On

By default the CKAN harvester extension allows you to filter by organization or group datasets to be included or excluded from harvesting. For e.g you can harvest the dataset only for a org_1 by adding the "organizations_filter_include": ['org_1'] key. For more info check the harvester docs in Github. If that doesn't work for you, you can use the IHarvester interface and add your own logic, which could be more complicated than using the one above provided by the harvester.