WatchService: missing events

Question

I've created a small Java app which monitors folders. It uses the observer pattern to notify different fileHandlers. The filehandler operation can take a few seconds to process the files. All works well if it processes only 1 item at a time. If a new file arrives, before the previous one is finished, it misses the event and the new file is not detected. What is the best pattern to solve this issue? Some part of my code ...

        WatchService watcher = FileSystems.getDefault().newWatchService();

        List<Path> allPaths  = new ArrayList<Path>();
        Path path = Paths.get("c:\InPutFolder");
        allPaths.add(path);


        // Register all paths in the WatchService
        Map<WatchKey, Path> allKeys = new HashMap<>();
        for (Path path : allPaths) {
            WatchKey key = path.register(watcher, StandardWatchEventKinds.ENTRY_CREATE);
            allKeys.put(key, path);
        }

        // Monitor the folders and listen for change notification.
        while (true) {
            WatchKey key = watcher.take();
            for (WatchEvent<?> event : key.pollEvents()) {
                WatchEvent.Kind<?> kind = event.kind();

                if (ENTRY_CREATE.equals(kind)) {
                    Path path = allKeys.get(key);

                    // Store item in arrayList of objects 
                    Item item = new Item(path.toString(), event.context().toString());
                    ic.addItem(item);

                    notifyObservers(); // NOTIFY OBSERVERS & EXECUTE LONG PROCESS
                } 
            }
            key.reset();
        }

Answer 1

I have a similar implementation with a service that waits for files. From this article

when the WatchKey instance is returned by either of the poll or take APIs, it will not capture more events if it's reset API is not invoked

so in-between the take and reset calls you are not receiving events. The solution is to grab the events (or data from the events) and call reset asap. I use a thread pool and process each event in a Runnable instance. Have a look at java's ExecutorService if you want to go down the threaded path. You need to make sure however that whatever your long running process is calling will be thread-safe.

There's no reason why you can't grab all the event information, then call reset and process the events synchronously after the reset call. If you do, you should probably catch OVERFLOW events such that you know when you've missed events.