Docker container running Gulp runs tasks, but not when "watching"

Question

I've put my Gulp stuff into a Dockerfile:

FROM node:14
WORKDIR /app
COPY package*.json ./
COPY gulpfile.js ./
...
RUN npm install
RUN npm install -g gulp
EXPOSE 3000
CMD ["gulp"]

When I run the following build and run it works and files are generated:

docker build . -t webteam/gulp
docker run --rm -it -v ${pwd}/src:/app/src -v ${pwd}/dist:/app/dist -p 3000:3000 webteam/gulp styles

Here's the styles task in the gulpfile:

const styles = () => src(SASS_SRC)
  .pipe(sourcemaps.init())
  .pipe(sass().on('error', sass.logError))
  .pipe(autoprefixer({
    cascade: false,
  }))
  .pipe(sourcemaps.write())
  .pipe(dest('./dist/css'));

This works perfectly.

However, when I run the default task which is written to "watch" for file changes, the task seems to start up OK. Browsersync even works from the container to my host machine. And it looks OK in the terminal.... but any file changes are not detected.

Here's my default task:

exports.default = () => {
  browserSync.init({
    server: {
      baseDir: './dist/',
    },
    startPath: '/homepage.html',
    ghostMode: false,
  });

  // watch the src files and html files for changes
  watch(SASS_SRC, exports.styles);
  watch(JS_SRC, exports.scriptsDev);
  watch(JS_SRC, exports.scriptsProd);

  // watch the dist files, and browser sync when changes
  watch(['./dist/**/*', './demo/**/*']).on('change', browserSync.reload); // TODO remove demo at some stage
};

What is needed for Docker environment to notice changes within files of the linked directory (src) to trigger the watch functions?

UPDATED

Added usePolling to watch options, as I think this is a solution to my problem if I can get it working. Add added some addition options from the docs that work with usePolling - https://gulpjs.com/docs/en/api/watch#options:

const SASS_SRC = './src/scss/**/*.scss';
const JS_SRC = ['./src/js/**/*.js', './src/js/modules/**/*.js'];
const HTML_FILES = './dist/**/*';

...

exports.default = () => {

  // Enable polling
  const watchOptions = {

    // When false, the watcher will use fs.watch() (or fsevents on Mac) for watching. If true, 
    // use fs.watchFile() polling instead - needed for successfully watching files over a network 
    // or other non-standard situations. Overrides the useFsEvents default. This option is passed 
    // directly to chokidar.
    usePolling: true,

    // Combine with usePolling: true. Interval of file system polling. This option is passed directly to chokidar.
    interval: 100,

    // Combine with usePolling: true. Interval of file system polling for binary files. This option 
    // is passed directly to chokidar.
    binaryInterval: 300,

    // When true, uses fsevents for watching if available. If explicitly set to true, supersedes the 
    // usePolling option. If set to false, automatically sets usePolling to true. This option is passed 
    // directly to chokidar.
    useFsEvents: false,
    
  };
  
  browserSync.init({
    server: {
      baseDir: './dist/',
    },
    startPath: '/homepage.html',
    ghostMode: false,
    watchOptions,
  });

  // watch the src files and html files for changes
  // @see https://gulpjs.com/docs/en/api/watch
  watch(SASS_SRC, watchOptions, exports.styles);
  watch(JS_SRC, watchOptions, exports.scriptsDev);
  watch(JS_SRC, watchOptions, exports.scriptsProd);

  // watch the dist files, and browser sync when changes
  watch(HTML_FILES, watchOptions).on('change', browserSync.reload); // TODO remove demo at some stage
};

However, it still doesn't register when I change a SCSS file (run styles) or change an HTML file (reload the browser)

I also tried changing the watch first argument (the path to watch files) to the container path:

const SASS_SRC = '/app/src/scss/**/*.scss';
const JS_SRC = ['/app/src/js/**/*.js', '/app/src/js/modules/**/*.js'];
const HTML_FILES = '/app/dist/**/*';

But it doesn't make any difference. Anything else I could try, or am I doing something wrong?

Answer 1

Docker's file sharing does not always propagate file system events (particularly on Windows or Mac), causing tools that rely on these events to detect changes to fail.

To work around this issue, you can use polling as a fallback mechanism for file watching. That means instead of relying on file system events, Gulp will periodically check for file changes.
That would have higher CPU usage, but is a reliable workaround when working within Docker containers on Windows or Mac.

You can modify your Gulp watch task to use polling by specifying the usePolling option.
Your default task would then be:

const { watch, series } = require('gulp');
const browserSync = require('browser-sync').create();

// Assuming exports.styles, exports.scriptsDev, and exports.scriptsProd are defined elsewhere

exports.default = () => {
  browserSync.init({
    server: {
      baseDir: './dist/',
    },
    startPath: '/homepage.html',
    ghostMode: false,
  });

  // Enable polling
  const watchOptions = {
    usePolling: true,
  };

  // watch the src and html files for changes with polling
  watch(SASS_SRC, watchOptions, exports.styles);
  watch(JS_SRC, watchOptions, exports.scriptsDev);
  watch(JS_SRC, watchOptions, exports.scriptsProd);

  // watch the dist files, and browser sync when changes
  watch(['./dist/**/*', './demo/**/*'], watchOptions).on('change', browserSync.reload);
};

That introduces the watchOptions object with usePolling: true, which is passed as an argument to Gulp's watch method.

---

However, it still doesn't register when I change a SCSS file (run styles) or change an HTML file (reload the browser)

If the core implementation of polling with Gulp's watch task within Docker is correctly set up, but the problem persists, then the issue might not solely be related to file watching mechanisms but could involve other aspects of the Docker or Gulp configuration.

Make sure the Docker volume mounts are correctly configured to reflect changes made on the host system within the container. Incorrect or missing volume mounts can lead to situations where changes are not visible inside the container, hence not detectable by Gulp's watch mechanism, even with polling enabled.

docker exec -it <container_name_or_id> /bin/sh
cd /app/src
ls -l
# Modify a file within the mounted directory on your host system.
# Back in your container terminal, list the contents again to check if the change is reflected:

File watching can sometimes be affected by file system permissions. Make sure the Docker container runs with a user having adequate permissions to read and watch the directories in question.

To isolate the issue, try testing the polling mechanism outside of Docker with a simple Gulp watch task on your host system. If polling works as expected outside of Docker, the issue likely lies with the Docker environment or configuration.
Simplify your Gulp and Docker setup to the bare minimum needed to test the watch functionality.
And increase the verbosity of Gulp's output to get more insights into what Gulp is doing.

// Example for increasing verbosity in Gulp tasks
const gulp = require('gulp');
gulp.on('start', function(evt) {
  console.log('Starting task:', evt.name);
}).on('stop', function(evt) {
  console.log('Finished task:', evt.name);
}).on('error', function(evt) {
  console.log('Error in task:', evt.name);
});

---

The OP Martyn adds in the comments:

I wasn't rebuilding my image correctly.
Rebuilt now correctly:

docker build --no-cache . -t my-stuff/gulp. 

OK, usePolling is working really well.