Caching npm dependencies in circleci

Question

I'm setting up the CI for an existing Express server project that lives in my repo's backend/core folder. Starting with just basic setup and linting. I was able to get npm install and linting to work but I wanted to cache the dependencies so that it wouldn't take 4 minutes to load for each push.

I used the caching scheme they describe here but it still seemed to run the full install each time. Or if it was using cached dependencies, it installed grpc each time which took a while. Any ideas what I can do?

My config.yml for reference:

# Use the latest 2.1 version of CircleCI pipeline process engine. See: https://circleci.com/docs/2.0/configuration-reference

# default executors
executors:
  core-executor:
    docker:
      - image: 'cimg/base:stable'

commands:
  init_env:
    description: initialize environment
    steps:
      - checkout
      - node/install
      - restore_cache:
          keys:
            # when lock file changes, use increasingly general patterns to restore cache
            - node-v1-{{ .Branch }}-{{ checksum "backend/core/package-lock.json" }}
            - node-v1-{{ .Branch }}-
            - node-v1-

      - run: npm --prefix ./backend/core install

      - save_cache:
          paths:
            - ~/backend/core/usr/local/lib/node_modules  # location depends on npm version
          key: node-v1-{{ .Branch }}-{{ checksum "backend/core/package-lock.json" }}


jobs:
  install-node:
    executor: core-executor
    steps:
      - checkout
      - node/install
      - run: node --version
      - run: pwd
      - run: ls -A
      - run: npm --prefix ./backend/core install

  lint:
    executor: core-executor
    steps:
      - init_env
      - run: pwd
      - run: ls -A
      - run: ls backend
      - run: ls backend/core -A
      - run: npm --prefix ./backend/core run lint

orbs:
  node: circleci/[email protected]
version: 2.1
workflows:
  test_my_app:
    jobs:
      #- install-node
      - lint
          #requires:
            #- install-node

Answer 1

I think the best thing to do is to use npm ci which is faster. Best explanation of this is here: https://stackoverflow.com/a/53325242/4410223. Even though it will reinstall every time, it is consistent so better than caching. Although when using this, I am unsure what the point of continuing to use cache in your pipeline is, but caching still seems to be recommended with npm ci.

However, the best way to do this is to just use the node orb you already have in your config. A single step of - node/install-packages will do all that work for you. You will be able to replace it with your restore_cache, npm install and save_cache steps. You can even see all the steps it does here: https://circleci.com/developer/orbs/orb/circleci/node#commands-install-packages. Just open the command source and look at the steps on line 71.