How come my async function is not working in Node JS?

Question

I'm trying to make a REST api to hold json files for contacts, for some reason my endpoint responds before it should.

const fs = require("fs/promises");

app.get("/contacts", async (req, res) => {
    let dir_name = "./contacts";
    let contacts = [];

    
    
    await fs.readdir(dir_name).then(async (filenames) => {
        filenames.forEach(async (filename) => {
        await fs.readFile(`${dir_name}/${filename}`, "utf-8").then((jsonString) => {
            contacts.push(JSON.parse(jsonString));
            console.log(contacts);
        });
        });
    });

    console.log(contacts);
    res.send(contacts);
    
});

This code just results in [] being sent in response but the console.log under the json parse prints the result I'm expecting. I figured that I'm using the await keyword wrong but I'm stuck. Note: /contacts is a valid directory.

Answer 1

You can use a regular for..of loop (which is "async aware" since it's not a function but a syntax construct) to read the files one by one while remaining async and not using blocking calls.

const fs = require("fs/promises");

app.get("/contacts", async (req, res) => {
    const dir_name = "./contacts";
    const contacts = [];
    const filenames = await fs.readdir(dir_name);
    for(const filename of filenames) {
        const content = await fs.readFile(`${dir_name}/${filename}`, "utf-8");
        contacts.push(JSON.parse(content));
    }
    console.log(contacts);
    res.send(contacts);
});

If you wanted to read them concurrently, you'd need to use Promise.all(filenames.map(...)):

const fs = require("fs/promises");

app.get("/contacts", async (req, res) => {
    const dir_name = "./contacts";
    const filenames = await fs.readdir(dir_name);
    const contacts = await Promise.all(filenames.map(async (filename) => {
        const content = await fs.readFile(`${dir_name}/${filename}`, "utf-8");
        return JSON.parse(jsonString);
    }));
    console.log(contacts);
    res.send(contacts);
});

That approach runs into the possibility of opening too many files at once, in which case you'd need to limit concurrency with e.g. p-queue or p-limit.