Dynamically generate Playwright test asynchronously, shows No test found

Question

The Playwright documentation has a parameterize sample like this. It works:

const people = ['Alice', 'Bob'];
for (const name of people) {
  test(`testing with ${name}`, async () => {
    // ...
  });
  // You can also do it with test.describe() or with multiple tests as long the test name is unique.
}

However, I want to get data array before each test，and generate test loop

//data like data[] = [{name:'Alice',age:'20'},..]
let data: any[] = []; 

test.beforeEach(async ({ request }) => {
    
    data = await getData(request);
});

for (let i = 0; i < data.length; i++) {
  test(`testing with ${name}`, async (page) => {
    // ...
  });

}

In visual studio code, It shows the error: No Test found

But it works when I assign a exact value let data: any[] =[{name:'Alice',age:'20'}];.

I test similar case https://github.com/microsoft/playwright/issues/9916 How do I properly run individual Playwright tests from a dynamic array of built urls?

Answer 1

The fundamental problem is that Playwright runs the test code twice. The first pass is synchronous, presumably used to detect the number of test() calls. The second pass actually executes the tests. Logging something in the top-level of the test file shows this two-step process.

Unfortunately, at the current time, there's no obvious way to use async code at the top level during the initial test detection step.

This works:

import {test} from "@playwright/test"; // ^1.42.1

(async () => {
  for (const name of ["a", "b", "c"]) {
    test(`testing with ${name}`, async () => {
      // ...
    });
  }
})();

but this doesn't detect tests:

import {setTimeout} from "node:timers/promises";
import {test} from "@playwright/test";

(async () => {
  await setTimeout(0); // actually do something async
  for (const name of ["a", "b", "c"]) {
    test(`testing with ${name}`, async () => {
      // ...
    });
  }
})();

Furthermore, before hooks aren't involved as part of the test discovery process. The following code

import {test} from "@playwright/test";

console.log("TEST SUITE");
test.beforeAll(() => console.log("BEFORE"));

for (const name of ["a", "b", "c"]) {
  test(`testing with ${name}`, async () => {
    // ...
  });
}

outputs:

$ npx playwright test
TEST SUITE

Running 3 tests using 1 worker

TEST SUITE
  ✓  1 pw.test.js:7:7 › testing with a (5ms)
BEFORE
  ✓  2 pw.test.js:7:7 › testing with b (4ms)
  ✓  3 pw.test.js:7:7 › testing with c (2ms)

  3 passed (270ms)

According to this comment, one workaround is to use test.step():

import {test} from "@playwright/test";

let people = [];

test.beforeAll(async ({request}) => {
  const url = "https://jsonplaceholder.typicode.com/users";
  const response = await request.get(url);
  people = (await response.json()).map(e => e.name);
});

test("test", async () => {
  for (const name of people) {
    await test.step(`testing with ${name}`, async () => {
      // ...
    });
  }
});

The problem is the output is missing the test case names:

$ npx playwright test

Running 1 test using 1 worker

  ✓  1 pw.test.js:11:5 › test (8ms)

  1 passed (407ms)

Another workaround is to generate the data in a separate script before running the tests, store it in a file and read the file synchronously at the start of the tests. For example:

generate-people-data.js:

const fs = require("node:fs/promises");

(async () => {
  const url = "https://jsonplaceholder.typicode.com/users";
  const response = await fetch(url);
  const people = (await response.json()).map(e => e.name);
  await fs.writeFile("people.json", JSON.stringify(people));
})();

people.test.js:

import fs from "node:fs";
import {test} from "@playwright/test";

const json = fs.readFileSync("people.json", {encoding: "utf-8"});
const people = JSON.parse(json);

for (const name of people) {
  test(`testing with ${name}`, async () => {
    // ...
  });
}

Run:

$ node generate-people-data && npx playwright test

Running 10 tests using 1 worker

  ✓  1 pw.test.js:7:7 › testing with Leanne Graham (9ms)
  ✓  2 pw.test.js:7:7 › testing with Ervin Howell (4ms)
  ✓  3 pw.test.js:7:7 › testing with Clementine Bauch (1ms)
  ✓  4 pw.test.js:7:7 › testing with Patricia Lebsack (1ms)
  ✓  5 pw.test.js:7:7 › testing with Chelsey Dietrich (1ms)
  ✓  6 pw.test.js:7:7 › testing with Mrs. Dennis Schulist (1ms)
  ✓  7 pw.test.js:7:7 › testing with Kurtis Weissnat (1ms)
  ✓  8 pw.test.js:7:7 › testing with Nicholas Runolfsdottir V (1ms)
  ✓  9 pw.test.js:7:7 › testing with Glenna Reichert (1ms)
  ✓  10 pw.test.js:7:7 › testing with Clementina DuBuque (3ms)

  10 passed (253ms)

If you don't like the &&, you can run npx playwright test as the final step of the generator script.

Another option is the synchronous fetch approach from the other thread.

Perhaps there's a more elegant approach in the current version, or future versions of Playwright will support async test detection.

Answer 2

I have a bad solution which is define length of Array

let data: any[] = new Array(10);

Then the tests can be determined by VScode

for (let i = 0; i < data.length; i++) {
  test(`testing with ${name}`, async page => {
    if (data[i] !== undefined) {
      // test
    }
    // ...
  });
}

Answer 3

That's because you are using beforeEach hook and most probably getData method is assigning undefined value to data or data length is zero.
I recommend you debugging what's being assigned to data in beforeEach hook.

One more important thing: beforEach hook run once before EACH test. In your case,

1. your test starts with and empty array data;
2. Your hook works once;
3. But since data is empty, test run stops and process exits with error: No Test found.
   
   

Solution:

import { test } from '@playwright/test';
import { getData } from '../utils/getData';

test.describe('Some feature', () => {
  let data: string[] = [];
  data = getData(); // Assuming it returns an array of strings

  data.forEach(testName => {
    test(`Test name: ${testName}`, () => {
      console.log('Logging test name:', testName);
    });
  });
});

It looks like the length of the data should be more than 0 before the tests start. Otherwise, since the array is empty and tests in the loop iteration won't be triggered.