Common Pitfalls when writing E2E tests with Playwright and Prisma & how to avoid them

Testing is an essential part of any software development process. It helps you to ensure that your code is working as expected and that you don’t break anything when you make changes. In this article, we will look at how to create a test environment that you can rely on using Remix, Prisma and Playwright.

Pitfalls when writing E2E tests

When you start writing tests, you will quickly notice that they are not always reliable. This is called a flaky test.

Flaky Test

A test is a test that fails to produce consistent results given the same input. This makes it difficult to determine whether the test is failing because of a bug in the code or because of a problem with the test itself. Common reasons for flaky tests are:

• Tests that depend on external services, like database, network, etc.
• Tests that depend on the order in which they are run
• Tests that depend on the state of the system

When writing E2E tests, you will often run into these problems. We want to test every interaction that a user can make with our app in isolation, however, this gets harder when interactions mutate the database state or when tests run in parallel.

Database state

When you run tests in isolation, they should not depend on the state of the database. Imagine that you have a test that creates a new user and then logs in with that user. If you run this test in isolation, it will work fine. However, if you run it after another test that creates that same user, it will fail because the user already exists.

Parallel tests

When you run tests in parallel, they can interfere with each other when accessing and mutating shared resources. We call this problem a “race condition”.

Race Condition

A scenario where two or more threads (instances of running software) are accessing shared data and they are working on the same data at the same time. This can lead to unexpected results.

Imagine that you have two tests that are trying to get access a shared resource that can only be user by one user at a time, such as a printer. If the printer is not fast enough, one of the tests will fail because the printer is busy. We can not determine which test will fail, because it depends on the order in which the tests are run.

Avoiding the pitfalls

When running tests with Playwright and Prisma, you will quickly notice that there are a few problems that you need to solve. The first is that it is not trivial to get a clean database for each test. The second is that tests are running in parallel by default and can interfere with each other.

Clean database for each test

The first problem is that you need a clean database for each test. I found that the easiest way to achieve this is to write a cleanup function that deletes all data from the database.

export async function cleanup() {
  await prisma.$transaction([
    prisma.user.deleteMany(),
    prisma.post.deleteMany(),
    prisma.comment.deleteMany(),
    //... add more models here
  ]);
}

Note: This method will work great for small projects, but if you have a lot of data, it might be better to use a different approach. You can read more about this in the Prisma docs.

Now you can call this function before each test to ensure that the database is clean.

test.describe("Some Test Suit", async () => {
  test.beforeEach(async () => {
    await cleanup();
  });
  // ..rest of the test suite
});

We still have a problem though. We want to re-seed the database with some data before each test. Thankfully, Prisma has a great feature called seed that we can utilize for this.

We need to define a seed file in ‘prisma/seed.ts’.

import { PrismaClient } from "@prisma/client";

const prisma = new PrismaClient();

async function seed() {
  // ... seed the database
}

if (process.env.NODE_ENV !== "test") {
  seed()
    .catch((e) => {
      console.error(e);
      process.exit(1);
    })
    .finally(async () => {
      await prisma.$disconnect();
    });
}

Note: The if statement is important, because we don’t want to seed the database automatically when running our tests. By default, Prisma will run the seed file when you run prisma migrate dev or prisma migrate deploy. In our case we want to import the seed function and call it manually. If we don’t add the if statement, the seed function will be called twice.

Make sure to add NODE_ENV=test to your test command.

Now in our test file, we can call the seed function after we cleanup the database.

test.describe("Some Test Suit", async () => {
  test.beforeEach(async () => {
    await cleanup();
    await seed();
  });
  // ..rest of the test suite
});

Parallel tests

The second problem is that tests running in parallel can interfere with each other by changing the database state. Unfortunately, I didn’t find a good solution for this problem other than to run the tests sequentially. This is not ideal, but it works for now.

In your playwright.config.ts file, you can set the workers option to 1 to run the tests sequentially.

import { type PlaywrightTestConfig } from "@playwright/test";
const config: PlaywrightTestConfig = {
  workers: 1,
  // ... rest of the config
};
export default config;

Conclusion

Testing, and especially end-to-end testing, is a complex topic. There are many different approaches and tools that you can use. In this article, we looked at how to create a test environment that you can rely on when using Remix, Prisma and Playwright. I hope that this article will help you to get started with testing your Remix app.