How to determine the ratio between unit and end to end tests in web application?

Question

We know by test pyramid that there should be more unit tests than end to end tests.

But question is how to get the right ratio?

Is it good if we have 49 % end to end tests, and 51 % unit tests?

I am having in mind web application api endpoints tested with codeception and phpunit.

Or should we aim for 10 % end to end tests and 90 % of unit tests.

By those percents I mean if we covered x % with end to end tests, then do not cover same code with unit tests.

But I mihgt be wrong, maybe we need to cover same lines with unit tests even if they are covered by end to end tests, as Martin Fowler writes here:

https://martinfowler.com/bliki/TestPyramid.html

I always argue that high-level tests are there as a second line of test defense. If you get a failure in a high level test, not just do you have a bug in your functional code, you also have a missing or incorrect unit test. Thus I advise that before fixing a bug exposed by a high level test, you should replicate the bug with a unit test. Then the unit test ensures the bug stays dead.

We usually do not cover same code with unit tests now if it is covered by end to end tests to save development time.

But if we need to cover same code with unit tests, then it means we should cover with end to end as little as possible but at least one basic call to API endpoint, because it should be covered by unit tests anyway.

Answer 1

The tests, even if they cover the same code, have different purposes. The ratio between them is not particularly meaningful.

Unit tests cover individual functional units; end to end tests cover logical flows. Unit tests are telling you that isolated pieces of code function as expected; flow tests are telling you that the code functions together as an application.

You should be aiming to maximise coverage with both. Aim to have unit tests cover all of your code, and for flow tests to cover all of your flows (which should be the vast majority of your code).

Really, it's completely dependent and your application. As a rule of thumb, I'd expect at least an order of magnitude (10x) more unit tests than flow tests. Both should be approaching 100% coverage of your codebase. But like I said, I don't think it's a meaningful metric.

Answer 2

If I were to set the target - it'd be 100:1 ratio. But of course the projects are different and the stage at which the project is would also matter.

My rule of thumb is - if it's possible to cover logic with unit test - it should be covered with unit test. That's where I'd disagree with Martin Fowler since if you can cover the logic with unit test - why would you even bother writing a system test?

To figure out which test to write you can divide them into 2 categories:

• Unit tests check the logic
• Higher-level tests check that the logic is invoked

This logic would keep the number of high-level tests at minimum. So now if a System test is written it's written because you couldn't write a unit test.

But besides unit and system (e2e) tests there is also something in between - component (a.k.a integration) tests. They initialize a big chunk of the app (which makes them not unit), but don't require the whole app to be running (which makes them not system). Many of the checks could be done this way and it's much (!) faster and more reliable than system tests.

You can see how this logic can be applied in practice in this article.

Answer 3

Personally, I don't care about the ratio. I may even have more E2E tests than unit/integration tests.

I mainly develop web applications using React:

• I write unit tests for functions computing some data. For instance, a getFilteredItems
• I write integration tests (not really unit, as I don't shallow render, using react-testing-library) for all presentation (dumb in React terms) components
• I finally write E2E tests for all data connected (smart) components.

This split ensures I don't get into the weird feeling of having a well-tested but not working project, perfectly summed up in this GIF. And it prevents to test several times the same code at different level, improving the maintainability.

Finally, to answer your question: I should have a ratio of 10/30/60 on my current project.