Why do packages loaded inside `test_that()` provide their methods outside of `test_that()` and how can I prevent that?

Question

In my understanding, anything put inside test_that() should be compartmentalized, meaning that if I load a package in test_that(), its functions and methods shouldn't be available in other test_that() calls.

In the example below, there are 3 (empty) tests:

• In the first one, we can see that the method as.matrix.get_predicted is not available in the namespace.
• In the second one, I load the package insight, which provides the method as.matrix.get_predicted. In my understanding, this method should only be available in this test_that() call.
• In the third one, we can see that the method is available.

library(testthat)

test_that("foo 1", {
  print("as.matrix.get_predicted" %in% methods(as.matrix))
})
#> [1] FALSE
#> ── Skip (???): foo 1 ───────────────────────────────────────────────────────────
#> Reason: empty test

test_that("foo 2", {
  invisible(insight::get_parameters)
})
#> ── Skip (???): foo 2 ───────────────────────────────────────────────────────────
#> Reason: empty test

test_that("foo 3", {
  print("as.matrix.get_predicted" %in% methods(as.matrix))
})
#> [1] TRUE
#> ── Skip (???): foo 3 ───────────────────────────────────────────────────────────
#> Reason: empty test

Why is that? Are there some workarounds?

---

Edit: I'm looking for a solution specific to testthat, not another testing framework.

Answer 1

Too long for a comment, but reproducible. The test files are sourced in collation order, each in a new R process, hence the library call in testB.R does not cause the test in testC.R to fail.

pkgname <- "testpackage"
testdir <- file.path(pkgname, "tests")

.add <- function(a, b) a + b
package.skeleton(pkgname, list = ".add")
dir.create(testdir, recursive = TRUE)
writeLines("stopifnot(!any(.S3methods(as.data.frame) == \"as.data.frame.lm\"))",
           file.path(testdir, "testA.R"))
writeLines("library(parameters); stopifnot(any(.S3methods(as.data.frame) == \"as.data.frame.lm\"))",
           file.path(testdir, "testB.R"))
writeLines("stopifnot(!any(.S3methods(as.data.frame) == \"as.data.frame.lm\"))",
           file.path(testdir, "testC.R"))

tools::Rcmd(c("check", pkgname))

* checking tests ...
  Running ‘testA.R’
  Running ‘testB.R’
  Running ‘testC.R’
 OK

This is the "vanilla" approach to compartmentalizing tests that many people still prefer over testthat and other frameworks, at least partly because it heeds the warning in ?detach:

The most reliable way to completely detach a package is to restart R.

and therefore is significantly easier for experts to debug and reason about, but that is perhaps a controversial opinion these days ...

Answer 2

As a complement to @MikaelJagan's answer, there is a section in "R Packages" that addresses this problem:

It’s fair to say that library(somePkg) in the tests should be about as rare as taking a dependency via Depends, i.e. there is almost always a better alternative.

Unnecessary calls to library(somePkg) in test files have a real downside, because they actually change the R landscape. library() alters the search path. This means the circumstances under which you are testing may not necessarily reflect the circumstances under which your package will be used. This makes it easier to create subtle test bugs, which you will have to unravel in the future.