Skip to content

Releases: r-lib/testthat

testthat 3.2.2

10 Dec 23:30
Compare
Choose a tag to compare

New expectations

  • expect_s7_class() tests if an object is an S7 class (#1580).

  • expect_no_failure(), expect_no_success() and expect_snapshot_failure()
    provide more options for testing expectations.

Bug fixes and minor improvements

  • testthat now requires waldo 0.6.0 or later to access the latest features
    (#1955).

  • expect_condition() and related functions now include the class of the
    expected condition in the failure message, if provided (#1987).

  • expect_error() and friends now error if you supply ... but not pattern
    (#1932). They no longer give an uninformative error if they fail inside
    a magrittr pipe (#1994).

  • expect_no_*() expectations no longer incorrectly emit a passing test result
    if they in fact fail (#1997).

  • expect_setequal() correctly identifies what is missing where (#1962).

  • expect_snapshot() now strips line breaks in test descriptions
    (@LDSamson, #1900), and errors when called from a test_that() that has an
    empty description (@kevinushey, #1980).

  • expect_true() and expect_false() give better errors if actual isn't a
    vector (#1996).

  • expect_visible() and expect_invisible() have clearer failure messages
    (#1966).

  • local_reproducible_output() (used in test_that() blocks) now sets
    LANGUAGE to "C" instead of "en" to disable translations,
    avoiding warnings on some platforms (#1925).

  • skip_if_not_installed() generates a clearer message that sorts better
    (@MichaelChirico, #1959).

  • with_mock() and local_mock() have been unconditionally deprecated as
    they will no longer work in future versions of R (#1999).

testthat 3.2.1

02 Dec 15:17
Compare
Choose a tag to compare
  • Fix incorrect format string detected by latest R-devel. Fix thanks to
    Tomas Kalibera.

  • expect_snapshot() handles unexpected errors like errors outside of
    snapshots, i.e. they terminate the entire test and get a traceback (#1906).

  • JunitReporter() now uses ensures numeric values are saved the xml file
    with . as decimal separator. (@maksymiuks, #1660)

  • local_mocked_bindings() can now mock any object, not just functions
    (#1896).

  • skip_if_offline() now uses captive.apple.com by default. This is the
    hostname that Apple devices use to check that they're online so it should
    have a higher reliability than r-project.org (@jdblischak, #1890).

  • test_file(desc = ) will now find describe() tests as well as test_that()
    tests (#1903).

testthat 3.2.0

07 Oct 19:28
Compare
Choose a tag to compare

Lifecycle changes

  • is_informative_error() and the wrap argument to test_dir() and friends
    are now defunct.

  • expect_no_error(), expect_no_warning(), expect_no_message(),
    expect_no_condition(), local_mocked_bindings(), and
    with_mocked_bindings() are now stable, not experimental.

New features

  • All packages, regardless of whether or not they use rlang 1.0.0, now
    use the new snapshot display for errors, warnings, and messages (#1856).
    This no longer shows the class name, instead focussing on a display that
    more closely mimics what you'll see interactively, including showing the
    error call.

  • testthat uses an improved algorithm for finding the srcref associated with
    an expectation/error/warning/skip. It now looks for the most recent call
    that has known source and is found inside the test_that() call. This
    generally gives more specific locations than the previous approach and
    gives much better locations if an error occurs in an exit handler.

Minor features and bug fixes

  • Helpers are no longer run twice.

  • expect_setequal() correctly displays results when only one of actual and
    expected is missing elements (#1835).

  • expect_snapshot() and friends no longer create a temporary file on every
    invocation.

  • expect_snapshot_file() now generates clickable links to review changes
    (#1821).

  • expect_snapshot_value() has an improved error if the object can't be
    safely serialized using the specified style (#1771).

  • options(rlang_interactive = TRUE) no longer causes skip_on_cran() to
    not run on CRAN (#1868).

  • skip_if_offline() now errors if you don't have curl installed (#1854).

  • StopReporter gains the ability to suppress praise when a test passes.

  • ProgressReporter now uses is a two characters wide skip column in order
    to have a consistent width when 10 or more tests are skipped in a single file
    (@mgirlich, #1844).

  • test_file() gains a desc argument which allows you to run a single
    test from a file (#1776).

testthat 3.1.10

07 Jul 13:27
Compare
Choose a tag to compare
  • Fix for upcoming R-devel release.

  • testthat now sets the _R_CHECK_BROWSER_NONINTERACTIVE_ environment variable
    when running tests. This should ensure that left-over browser() statements
    will trigger an error if encountered while running tests. This functionality
    is only enabled with R (>= 4.3.0). (#1825)

testthat 3.1.9

16 Jun 12:13
Compare
Choose a tag to compare
  • New expect_contains() and expect_in() that works similarly to
    expect_true(all(expected %in% object)) or
    expect_true(all(object %in% expected)) but give more informative failure
    messages (#1346).

  • New is_snapshot() returns TRUE if code is running inside a snapshot test
    (#1796) and is_checking() returns TRUE if test is running inside of
    R CMD check (#1795)

  • ProgressReporter only reports the run time of test files that take longer
    than 1s, rather than 0.1s. (#1806) and re-displays all failures at the end
    of the results. Skips are now only shown at the end of reporter summaries,
    not as tests are run. This makes them less intrusive in interactive tests
    while still allowing you to verify that the correct tests are skipped (#1801).
    When using parallel tests, links to failed tests (#1787) and links to
    accept/review snapshot (#1802) now work.

  • set_state_inspector() allows to to register a function that's called
    before and after every test, reporting on any differences. This
    is very useful for detecting if any of your tests have made changes to
    global state (like options, env vars, or connections) (#1674). This
    function was inspired by renv's testing infrastructure.

  • skip_on_cran() no longer skips (errors) when run interactively.

  • teardown_env() works in more cases.

  • All packages, regardless of whether or not they use rlang, now
    use the new snapshot display for errors, warnings, and messages.

  • testthat no longer truncates tracebacks and uses rlang's default tree
    display.

testthat 3.1.8

04 May 13:36
Compare
Choose a tag to compare
  • expect_snapshot() differences no longer use quotes.

  • expect_error(), expect_warning(), and expect_message() now correctly
    enforce that the condition is of the expected base class (e.g. error,
    warning, messsage) even when the class argument is used (#1168).

  • it() now calls local_test_context() so that it behaves more
    similarly to test_that() (#1731), and is now exported so that you
    can more easily run BDD tests interactively (#1587)

  • skip_on_bioc() now uses the documented environment variable
    (IS_BIOC_BUILD_MACHINE) (#1712).

  • source_file(), which is used by various parts of the helper and
    setup/teardown machinery, now reports the file name in the case of
    errors (#1704).

  • test_path() now works when called within helper files (#1562).

  • New vignette("special-files") describes the various special files
    that testthat uses (#1638).

  • with_mocked_bindings() and local_mocked_bindings() now also bind in the
    imports namespace and can mock S3 methods. These changes make them good
    substitutes for the deprecated functions with_mock() and local_mock(), so
    those older functions now recommend switching to the newer equivalents
    instead of using the mockr or mockery packages.

testthat 3.1.7

12 Mar 18:14
Compare
Choose a tag to compare
  • expect_setequal() gives more actionable feedback (#1657).

  • expect_snapshot() no longer elides new lines when run interactively (#1726).

  • Experimental new with_mocked_bindings() and local_mocked_bindings()
    (#1739).

testthat 3.1.6

10 Dec 18:24
Compare
Choose a tag to compare
  • The embedded version of Catch no longer uses sprintf().

testthat 3.1.5

08 Oct 13:33
Compare
Choose a tag to compare
  • Deprecation warnings are no longer captured by expect_warning(code, NA),
    expect_no_warning(code), or expect_silent(code). This ensures that they
    bubble up to the top level so that you can address them (#1680). If you want
    to assert that code does not throw a deprecation warning, use
    expect_no_condition(code(), class = "lifecycle_warning_deprecation").

  • New experimental expect_no_error(), expect_no_warning(),
    expect_no_message(), and expect_no_condition() for asserting
    the code runs without an error, warning, message, or condition (#1679).

  • Fixed a warning in R >=4.2.0 on Windows that occurred when using the C++
    testing infrastructure that testthat provides (#1672).

  • Fixed an issue that could prevent compilation of Catch unit tests with
    LLVM 15. In the interim, packages needing a local workaround can set
    PKG_CPPFLAGS = -DCATCH_CONFIG_CPP11_NO_SHUFFLE in their src/Makevars.
    (@kevinushey, #1687)

  • Improve way capture_output() handles encoding thanks to suggestion from
    Kurt Hornik (#1693). This means that snapshots using UTF-8 encoded text on
    windows work once again.

  • local_reproducible_output() will no longer attempt to set the local language
    when LANG='C' is set or an R version is used that was not compiled with
    natural language support (NLS), which would previously emit non-test-related
    warnings during testing (@dgkf, #1662; @heavywatal, #1689).

  • test_check() now suppresses hyperlinks since they'll take you to the wrong
    places (#1648).

  • New set_max_fails() helper to make it easier to set the maximum number of
    failures before stopping the test suite. And the advice to set to Inf is
    now clickable (#1628).

  • You can now configure the behaviour of the implicit
    devtools::load_all() call performed by devtools::test() in your
    package DESCRIPTION file (#1636). To disable exports of internal
    functions and of testthat helpers, use:

    Config/testthat/load-all: list(export_all = FALSE, helpers = FALSE)
    

    Helpers are now attached on the search path by default after calling
    devtools::test().

testthat 3.1.4

26 Apr 13:34
Compare
Choose a tag to compare
  • Minor tweaks to output for latest cli (#1606).