v0.5.0

• Added make_run_as fixture (#82). A new pytest fixture, make_run_as, has been added to create an account service principal via the acc fixture and assign it to a workspace with default permissions, which is removed after the test is complete. The fixture creates a service principal with a random display name and assigns it to the workspace. Users can optionally assign the service principal to account groups for specific actions using the account_groups argument. The returned object contains properties for the workspace client, SQL backend, and SQL execution functions, as well as the display name and application ID of the ephemeral service principal. If desired, the ws fixture can be overridden to make all workspace fixtures provided by the plugin run as the ephemeral service principal, allowing for testing with lower privilege ephemeral service principals and improving security and isolation. This feature is not currently supported with Databricks Metadata Service authentication on Azure Databricks.
• Bump codecov/codecov-action from 4 to 5 (#85). In this release, the Codecov GitHub Action has been updated from version 4 to 5, introducing several new features and changes. The new version uses the Codecov Wrapper to encapsulate the CLI, allowing for quicker updates to the Action. Additionally, version 5 includes an opt-out feature for tokens in public repositories, enabling contributors and other members to upload coverage reports without requiring access to the Codecov token. This can be accomplished by setting the ability for Codecov to receive a coverage report from any source in the Global Upload Token section of the settings page on codecov.io. Furthermore, the updated version introduces several new arguments, including binary, gcov_args, gcov_executable, gcov_ignore, gcov_include, report_type, and skip_validation, and changes the file and plugin arguments to files and plugins, respectively.
• Bump databrickslabs/sandbox from acceptance/v0.3.1 to 0.4.2 (#80). In this release, the databrickslabs/sandbox dependency has been updated from version acceptance/v0.3.1 to 0.4.2. This update includes the addition of install instructions, more go-git libraries, and modifications to the README to explain how to use the library with the databricks labs sandbox command. Dependency updates include golang.org/x/crypto from version 0.16.0 to 0.17.0. The Run nightly tests job in the workflow has also been updated to use the new version of the databrickslabs/sandbox/acceptance image. The commit history for this release shows several commits, including the creation of "[TODO] XXX" issues and a full diff comparison. The pull request includes instructions for triggering Dependabot actions through comments. Reviewers are encouraged to thoroughly examine the changelog and commit history for more detailed information on the changes in this release.
• Force keyword argument in make_query fixture (#81). In the latest update, the make_query fixture in the redash.py file has undergone changes to enhance code readability and maintainability. The sql_query parameter is now required to be passed as a keyword argument, preventing the possibility of it being mistakenly passed as a positional argument. Furthermore, the create function's signature has been revised to include an explicit * before the sql_query parameter. It is important to note that these changes have not affected the functionality of the method, but have instead altered the parameter passing style for improved clarity. This minor update is intended to elevate the overall quality of the codebase and promote best practices.
• Renamed internal notebooks module to workspace (#86). In this release, the internal notebooks module has been renamed to workspace to better reflect its current functionality of managing and interacting with notebooks and other resources in a Databricks workspace. This renaming applies to the import statements in plugin.py and test_notebooks.py, which has been renamed to test_workspace.py. Additionally, import statements for making cluster policy and instance pool permissions have been added. The functionality of the imported functions, such as make_directory, make_workspace_file, make_notebook, and make_repo, remains unchanged. This change is part of the fix for issue #59 and aims to improve the clarity and consistency of the codebase. Software engineers adopting this project should update any imports or references to the notebooks module to use the new workspace module instead.

Dependency updates:

• Bump databrickslabs/sandbox from acceptance/v0.3.1 to 0.4.2 (#80).
• Bump codecov/codecov-action from 4 to 5 (#85).

Contributors: @dependabot[bot], @nfx, @JCZuurmond

v0.4.0

• Added Volume Fixture (#72). This commit introduces a Managed Volume fixture, make_volume, to the Unity Catalog in the test suite, facilitating the creation and use of a random volume for testing purposes. The fixture, when called without arguments, generates a volume with a random name. Alternatively, specifying the name argument creates a volume with the given name, using the MANAGED volume type, and associating it with a randomly generated catalog and schema. This promotes test isolation and prevents unintended interference. Additionally, this PR resolves issue #70 and includes unit and integration tests that have been manually verified to ensure the fixture's proper functioning. The commit also demonstrates a bug fix related to table retrieval in the test_remove_after_property_table test case.

Contributors: @christophergrant

v0.3.1

• Updated databrickslabs/sandbox requirement to acceptance/v0.3.1 (#64). In this pull request, we are updating the requirement for the databrickslabs/sandbox package to version acceptance/v0.3.1. This update is necessary to resolve any conflicts and ensure compatibility with the latest version of the package. The update includes several bug fixes, dependency updates, and the addition of install instructions in the changelog. The package also includes new git-related libraries and modifications to the README to explain how to use the package with the databricks labs sandbox command. Additionally, there are dependency updates for golang.org/x/crypto in the go-libs and runtime-packages directories. Keeping dependencies up-to-date is important for taking advantage of new features and bug fixes, so it is recommended to merge this pull request and address any conflicts or issues that may arise. This update is generated by Dependabot, a tool used to keep dependencies up-to-date, and it will handle any conflicts as long as the pull request is not modified. The databrickslabs/sandbox package is used for building and testing the project.
• Updating make_schema fixture to include location to create managed schema (#66). In this update, the make_schema fixture has been enhanced to include an optional location parameter, enabling users to specify the location for creating a managed schema. This change addresses issue #6
• [chore] update acceptance.yml and remove circular downstreads. In this release, the acceptance.yml file in the .github/workflows directory has been updated to use the acceptance/v0.3.1 Docker image for the Run integration tests job, replacing the previous acceptance/v0.2.2 version. This change ensures that the latest version of the integration tests are being run, which may include bug fixes, new features, and other improvements. The timeout parameter has been removed from the job, which may result in longer test execution times but ensures that the tests have enough time to complete. Additionally, the ARM_CLIENT_ID and GITHUB_TOKEN secrets are now passed as environment variables to the job. The downstreams.yml file has also been updated to remove circular dependencies and streamline the build process, while maintaining the use of the ubuntu-latest runtime and fail-fast option. The Run nightly tests job has been updated to use the acceptance/v0.3.1 Docker image, which may improve test execution during the nightly build due to potential bug fixes, new functionality, or better handling of timeout scenarios. The create_issues parameter remains set to true, ensuring that any test failures will result in the creation of GitHub issues to track them. These changes aim to improve the reliability and efficiency of the build and test processes.

Dependency updates:

• Updated databrickslabs/sandbox requirement to acceptance/v0.3.1 (#64).

Contributors: @nfx, @dependabot[bot], @HariGS-DB

v0.3.0

• Documentation: fix make_query() parameter name (#61). The make_query() fixture's documentation has been updated to correct the name of the query parameter to sql_query. The sql_query parameter is used to specify the SQL query stored in the fixture, with the default value being "SELECT * FROM ". This change aims to enhance clarity and consistency in the naming of the argument, making it easier for users of the make_query() fixture to comprehend its purpose and usage. By correcting the parameter name, the documentation provides a clearer and more consistent user experience.
• Removed references to UCX (#56). This release includes changes to remove references to UCX in fixture names and descriptions within the testing process. The create function in catalog.py now appends a random string to "dummy_t", "dummy_s", or dummy_c for table, schema, and catalog names, respectively, instead of using "ucx_t", "ucx_S", and "ucx_C". The test_catalog_fixture function has also been updated to replace dummy with dummy_c and dummy_s for catalogs and schemas. Additionally, the description of a test query in redash.py has been updated to remove the reference to UCX. Lastly, fixture names in the unit tests for a catalog have been updated to use dummy instead of "ucx". These changes improve the independence of the testing process by removing technology-specific references, without affecting functionality.
• Store watchdog tags in storage credentials comment (#57). In this release, the watchdog's behavior has been modified to retain properly tagged credentials when deleting them, as previously all credentials were removed without discrimination. This change introduces tagging for preserving specific credentials, and the watchdog_remove_after fixture has been added to the README file for documentation. The make_storage_credential fixture has been updated to include a new parameter, watchdog_remove_after, which specifies the time at which the storage credential should be removed by the watchdog. The create function has been updated to accept this parameter and adds it as a comment to the storage credential. The remove function remains unmodified. The related fixtures section has been updated to include the new watchdog_remove_after fixture. This change was co-authored by Eric Vergnaud, but please note that it has not been tested yet.
• [FEATURE] Extend make_job to run SparkPythonTask (#60). The make_job fixture has been extended to support running SparkPythonTask in addition to notebook tasks. A new make_workspace_file fixture has been added to create and manage Python files in the workspace. The make_job fixture now supports SQL notebooks and files and includes a task_type parameter to specify the type of task to run and an instance_pool_id parameter to reuse an instance pool for faster job execution during integration tests. Additionally, unit and integration tests have been added to ensure the proper functioning of the new and modified fixtures. These changes allow for more flexible and efficient testing of Databricks jobs with different task types and configurations. The make_notebook fixture has also been updated to accept a content parameter for creating notebooks with custom content. The Language enum from the databricks.sdk.service.workspace module is used to specify the language of a notebook or workspace file.

Contributors: @JCZuurmond, @ericvergnaud, @asnare

v0.2.4

• Fixed PyPI metadata (#54). In this commit, the PyPI metadata for the pytester project has been updated with the new repository location at https://github.com/databrickslabs/pytester. The URLs for issues and source have been changed to point to the new repository, with the issues URL now directing to https://github.com/databrickslabs/pytester/issues and the source URL to https://github.com/databrickslabs/pytester. Furthermore, the versioning tool hatch has been configured to manage the version number in the "src/databricks/labs/pytester/about.py" file. This ensures accurate and consistent versioning for the pytester project moving forward.
• Improve make_group/make_acc_group fixture consistency (#50). This PR introduces improvements to the make_group and make_acc_group fixtures, designed for managing Databricks workspace groups. The enhancements include a double-check approach to ensure group visibility by requiring the group to be retrievable via both .get() and .list() calls. This mitigates, but does not entirely eliminate, consistency issues with the APIs used for managing groups. The wait_for_provisioning argument has been removed and replaced with an internal wait mechanism. The argument is still accepted but triggers a deprecation warning. Internal unit-test plumbing has been updated to use mock fixtures tailored for each test, ensuring double-check implementation testability. New and updated unit tests are included in the test_iam.py file, along with the introduction of the _setup_groups_api function, which mocks specific clients to ensure group visibility when created. These changes improve consistency and reliability when working with Databricks workspace groups, making it easier for users to adopt the project.

Contributors: @asnare, @nfx

v0.2.3

• Support providing name in make_catalog fixture (#52). The make_catalog fixture in our open-source library has been updated to allow users to specify a name for the catalog using a new name parameter. Previously, the catalog was given a random name, but now users can have more control and customization over catalog names in their tests. This change includes updates to the docstring and the addition of unit tests to ensure the fixture behaves as expected with the new parameter. Additionally, the underlying call_stateful function was updated to expect a callable that returns a generator of callables, enabling the support for providing a name. The test_make_catalog_creates_catalog_with_name and test_make_catalog tests have been added to verify the behavior of the fixture with the new name parameter.

Contributors: @asnare, @JCZuurmond

v0.2.2

• Use watchdog timeout to catalog properties (#48). This pull request introduces a new RemoveAfter property for catalogs, which allows for marking them for skipping by the watchdog. This change addresses the current implementation gap, which does not explicitly indicate when catalogs are being used. The new property will specify the time from which objects can be purged. A corresponding fixture watchdog_remove_after has been added to the list of available fixtures, and the make_catalog fixture has been updated to include this new property. Additionally, a timeout mechanism for catalogs has been implemented, which improves the system's efficiency and safety by marking catalogs as in use. A test for the make_catalog function has been included to ensure that the RemoveAfter entry is correctly added to the catalog properties. However, the specific call parameters for the catalogs.create method cannot be accurately determined in the test.
• use tags instead of name suffix for queries (#47). This release introduces updates to the testing library for Databricks, enhancing the naming conventions for queries to improve readability and comprehension. The previous implementation used name suffixes, which have been replaced with watchdog query tags. The watchdog_purge_suffix fixture has been renamed to watchdog_remove_after, and the new make_query fixture has been added to the documentation. In addition, the make_query and create functions now accept an optional tags argument, and the query name is generated with a unique identifier. If tags are provided, the RemoveAfter tag is added. The original_query_tag is no longer hardcoded in the create function and has been removed. These changes improve the overall user experience and maintainability of the project.

Contributors: @ericvergnaud

v0.2.1

• Moved remaining UCX integration tests and fixtures (#45). In this release, we have made significant changes to the UCX integration tests and fixtures, as indicated by multiple commit messages. Firstly, we have moved remaining UCX integration tests and fixtures, introducing a new PyTest fixture called Installation in the README.md file, providing instructions on how to add databricks-labs-pytester as a test-time dependency when using hatch as the build system. Additionally, we have added the make_feature_table fixture, which creates a Databricks feature table and cleans it up after the test, taking optional parameters for customization. We have also modified the mypy configuration in the pyproject.toml file to allow untyped imports during the type-checking process. In the compute.py file, we have updated the make_job fixture to return a function that creates a databricks.sdk.service.jobs.Job instance, and modified the create function to return the databricks.sdk.service.jobs.Job instance directly. We have also added a new fixture called make_feature_table in the plugin file, which simulates the lifecycle of a feature table in the machine learning service, with functions to generate a unique name and create/remove the feature table. In the test_catalog.py file, we have made changes to clean up the file and ensure proper logging of test events and errors. Overall, these changes aim to refactor, expand functionality, and improve user-friendliness for the adopters of the project, ensuring proper logging and debugging capabilities.
• [internal] port over existing UCX integration tests (#44). Three new integration tests have been added to the UCX project to verify the functionality of the RemoveAfter property for tables and schemas. The test_remove_after_property_table and test_remove_after_property_schema tests create new tables and schemas, respectively, and check if the RemoveAfter property is included in their properties. However, these tests are still marked as TODO due to existing issues with the tables.get and schemas.get functions. In addition, existing UCX integration tests have been ported over, which include new functions for testing the removal of resources based on the RemoveAfter tag. These tests are located in the tests/integration/fixtures/test_compute.py file and test the removal of various types of resources, including jobs, clusters, warehouses, and instance pools. The tests ensure that the time until purge is less than the TEST_RESOURCE_PURGE_TIMEOUT value plus one hour and import the datetime module and the TEST_RESOURCE_PURGE_TIMEOUT constant from the watchdog fixture, as well as the logging and databricks.sdk.service.iam modules.

Contributors: @nfx

v0.2.0

• Added acc and make_acc_group fixtures (#42). In this release, we have added two new fixtures, acc and make_acc_group, to the open-source library. The acc fixture provides a Databricks AccountClient object for use in tests, which can interact with the Databricks account API and automatically determines the account host from the DATABRICKS_HOST environment variable. The make_acc_group fixture is used for managing Databricks account groups, creating them with specified members and roles, and automatically deleting them after the test is complete. This fixture mirrors the behavior of the make_group fixture but interacts with the account client instead of the workspace client. These fixtures enable more comprehensive integration tests for the acc object and its various methods, enhancing the testing and management of Databricks account groups.

Contributors: @nfx

v0.1.1

• Fixed nightly CI builds (#40). In this release, we have removed the no-cheat GitHub Actions workflow that checked for disables pylint directives in new code. We have also updated the pytest requirement version to ~8.3.3 and added badges for Python version support and lines of code to the README file. The permissions.py file in the databricks/labs/pytester/fixtures directory has been updated to fix nightly CI builds by improving import statements and updating types. The SqlPermissionLevel class has been imported from the databricks.sdk.service.sql module, and an existing test case has been updated to use this new permission level for SQL-specific queries. Additionally, we have updated the version constraints for three dependencies in the pyproject.toml file to allow for more flexibility in selecting compatible library versions. These changes may simplify the project's GitHub Actions workflows, reduce maintenance overhead, and enhance the testing process and code quality.

Contributors: @nfx