Add logging support to PennyLane's execution pipeline (#4383)

* Add initial logging functionality

* Add logging to autograd execution interface

* Add logging to qnode creation

* Allow more verbose messages

* Add support for log filters

* Add logging to fallback execution pipeline

* Log torch native and interface usage

* simplify config file

* Add TF logging support

* Improve docs

* Add more guidelines for logging

* Rewrite log toml for expressive control

* Ensure NullHandler always defined

* Add null handlers for loggers

* Show expressivity of controls with config

* Update docs and add default log config

* fix spelling errors

* Move color coding section

* Fix wording

* Update pennylane/interfaces/autograd.py

* Update pennylane/logging/log_config.yaml

* Update pennylane/logging/log_config.yaml

* Update pennylane/logging/log_config.yaml

* Update pennylane/logging/log_config.yaml

* Fix formatting

* Follow pylint recs

* Readd trace level for qnode

* Move colormap

* Remove dataclass

* Update toml lib deps

* Disable linter for custom log-levels

* Add formatter module docs

* Update pennylane/logging/README.md

Co-authored-by: Tom Bromley <49409390+trbromley@users.noreply.github.com>

* Provide user access to configuration file path and improve configuration structure (#4377)

* move and add logging configuration logic to the configuration module

* update __init__ file

* update toml configuration file

* update yaml configuration file

* format

* Update custom implementation

* add __init__ docstring

* review suggestions

* move configure_logging to internal and add config_path

* format

* update changelog

* Update doc/releases/changelog-dev.md

* Remove redundancies and move trace level definition

* Update propagation for test coverage

* Add autograd log tests

* Remove shortname log-statement

* Update log-test messaging and structure

* Update changelog

* Add log-level docs to site

* Remove unneeded import

* Undo propagate logger records

* Ensure log propagation is handled for test capture

* Update doc/development/guide/logging.rst

Co-authored-by: Matthew Silverman <matthews@xanadu.ai>

* Update doc/development/guide/logging.rst

Co-authored-by: Christina Lee <christina@xanadu.ai>

* Update pennylane/logging/log_config.toml

Co-authored-by: Matthew Silverman <matthews@xanadu.ai>

* Update doc/development/guide/logging.rst

Co-authored-by: Matthew Silverman <matthews@xanadu.ai>

* Update doc/development/guide/logging.rst

Co-authored-by: Matthew Silverman <matthews@xanadu.ai>

* Fix guide wording and log levels

* Remove redundant files and add licenses

* Pylint fix

* Create logging quickstart page

* Disable qml_debug.log unless configured for use

* Ensure qml_debug.log ignored

* Explain stdout vs file logging outputs

* Update inspection and logging supports

* Simplify logging tests for gradients

* Remove readme in favour of developer docs

* Update tests/logging/test_logging_autograd.py

* Appease pylint

* update copyright year in logging files

* Update tests/logging/test_logging_autograd.py

* Ensure callable checks for tracing levels

* Update doc/development/guide.rst

Co-authored-by: Tom Bromley <49409390+trbromley@users.noreply.github.com>

* Update doc/development/guide/logging.rst

Co-authored-by: Tom Bromley <49409390+trbromley@users.noreply.github.com>

* Update doc/development/guide/logging.rst

Co-authored-by: Tom Bromley <49409390+trbromley@users.noreply.github.com>

* Add mention of the config file path

* Add space

* Update doc/development/guide/logging.rst

Co-authored-by: Frederik Wilde <42576579+frederikwilde@users.noreply.github.com>

* Update doc/development/guide/logging.rst

Co-authored-by: Frederik Wilde <42576579+frederikwilde@users.noreply.github.com>

* Update links, and docs based on review comments

* Add logging code to docs

* Pin to same sphinx version across all Python versions

* Restore sphinx3.5, but incr bugfix version

* Default JAX to WARN log level

* Make main logger config optional in dev guide

* Update pennylane/logging/configuration.py

Co-authored-by: Josh Izaac <josh146@gmail.com>

* Remove inheritence hierarchy from docs

* Update logging method docstrings

* Update docstrings and info

* Ensure toml config is tracked for wheel distribution

* Ensure formatter links display in docs

* Finally fix sphinx ref

* Remove TRACE from external docstrings

---------

Co-authored-by: Tom Bromley <49409390+trbromley@users.noreply.github.com>
Co-authored-by: Amintor Dusko <87949283+AmintorDusko@users.noreply.github.com>
Co-authored-by: Matthew Silverman <matthews@xanadu.ai>
Co-authored-by: Christina Lee <christina@xanadu.ai>
Co-authored-by: Frederik Wilde <42576579+frederikwilde@users.noreply.github.com>
Co-authored-by: Josh Izaac <josh146@gmail.com>

.gitignore

@@ -24,4 +24,5 @@ benchmark/revisions/
 venv
 config.toml
 .envrc
-datasets/*
+qml_debug.log
+datasets/*

MANIFEST.in

@@ -4,3 +4,4 @@ include LICENSE
 include tests/*
 include example_config.toml
 include pennylane/transforms/qcut/_cut_kKaHyPar_sea20.ini
+include pennylane/logging/log_config.toml

doc/code/qml_logging.rst

@@ -0,0 +1,9 @@
+qml.logging
+================
+
+.. currentmodule:: pennylane.logging
+
+.. automodapi:: pennylane.logging
+    :no-heading:
+    :include-all-objects:
+    :skip: TRACE

doc/development/guide.rst

@@ -40,6 +40,11 @@ of PennyLane, plugins, and devices.
     :description: Proposing important PennyLane architectural decisions.
     :link: guide/adr.html
 
+.. title-card::
+    :name: Logging guidelines
+    :description: Defining how to develop using logging supports in PennyLane.
+    :link: guide/logging.html
+
 .. .. title-card::
 ..     :name: Best practices
 ..     :description: General best practices for contributing and developing for PennyLane.
@@ -65,4 +70,5 @@ of PennyLane, plugins, and devices.
     guide/pullrequests
     guide/adr
     guide/architecture
+    guide/logging
 ..    guide/bestpractices