[WIP] feat: Weights and Biases #1513

Ihab-Asaad · 2022-01-06T12:37:52Z

No description provided.

IgnatovFedor · 2022-01-21T10:34:16Z

deeppavlov/configs/classifiers/sentiment_twitter.json

@@ -3,7 +3,7 @@
    "class_name": "basic_classification_reader",
    "x": "Twit",
    "y": "Class",
-    "data_path": "{DOWNLOADS_PATH}/sentiment_twitter_data"
+    "data_path": "{DOWNLOADS_PATH}/sentiment_twitter_data/modified_data"


revert changes

IgnatovFedor · 2022-01-21T10:34:27Z

deeppavlov/configs/classifiers/sentiment_twitter.json

+        "save_path": "{MODEL_PATH}/new_model",
+        "load_path": "",


revert changes

IgnatovFedor · 2022-01-21T10:34:34Z

deeppavlov/configs/classifiers/sentiment_twitter.json

@@ -74,7 +74,7 @@
        "filters_cnn": 256,
        "optimizer": "Adam",
        "learning_rate": 0.01,
-        "learning_rate_decay": 0.1,
+        "learning_rate_decay": 0.01,


revert changes

IgnatovFedor · 2022-01-21T10:34:40Z

deeppavlov/configs/classifiers/sentiment_twitter.json

@@ -100,14 +100,17 @@
    ]
  },
  "train": {
-    "epochs": 100,
+    "epochs": 10,


revert changes

IgnatovFedor · 2022-01-21T10:34:53Z

deeppavlov/configs/classifiers/sentiment_twitter.json

+        "inputs": [
+          "y_onehot",
+          "y_pred_probas"
+        ]


Suggested change

"inputs": [

"y_onehot",

"y_pred_probas"

]

"inputs": ["y_onehot", "y_pred_probas"]

IgnatovFedor · 2022-01-21T10:36:42Z

deeppavlov/core/common/logging_class.py

@@ -0,0 +1,378 @@
+# Copyright 2019 Neural Networks and Deep Learning lab, MIPT


Suggested change

# Copyright 2019 Neural Networks and Deep Learning lab, MIPT

# Copyright 2022 Neural Networks and Deep Learning lab, MIPT

IgnatovFedor · 2022-01-21T10:40:30Z

requirements.txt

+wandb==0.12.7
+pybind11==2.2
+fasttext


remove additional requirements from library general requirements. Move wandb requirement to deeppavlov/requirements/wandb.txt

IgnatovFedor · 2022-01-21T10:46:02Z

deeppavlov/core/common/logging_class.py

+    There are three types of logging:
+        - StdLogger: for logging report about current training and validation processes to stdout.
+        - TensorboardLogger: for logging to tensorboard.
+        - WandbLogger: for logging to WandB.


remove types of logging description

IgnatovFedor · 2022-01-21T10:49:32Z

deeppavlov/core/common/logging_class.py

+
+    """
+
+    @abstractmethod


remove __init__ as it's empty (+ method is incorrect because there is no self argument)

IgnatovFedor · 2022-01-21T10:54:33Z

deeppavlov/core/common/logging_class.py

+        """
+        raise NotImplementedError
+
+    @abstractmethod


no need to make get_repost abstract

…ules

IgnatovFedor · 2022-01-24T11:14:38Z

requirements.txt

@@ -23,4 +23,4 @@ tqdm==4.62.0
 click==7.1.2
 uvicorn==0.11.7
 sacremoses==0.0.35
-uvloop==0.14.0
+uvloop==0.14.0


Suggested change

uvloop==0.14.0

uvloop==0.14.0

IgnatovFedor · 2022-01-24T13:37:35Z

deeppavlov/requirements/wandb.txt

+pybind11==2.2
+fasttext


Remove, because it doesn't used in WandbLogger

IgnatovFedor · 2022-01-31T09:17:05Z

deeppavlov/core/trainers/fit_trainer.py

@@ -50,7 +49,7 @@ class FitTrainer:
        evaluation_targets: data types on which to evaluate trained pipeline (default is ``('valid', 'test')``)
        show_examples: a flag used to print inputs, expected outputs and predicted outputs for the last batch
            in evaluation logs (default is ``False``)
-        tensorboard_log_dir: path to a directory where tensorboard logs can be stored, ignored if None
+        logger : list of dictionaries of possible loggers from deeppavlov.configs files.


Suggested change

logger : list of dictionaries of possible loggers from deeppavlov.configs files.

logger: list of dictionaries with train and evaluation loggers configuration

IgnatovFedor · 2022-01-31T09:18:06Z

deeppavlov/core/trainers/fit_trainer.py

+            log.info(
+                f'{self.__class__.__name__} got additional init parameters {list(kwargs)} that will be ignored:')


revert the change, because the initial line is shorter than 120 characters

IgnatovFedor · 2022-01-31T09:19:19Z

deeppavlov/core/trainers/fit_trainer.py

+        self._chainer = Chainer(
+            chainer_config['in'], chainer_config['out'], chainer_config.get('in_y'))


revert the change, because the initial line is shorter than 120 characters

IgnatovFedor · 2022-01-31T11:18:16Z

deeppavlov/core/trainers/nn_trainer.py

+            log.warning(
+                f"Using {self.__class__.__name__} for a pipeline without batched training"
+            )


revert changes

IgnatovFedor · 2022-01-31T11:18:23Z

deeppavlov/core/trainers/nn_trainer.py


        # Run the at-train-exit model-saving logic
        if self.validation_number < 1:
-            log.info('Save model to capture early training results')
+            log.info("Save model to capture early training results")


revert changes

IgnatovFedor · 2022-01-31T11:18:38Z

deeppavlov/core/trainers/nn_trainer.py

+from deeppavlov.core.common.logging.wandb_logger import WandbLogger
+from deeppavlov.core.common.logging.std_logger import StdLogger
+from deeppavlov.core.common.logging.tensorboard_logger import TensorboardLogger


remove imports

IgnatovFedor · 2022-01-31T11:19:33Z

deeppavlov/core/trainers/nn_trainer.py


    def save(self) -> None:
        if self._loaded:
-            raise RuntimeError('Cannot save already finalized chainer')
+            raise RuntimeError("Cannot save already finalized chainer")


revert changes

IgnatovFedor · 2022-01-31T11:20:26Z

deeppavlov/core/trainers/nn_trainer.py

+        if self.stdlogger_idx is not None:
+            self.std_logger = StdLogger(stdlogging=True)
+
+        if self.tensorboard_idx is not None:
+            self.tensorboard_logger = TensorboardLogger(
+                log_dir=self.logger[self.tensorboard_idx]["log_dir"]
+            )
+
+        if self.wandblogger_idx is not None:
+            if WandbLogger.login(
+                API_Key=self.logger[self.wandblogger_idx].get("API_Key", None),
+                relogin=True,
+            ):
+                if self.log_every_n_epochs > 0 or self.val_every_n_epochs > 0:
+                    self.wandb_logger = WandbLogger(
+                        log_on="epochs",
+                        commit_on_valid=self.val_every_n_epochs > 0,
+                        **self.logger[self.wandblogger_idx].get("init", None),
+                    )
+                    if self.wandb_logger.init_succeed == False:
+                        self.wandblogger_idx = None
+                elif self.log_every_n_batches > 0 or self.val_every_n_batches > 0:
+                    self.wandb_logger = WandbLogger(
+                        log_on="batches",
+                        commit_on_valid=self.val_every_n_batches > 0,
+                        **self.logger[self.wandblogger_idx].get("init", None),
+                    )
+                    if self.wandb_logger.init_succeed == False:
+                        self.wandblogger_idx = None
+            else:
+                self.wandblogger_idx = None


remove those lines from nn_trainer.py and move to init methods of according classes.

Ihab-Asaad added 12 commits November 23, 2021 14:16

Test: modify .json file

3f639f8

Deleting _log funtion and _validate from nn_trainer.py

a4da2b6

add docstrings to logging_class

cd142f8

correct data_path in .json file

1b9361f

Add WandbLogger

be97470

Add init

9b6f71e

Add wandb

eaf908d

Testing

968e00d

Testing

2ce99d8

Update logging_class

fb5f950

fix WandB

6998d06

Update WandB

de8df4a

IgnatovFedor self-requested a review January 6, 2022 12:38

Update init in W&B

0fde139

IgnatovFedor reviewed Jan 21, 2022

View reviewed changes

Ihab-Asaad added 5 commits January 22, 2022 13:12

Create logging directory and place different logger in separate modules

95a090c

Create logging directory and place different loggers in separate mod…

c3ee83d

…ules

make std logger default logging method

145c311

make std logger the default logging method

987da10

Add logging methods to configuration.rst

6306c01

IgnatovFedor reviewed Jan 31, 2022

View reviewed changes

delete loggers init from nn_trainer.py

27a940b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] feat: Weights and Biases #1513

[WIP] feat: Weights and Biases #1513

Ihab-Asaad commented Jan 6, 2022

IgnatovFedor Jan 21, 2022

IgnatovFedor Jan 21, 2022

IgnatovFedor Jan 21, 2022

IgnatovFedor Jan 21, 2022

IgnatovFedor Jan 21, 2022

IgnatovFedor Jan 21, 2022

IgnatovFedor Jan 21, 2022

IgnatovFedor Jan 21, 2022

IgnatovFedor Jan 21, 2022

IgnatovFedor Jan 21, 2022

IgnatovFedor Jan 24, 2022

IgnatovFedor Jan 24, 2022

IgnatovFedor Jan 31, 2022

IgnatovFedor Jan 31, 2022

IgnatovFedor Jan 31, 2022

IgnatovFedor Jan 31, 2022

IgnatovFedor Jan 31, 2022

IgnatovFedor Jan 31, 2022

IgnatovFedor Jan 31, 2022

IgnatovFedor Jan 31, 2022

		@@ -0,0 +1,378 @@
		# Copyright 2019 Neural Networks and Deep Learning lab, MIPT

	# Copyright 2019 Neural Networks and Deep Learning lab, MIPT
	# Copyright 2022 Neural Networks and Deep Learning lab, MIPT

	logger : list of dictionaries of possible loggers from deeppavlov.configs files.
	logger: list of dictionaries with train and evaluation loggers configuration

		log.info(
		f'{self.__class__.__name__} got additional init parameters {list(kwargs)} that will be ignored:')

		self._chainer = Chainer(
		chainer_config['in'], chainer_config['out'], chainer_config.get('in_y'))

		pybind11==2.2
		fasttext

[WIP] feat: Weights and Biases #1513

Are you sure you want to change the base?

[WIP] feat: Weights and Biases #1513

Conversation

Ihab-Asaad commented Jan 6, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment