Use of rtdl package for multiple target variables. #43

VishnuTripathi · 2022-12-03T06:26:29Z

VishnuTripathi
Dec 3, 2022

Hi , Can we use rtdl package for multiple target variables (in a vector form) ?

Dec 3, 2022

Hi, yes you can, rtdl only provides PyTorch modules and does not impose any restrictions on the target. You can use the tutorial as a starting point: you will need to make targets 2-dimensional vectors instead of 1-dimensional. It is also possible to adjust the training loss and metrics if needed.

Does this help?

View full answer

Yura52 · 2022-12-03T08:04:42Z

Yura52
Dec 3, 2022
Maintainer

Hi, yes you can, rtdl only provides PyTorch modules and does not impose any restrictions on the target. You can use the tutorial as a starting point: you will need to make targets 2-dimensional vectors instead of 1-dimensional. It is also possible to adjust the training loss and metrics if needed.

Does this help?

0 replies

VishnuTripathi · 2023-02-05T18:19:25Z

VishnuTripathi
Feb 5, 2023
Author

Hi, as per our discussion I tried to predict 3 target variables (made it NumPy 2D array) from 134 predictors variables but I m getting the following error: -

===============================================================================================
Config: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced.toml
Output: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced

Creating the output...
[I 2023-02-05 18:01:58,652] A new study created in memory with name: no-name-cc21aa83-6ab7-4e1d-8dd7-7a4efdc0ad5d
0%| | 0/100 [00:00<?, ?it/s]=====================================
Config: /tmp/tmp9rksbq41/trial_0.toml
Output: /tmp/tmp9rksbq41/trial_0

Creating the output...
Using cached X: /home/tabular-dl-revisiting-models/data/fire/build_X__quantile__mean__new__indices__0.pickle
0%| | 0/6000000000 [00:00<?, ?it/s]

Epoch 1 | 0:00:00 | /tmp/tmp9rksbq41/trial_0
lr = 0.003481410351917467 | batch_size = 256 | epoch_size = 6 | n_parameters = 63507
/home/tabular-dl-revisiting-models/bin/mlp___a788218d03d748a1853f3cc87dc30492.py:233: UserWarning: Using a target size (torch.Size([256, 3])) that is different to the input size (torch.Size([256])). This will likely lead to incorrect results due to broadcasting. Please ensure they have the same size.
loss = loss_fn(
Traceback (most recent call last):
File "/home/tabular-dl-revisiting-models/bin/mlp___a788218d03d748a1853f3cc87dc30492.py", line 233, in
loss = loss_fn(
File "/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py", line 2659, in mse_loss
expanded_input, expanded_target = torch.broadcast_tensors(input, target)
File "/opt/conda/lib/python3.8/site-packages/torch/functional.py", line 71, in broadcast_tensors
return _VF.broadcast_tensors(tensors) # type: ignore
RuntimeError: The size of tensor a (256) must match the size of tensor b (3) at non-singleton dimension 1
0%| | 1/6000000000 [00:00<1501464:50:36, 1.11it/s]
[W 2023-02-05 18:02:05,936] Trial 0 failed because of the following error: CalledProcessError(1, ['python', '/home/tabular-dl-revisiting-models/bin/mlp___a788218d03d748a1853f3cc87dc30492.py', '/tmp/tmp9rksbq41/trial_0.toml'])
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/optuna/_optimize.py", line 217, in _run_trial
value_or_values = func(trial)
File "bin/tune.py", line 111, in objective
subprocess.run(
File "/opt/conda/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', '/home/tabular-dl-revisiting-models/bin/mlp___a788218d03d748a1853f3cc87dc30492.py', '/tmp/tmp9rksbq41/trial_0.toml']' returned non-zero exit status 1.
0%| | 0/100 [00:07<?, ?it/s]
Traceback (most recent call last):
File "bin/tune.py", line 185, in
study.optimize(
File "/opt/conda/lib/python3.8/site-packages/optuna/study.py", line 385, in optimize
_optimize(
File "/opt/conda/lib/python3.8/site-packages/optuna/_optimize.py", line 66, in _optimize
_optimize_sequential(
File "/opt/conda/lib/python3.8/site-packages/optuna/_optimize.py", line 163, in _optimize_sequential
trial = _run_trial(study, func, catch)
File "/opt/conda/lib/python3.8/site-packages/optuna/_optimize.py", line 268, in _run_trial
raise func_err
File "/opt/conda/lib/python3.8/site-packages/optuna/_optimize.py", line 217, in _run_trial
value_or_values = func(trial)
File "bin/tune.py", line 111, in objective
subprocess.run(
File "/opt/conda/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', '/home/tabular-dl-revisiting-models/bin/mlp___a788218d03d748a1853f3cc87dc30492.py', '/tmp/tmp9rksbq41/trial_0.toml']' returned non-zero exit status 1.

5 replies

Yura52 Feb 5, 2023
Maintainer

Hi! The message says that:

your model outputs 1d tensor (shape [256]). In other words, it produces predictions for only one target
but the target shape is (shape [256, 3]). Which is correct since you have three targets.

So you have to change the last layer of your model to make it produce the predictions for three targets. For different models the code will be different:

# MLP
model.head = nn.Linear(model.head.in_features, 3)

# ResNet
model.head.linear = nn.Linear(model.head.linear.in_features, 3)

# FTTransformer (same as for ResNet)
model.head.linear = nn.Linear(model.head.linear.in_features, 3)

Does this help?

which is suitable for a problem with one target. But you have three targets (shape [256, 3]), so you have to change t

VishnuTripathi Feb 5, 2023
Author

Hi, its throwing error like Tensor for 'out' is on CPU, Tensor for argument #1 'self' is on CPU, but expected them to be on GPU. I'm unable to figure it out.

===============================================================================================
Config: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced.toml
Output: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced

Creating the output...
[I 2023-02-05 18:45:16,856] A new study created in memory with name: no-name-c7146252-6f97-49ad-bbf4-96b2e88a39f3
0%| | 0/100 [00:00<?, ?it/s]=====================================
Config: /tmp/tmppp6_pziu/trial_0.toml
Output: /tmp/tmppp6_pziu/trial_0

Creating the output...
Using cached X: /home/tabular-dl-revisiting-models/data/fire/build_X__quantile__mean__new__indices__0.pickle
0%| | 0/6000000000 [00:00<?, ?it/s]

Epoch 1 | 0:00:00 | /tmp/tmppp6_pziu/trial_0
lr = 0.003481410351917467 | batch_size = 256 | epoch_size = 6 | n_parameters = 63507
Traceback (most recent call last):
File "/home/tabular-dl-revisiting-models/bin/mlp___7b9823c119ec4f6295b02699d70cce3b.py", line 237, in
model(
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/tabular-dl-revisiting-models/bin/mlp___7b9823c119ec4f6295b02699d70cce3b.py", line 63, in forward
x = self.head(x)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 93, in forward
return F.linear(input, self.weight, self.bias)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py", line 1690, in linear
ret = torch.addmm(bias, input, weight.t())
RuntimeError: Tensor for 'out' is on CPU, Tensor for argument #1 'self' is on CPU, but expected them to be on GPU (while checking arguments for addmm)
0%| | 1/6000000000 [00:00<1394249:59:43, 1.20it/s]
[W 2023-02-05 18:45:23,417] Trial 0 failed because of the following error: CalledProcessError(1, ['python', '/home/tabular-dl-revisiting-models/bin/mlp___7b9823c119ec4f6295b02699d70cce3b.py', '/tmp/tmppp6_pziu/trial_0.toml'])
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/optuna/_optimize.py", line 217, in _run_trial
value_or_values = func(trial)
File "bin/tune.py", line 111, in objective
subprocess.run(
File "/opt/conda/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', '/home/tabular-dl-revisiting-models/bin/mlp___7b9823c119ec4f6295b02699d70cce3b.py', '/tmp/tmppp6_pziu/trial_0.toml']' returned non-zero exit status 1.
0%| | 0/100 [00:06<?, ?it/s]
Traceback (most recent call last):
File "bin/tune.py", line 185, in
study.optimize(
File "/opt/conda/lib/python3.8/site-packages/optuna/study.py", line 385, in optimize
_optimize(
File "/opt/conda/lib/python3.8/site-packages/optuna/_optimize.py", line 66, in _optimize
_optimize_sequential(
File "/opt/conda/lib/python3.8/site-packages/optuna/_optimize.py", line 163, in _optimize_sequential
trial = _run_trial(study, func, catch)
File "/opt/conda/lib/python3.8/site-packages/optuna/_optimize.py", line 268, in _run_trial
raise func_err
File "/opt/conda/lib/python3.8/site-packages/optuna/_optimize.py", line 217, in _run_trial
value_or_values = func(trial)
File "bin/tune.py", line 111, in objective
subprocess.run(
File "/opt/conda/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['python', '/home/tabular-dl-revisiting-models/bin/mlp___7b9823c119ec4f6295b02699d70cce3b.py', '/tmp/tmppp6_pziu/trial_0.toml']' returned non-zero exit status 1.

Yura52 Feb 5, 2023
Maintainer

The code snippets that I mentioned above should be added before the line model.to(device).

VishnuTripathi Feb 9, 2023
Author

Hi, can you tell me what are the modifications needed for running other algorithms like lightgbm, catboost, xgboost, dcn2 etc.

Yura52 Feb 10, 2023
Maintainer

The rtdl package only provides an interface to the main models from the paper "Revisiting Deep Learning Models for Tabular Data".
The source code for the paper itself with the algorithms you mentioned is available in a separate repository: https://github.com/yandex-research/rtdl

VishnuTripathi · 2023-02-05T19:41:53Z

VishnuTripathi
Feb 5, 2023
Author

It worked.
Thank you for prompt reply

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use of rtdl package for multiple target variables. #43

{{title}}

Replies: 3 comments 5 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Use of rtdl package for multiple target variables. #43

VishnuTripathi Dec 3, 2022

Replies: 3 comments · 5 replies

Yura52 Dec 3, 2022 Maintainer

VishnuTripathi Feb 5, 2023 Author

=============================================================================================== Config: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced.toml Output: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced

Creating the output... [I 2023-02-05 18:01:58,652] A new study created in memory with name: no-name-cc21aa83-6ab7-4e1d-8dd7-7a4efdc0ad5d 0%| | 0/100 [00:00<?, ?it/s]===================================== Config: /tmp/tmp9rksbq41/trial_0.toml Output: /tmp/tmp9rksbq41/trial_0

Yura52 Feb 5, 2023 Maintainer

VishnuTripathi Feb 5, 2023 Author

=============================================================================================== Config: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced.toml Output: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced

Creating the output... [I 2023-02-05 18:45:16,856] A new study created in memory with name: no-name-c7146252-6f97-49ad-bbf4-96b2e88a39f3 0%| | 0/100 [00:00<?, ?it/s]===================================== Config: /tmp/tmppp6_pziu/trial_0.toml Output: /tmp/tmppp6_pziu/trial_0

Yura52 Feb 5, 2023 Maintainer

VishnuTripathi Feb 9, 2023 Author

Yura52 Feb 10, 2023 Maintainer

VishnuTripathi Feb 5, 2023 Author

VishnuTripathi
Dec 3, 2022

Replies: 3 comments 5 replies

Yura52
Dec 3, 2022
Maintainer

VishnuTripathi
Feb 5, 2023
Author

===============================================================================================
Config: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced.toml
Output: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced

Creating the output...
[I 2023-02-05 18:01:58,652] A new study created in memory with name: no-name-cc21aa83-6ab7-4e1d-8dd7-7a4efdc0ad5d
0%| | 0/100 [00:00<?, ?it/s]=====================================
Config: /tmp/tmp9rksbq41/trial_0.toml
Output: /tmp/tmp9rksbq41/trial_0

Yura52 Feb 5, 2023
Maintainer

VishnuTripathi Feb 5, 2023
Author

===============================================================================================
Config: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced.toml
Output: /home/tabular-dl-revisiting-models/output/california_housing/mlp/tuning/reproduced

Creating the output...
[I 2023-02-05 18:45:16,856] A new study created in memory with name: no-name-c7146252-6f97-49ad-bbf4-96b2e88a39f3
0%| | 0/100 [00:00<?, ?it/s]=====================================
Config: /tmp/tmppp6_pziu/trial_0.toml
Output: /tmp/tmppp6_pziu/trial_0

Yura52 Feb 5, 2023
Maintainer

VishnuTripathi Feb 9, 2023
Author

Yura52 Feb 10, 2023
Maintainer

VishnuTripathi
Feb 5, 2023
Author