-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add auto_series_param and custom_series_param #58
base: master
Are you sure you want to change the base?
Conversation
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## master #58 +/- ##
==========================================
- Coverage 91.70% 83.23% -8.47%
==========================================
Files 19 25 +6
Lines 747 1038 +291
==========================================
+ Hits 685 864 +179
- Misses 62 174 +112 ☔ View full report in Codecov by Sentry. |
Hello @dkrako, |
I added a |
Hello @dkrako, Could you please provide more details on the exact use-case for these new components to help me better understand the PR? A Colab notebook or a reproducible snippet would greatly assist in this. I am eager to merge it as soon as I have a clear understanding of the use-case(s), which will allow me to update the documentation in the README accordingly. Thank you! |
@dkrako what is the shape of the tensor that your model expects for time series images? |
Thanks a lot! The time series I am referring to are not really 2d-images but multi-channel 1d-series. I myself am working mostly with eye tracking data, where you can have 2 channels (horizontal and vertical component of the eye gaze) but there is a range of different applications you can imagine (EEG data, seismographic data, acceleration sensors, gyroscopes, climate readings for a specific location). Regardless of the application, the expected data layout in pytorch is NCL (tensorflow expects NLC), where N is the batch dimension, C the channel dimension and L the length (time) dimension. A series with the shape (64, 4, 1000) has a batch size of 64, 4 channels and 1000 time steps. Analogous to the well known 2d convolution using the pytorch I'll provide a notebook with a general use-case ASAP. I will use this single channel toy dataset here for the example: https://github.com/greydanus/mnist1d |
here is an example colab notebook I have just created: https://colab.research.google.com/drive/152dNe02_MzSzCxQwAfXY7rtvql9NfPKh?usp=sharing It loads and plots some data, trains a simple 1d-cnn. |
Do you mind giving me permission to run test workflows without further approval, please? |
The failing tests are the same error as the one in the colab notebook ( |
The current state of the package only supports 2d image-type input. Theorywise, it shouldn't be a problem to apply the functionality to time series inputs.
This PR adds the class
AutoSeriesParam
, which is designed according toAutoImageParam
, but supports sequences instead of images. This way 1d-support can be integrated neatly without changing any existing code.It is therefore needed to to setup the dreamer manually this way:
Maybe the series transforms could be setup by default in the dreamer, if the image_parameter is an instance of
BaseSeriesParam
?There are still some issues left where I am unsure what to do:
The
AutoImageParam
class has anormalize()
method to normalize the output. It uses imagenet mean and std to apply z-score standardization. There's no such equivalent in time series data due to their heterogeneous nature so we need a way to feed in the mean and std values. TheDreamer
class has aset_custom_normalization()
method for this use, but theInverseTransform
relies on 4d-tensors. I'm not sure how to integrate time series support into this.The
AutoImageParam.postprocess()
method applies the functionlucid_colorspace_to_rgb()
. I don't really understand what it does and I'm clueless on how to treat this in the time series domain.I left out the implementation of a
CustomSeriesParam
class for now, but that should come in a follow-up PR.