-
Notifications
You must be signed in to change notification settings - Fork 13
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mistake: Cost New #8
Comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi, I am currently working on the paper that you published with this repository.
I am trying to add an example of your work in the galery of examples of
ruptures
. See the PR here.I think I spot a mistake that would change all the results. I would like to know if what I think is true @YKatser.
Let's take the case of Window Ensemble. Going through the code I realized that the object
CostNew()
is only initialized once before going through the entire dataset. For example, in the notebook TEP_experiment_with_NAB_metric there is:This seems not to be a problem since you call the method
cost.fit()
for each new signal.The main problem is coming from the following loop in
CostNew().fit()
:This means that for the first signal you go through,
self.metric
will be none. It will be changed and fixed for all the other signal coming afterwards. It is a shame since you have shown thatMahalanobis
is the best performing single cost. This is the reason why they have added the variableself.has_custom_metric
in the packageruptures
. This was a fix made by a PR.Here is what I got by initializing
CostNew()
for each signal:As a reminder, this is the former results that you have in the paper:
Here, you can see that the best performing aggregation function and scaling function have changed.
As I was curious to see what would be the results for the other experiments, I run the experiments again with this fix in a fork.
ps: I had a lot of fun working on your paper. Thanks a lot for your work and your interesting ideas :)
The text was updated successfully, but these errors were encountered: