Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UPDATE: proper evaluation + replicability + maintenance #63

Merged
merged 5 commits into from
Aug 11, 2024

Conversation

MarekWadinger
Copy link
Contributor

@MarekWadinger MarekWadinger commented May 20, 2024

Dear @YKatser @waico,

first of all, thank you for very promising dataset for evaluation of detection algorithms. I used it within my previous publication to assess performance of my method and intended to use it further. I believe this could be very important test bed for plenty of researchers but I see couple of things which should be handled first:

  • simple installation procedure and non-conflicting dependencies
  • consistent results in the notebooks and the leaderboard
  • proper evaluation of the results

I propose the following changes to fix these issues and increase the overall quality of this repository:

  • poetry handled build management
  • removing dependency on toad
  • formating and linting pipelines - using GitHub automation and pre-commits
  • reasoning on proper evaluation - in the README leaderboard
    • separated train and test part for evaluation
    • moved window to the right side of true change point and changed to 60s to match the transition

I understand these changes are significant and may not fully align with your goals. Mainly TSAD seems to be very huge and dependency strict library. I extracted relevant part and made updates to align with new versions of dependencies. Please feel free to reintroduce it with the proposed updates.

Looking forward to work on this together.
Marek

@YKatser
Copy link
Collaborator

YKatser commented Jun 25, 2024

Dear @MarekWadinger,

thank you for the suggested changes, it is tremendous work you've done! I need a couple more weeks to look into it and merge.

Iurii

@YKatser YKatser merged commit 646f1d1 into waico:master Aug 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants