Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Anomaly Likelihood does not work correctly! #665

Open
Thanh-Binh opened this issue Sep 12, 2019 · 6 comments · May be fixed by #666
Open

Anomaly Likelihood does not work correctly! #665

Thanh-Binh opened this issue Sep 12, 2019 · 6 comments · May be fixed by #666
Labels
anomaly bug Something isn't working

Comments

@Thanh-Binh
Copy link

Hi all,
I have just tested anomaly detection with sinus waves by using

  1. TM with embedded Anomaly Likelihood
  2. TM with embedded Anomaly disable + external AnomalyLikelihood class
    For both cases, I used default values of AnomalyLikelihood.

In my observation:

  1. one-step-prediction provides very good prediction data, (good)
  2. raw anomaly has more and more small anomaly scores ...(good)
  3. BUT anomaly likelihood jumps between 0.5 (for some time period) to 1 (for other time period) and back to 0.5 and so on.

Does anyone have something like my observation? Thanks

@Thanh-Binh
Copy link
Author

By comparing my codes ported originally from python version, this version is totally different! This explains me!
I see that TM.anomaly seems to be likelihood, the anomaly score should be
1.0 - TM.anomaly

@breznak breznak linked a pull request Sep 16, 2019 that will close this issue
1 task
@breznak
Copy link
Member

breznak commented Sep 16, 2019

Related to review of anomaly likelihood #469

I have just tested anomaly detection with sinus waves by using

@Thanh-Binh could you share your simple experimentation code for sine waves? I'd like to integrate it to our tests as quality benchmark for different anomaly modes.

@Thanh-Binh
Copy link
Author

@breznak I will share it asap...

@breznak
Copy link
Member

breznak commented Sep 18, 2019

This could likely be joined with #646 , as the experiment on sines is the same, and we'd do both predictions & anomaly.

AFAIK the hotgym runs sine wave, @Thanh-Binh if you have problems porting your codes, can you explain on the existing hotgym where the problems are?

@breznak
Copy link
Member

breznak commented Sep 18, 2019

I see that TM.anomaly seems to be likelihood, the anomaly score should be
1.0 - TM.anomaly

I was able to confirm this, escalating to a bug. Thank you for reporting!

I was confused by "likelihood of anomaly" (which would imply high likelihood => anomaly), but it is a likelihood of a score (anomaly score). And we want to detect unlikely scores -> 1.0 - likelihood is correct anomaly interpretation here.

@breznak breznak added the bug Something isn't working label Sep 18, 2019
@breznak
Copy link
Member

breznak commented Sep 19, 2019

See #469 (comment) for some high level ideas about anomaly likelihood.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
anomaly bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants