[Improvement] speed up confusion matrix calculation

[dreamerlin]

This PR uses np.bincount to speed up confusion matrix calculation

[codecov]

Codecov Report

Merging #465 (626e073) into master (30ff6b2) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master     #465   +/-   ##
=======================================
  Coverage   84.67%   84.67%           
=======================================
  Files         118      118           
  Lines        8347     8348    +1     
  Branches     1366     1365    -1     
=======================================
+ Hits         7068     7069    +1     
  Misses        932      932           
  Partials      347      347           

Flag	Coverage Δ
unittests	84.66% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmaction/core/evaluation/accuracy.py	93.18% <100.00%> (+0.03%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 30ff6b2...1d773d4. Read the comment docs.

[innerlee]

Any numbers on benchmark?

[dreamerlin]

classes = 10
length = 100
times = 10000

start = time.time()
for i in range(times):
    labels = np.random.randint(0, classes, length).astype(np.int64)
    pred = np.random.randint(0, classes, length).astype(np.int64)
    new_confusion_matrix(pred, labels)  # using np.bincount
end = time.time()
print(end - start)

start = time.time()
for i in range(times):
    labels = np.random.randint(0, classes, length).astype(np.int64)
    pred = np.random.randint(0, classes, length).astype(np.int64)
    confusion_matrix(pred, labels)  # original one
end = time.time()
print(end - start)

0.5623 vs 1.1669

[dreamerlin]

classes = 100
length = 10
times = 10000

start = time.time()
for i in range(times):
    labels = np.random.randint(0, classes, length).astype(np.int64)
    pred = np.random.randint(0, classes, length).astype(np.int64)
    new_confusion_matrix(pred, labels)
end = time.time()
print(end - start)

start = time.time()
for i in range(times):
    labels = np.random.randint(0, classes, length).astype(np.int64)
    pred = np.random.randint(0, classes, length).astype(np.int64)
    confusion_matrix(pred, labels)
end = time.time()
print(end - start)

0.5215 vs 0.5446

[dreamerlin]

The modification can truly speed up the calculation

[kennymckormick]

I think it's OK to use a dictionary instead of np array here, as the original one:

label_map = {label: i for i, label in enumerate(label_set)}

so that 4 lines -> 1 line
any performance concern here?

[dreamerlin]

label_map = {label: i for i, label in enumerate(label_set)}
y_pred_mapped = [label_map[i] for i in y_pred]
y_real_mapped = [label_map[i] for i in y_real]

This is much slower even than the original one.

[innerlee]

numbers would be appreciated

[kennymckormick]

That's right, the for loop is much slower.

[dreamerlin]

The proposal by kenny VS The original one before this PR:
for classes = 10, length = 100, times = 10000, 1.2643 vs 1.1462
for classes = 10, length = 100, times = 10000, 0.6333 vs 0.5554
even worse