Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

NDCG@10 fails when relevant documents are expressed in gain -> docs format #35

Closed
agazzarini opened this issue Jul 23, 2018 · 1 comment
Assignees
Labels
bug Something isn't working
Milestone

Comments

@agazzarini
Copy link
Member

Describe the bug
NDCG@10 fails when relevant documents are expressed in gain -> docs format

"relevant_documents": { "3": ["1658090", "8927398273", "9879878"] }
To Reproduce
Create a judgment set using the gain -> docs format for expressing judgments.

Expected behavior
The format should be correctly parsed by all RRE components

StackTrace
Caused by: java.lang.NullPointerException at io.sease.rre.core.domain.metrics.impl.NDCGAtTen.lambda$idealDcg$0(NDCGAtTen.java:75) at java.util.stream.Collectors.lambda$groupingBy$45(Collectors.java:907) at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at java.util.Iterator.forEachRemaining(Iterator.java:116) at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) at io.sease.rre.core.domain.metrics.impl.NDCGAtTen.idealDcg(NDCGAtTen.java:75) at io.sease.rre.core.domain.metrics.impl.NDCGAtTen.access$200(NDCGAtTen.java:24) at io.sease.rre.core.domain.metrics.impl.NDCGAtTen$1.value(NDCGAtTen.java:61) at io.sease.rre.core.domain.DomainMember.lambda$null$4(DomainMember.java:120) at java.util.Optional.ifPresent(Optional.java:159)

@agazzarini agazzarini added the bug Something isn't working label Jul 23, 2018
@agazzarini agazzarini added this to the 1.0 milestone Jul 23, 2018
@agazzarini agazzarini self-assigned this Jul 23, 2018
@agazzarini
Copy link
Member Author

Fixed. The patch provides also some useful logging messages during the evaluation process.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant