Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Incorporate facilities for computation of inter-annotator agreement and other evaluation #112

Open
proycon opened this issue May 19, 2017 · 3 comments

Comments

@proycon
Copy link
Owner

proycon commented May 19, 2017

We need user-friendly tools for researchers to compute inter-annotator agreement of arbitrary FoLiA annotations.

Computation of IAA is an important evaluation metric evaluation tools are currently not provided by FLAT at all; having such a facility as an interface for end-users in FLAT makes evaluation more accessible and prevent researchers from having to write their own evaluation scripts.

(proposed for CLARIAH-PLUS)

@proycon
Copy link
Owner Author

proycon commented Nov 22, 2017

Depends on proycon/folia#39

proycon added a commit to proycon/foliatools that referenced this issue Sep 6, 2018
@OsmanMutlu
Copy link

Hi proycon,

You can check this script that i wrote for our own annotations. It is not generic enough though.

@proycon
Copy link
Owner Author

proycon commented Apr 17, 2019

@OsmanMutlu Thanks! I'll take a look! I do already have a fairly generic foliaeval tool though, this issue is more about integrating it into FLAT and providing an interface.

# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

No branches or pull requests

2 participants