SemEval-2016 Task 9 (Chinese Semantic Dependency Parsing) Datasets

For more information: http://alt.qcri.org/semeval2016/task9/

Thanks to all participants!

Statistics

This task provides two distinguished corpora in appreciable quantity respectively in the domain of NEWS and TEXTBOOKS (from primary school textbooks).

Set	NEWS	TEXTBOOKS
train	8301	10754
validation	534	1535
test	1233	3073

Systems

Fifteen organizations were registered to participate in this task. Finally, five systems were received from three organizations. The main evaluation results are listed below, see the task description paper for detailed description and results.

Results on NEWS corpus:

System	LF	UF	NLF	NUF
IHS-RD-Belarus	59.06	77.64	40.84	60.20
OCLSP (lbpg)	57.22	74.93	45.57	58.03
OCLSP (lbpgs)	57.81	75.54	41.56	54.34
OCLSP (lbpg75)	57.78	75.40	48.89	58.28
OSU-CHGCG	55.69	73.72	49.23	60.71

Results on TEXTBOOKS corpus:

System	LF	UF	NLF	NUF
IHS-RD-Belarus	68.59	82.41	50.57	64.58
OCLSP (lbpg)	65.54	79.39	51.75	63.21
OCLSP (lbpgs)	66.21	79.85	47.79	55.51
OCLSP (lbpg75)	66.38	79.91	57.51	63.87
OSU-CHGCG	65.17	78.83	54.70	65.71

Reference

If you wish to use this data in your research, please cite:

@InProceedings{che-EtAl:2016:SemEval,
	author    = {Che, Wanxiang  and  Shao, Yanqiu  and  Liu, Ting  and  Ding, Yu},
	title     = {SemEval-2016 Task 9: Chinese Semantic Dependency Parsing},
	booktitle = {Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)},
	year      = {2016},
	publisher = {Association for Computational Linguistics},
	pages     = {1074-1080},
}

---

SemEval-2016 Task 9 Evaluation Committee