Skip to content

Latest commit

 

History

History
79 lines (68 loc) · 2.99 KB

README.md

File metadata and controls

79 lines (68 loc) · 2.99 KB

DynSP (Dynamic Neural Semantic Parser)

This project contains the source code of the Dynamic Neural Semantic Parser (DynSP), based on DyNet.

Detail of DynSP can be found in the following ACL-2017 paper:

Mohit Iyyer, Wen-tau Yih, Ming-Wei Chang. Search-based Neural Structured Learning for Sequential Question Answering. ACL-2017.

@InProceedings{iyyer-yih-chang:2017:Long,
  author    = {Iyyer, Mohit  and  Yih, Wen-tau  and  Chang, Ming-Wei},
  title     = {Search-based Neural Structured Learning for Sequential Question Answering},
  booktitle = {Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  month     = {July},
  year      = {2017},
  address   = {Vancouver, Canada},
  publisher = {Association for Computational Linguistics},
  pages     = {1821--1831},
}

The output files and the models for producing the reported results are also included. Below are the scripts that produces the results reported in Table 2 of the paper (DynSP and DynSP*).

$ ./check.sh moduleKwNoMap-b15-ind
moduleKwNoMap-b15-ind
Best Accuracy: 0.425963 (Reward: 0.479099) at epoch 28
Best Accuracy: 0.369095 (Reward: 0.423323) at epoch 10
Best Accuracy: 0.348668 (Reward: 0.405460) at epoch 24
Best Accuracy: 0.377477 (Reward: 0.439594) at epoch 29
Best Accuracy: 0.349951 (Reward: 0.413719) at epoch 20
Best Accuracy: 0.351802 (Reward: 0.409400) at epoch 19
0.359399 20
$ ./evalModel-indep.sh moduleKwNoMap-b15-ind 20
Sequence Accuracy = 10.15% (104/1025)
Answer Accuracy =   41.97% (1264/3012)
Break-down:
Position 0 Accuracy = 70.93% (727/1025)
Position 1 Accuracy = 35.84% (367/1024)
Position 2 Accuracy = 20.06% (137/683)
Position 3 Accuracy = 12.23% (28/229)
Position 4 Accuracy = 13.16% (5/38)
Position 5 Accuracy = 0.00% (0/9)
Position 6 Accuracy = 0.00% (0/4)
$ ./check.sh moduleKwNoMap-b15
moduleKwNoMap-b15
Best Accuracy: 0.450863 (Reward: 0.516281) at epoch 17
Best Accuracy: 0.379691 (Reward: 0.439837) at epoch 12
Best Accuracy: 0.366021 (Reward: 0.422335) at epoch 16
Best Accuracy: 0.391892 (Reward: 0.456894) at epoch 26
Best Accuracy: 0.370968 (Reward: 0.442918) at epoch 20
Best Accuracy: 0.368468 (Reward: 0.431721) at epoch 18
0.375408 18
$ ./evalModel.sh moduleKwNoMap-b15 18
Sequence Accuracy = 12.78% (131/1025)
Answer Accuracy =   44.65% (1345/3012)
Break-down:
Position 0 Accuracy = 70.44% (722/1025)
Position 1 Accuracy = 41.11% (421/1024)
Position 2 Accuracy = 23.57% (161/683)
Position 3 Accuracy = 13.97% (32/229)
Position 4 Accuracy = 18.42% (7/38)
Position 5 Accuracy = 11.11% (1/9)
Position 6 Accuracy = 25.00% (1/4)

The Sequential Question Answering (SQA) dataset, published and used in the same paper, can be downloaded separately.