Difference between revisions of "Resources"

Revision as of 16:24, 22 March 2010

Named Entity Recognition in Questions: Towards a Golden Collection

• A set of nearly 5,500 manually annotated questions to be used as training corpus in machine learning based NER systems. The named entities in these questions were identified and classified according to the categories: Person, Location and Organization. We extended and particularized the guidelines of the shared task of the Conference on Computational Natural Language Learning (CoNLL) 2003 on NER to face the demands presented by questions.

These corpora are freely available for research purposes. You can download the training corpus here, and the testing corpus here.

Further details on building this question corpora can be found in [2]. We kindly ask you to cite this publication whenever you use the resource.

Rule-based Question Classifier

Our rule-based question classifier couples two strategies to obtain its results:

• a direct (pattern) match is performed for specific questions. For instance, Who is Mozart? is directly mapped into Human:Description;
• headwords are identified (by a rule-based parser) and mapped into the question classification (by using WordNet). For example, in the question What is Australia's national flower? the headword flower is identified and mapped into the category Entity:Plant.

Here we make available the resources we used in the rule-based question classifier:

• The head-rules used in this work are a heavily modified version of those given in collins99, specifically tailored to extract headwords from questions:
  • Headword Extraction Rules
  • Headword Extraction Rules, specific to questions

• Question Patterns

• Question Tree Patterns