probnmn.utils.metrics

class probnmn.utils.metrics.SemanticQuestionReconstructionAccuracy(vocabulary: allennlp.data.vocabulary.Vocabulary)[source]

Bases: allennlp.training.metrics.sequence_accuracy.SequenceAccuracy

A metric which computes question reconstruction accuracy in a “semantic” sense, this considers certain synonym words equivalent, such as ("object", "thing").

The mapping of synonyms is taken from CLEVR dataset generation code.

Parameters
vocabulary: allennlp.data.vocabulary.Vocabulary

AllenNLP’s vocabulary. This vocabulary has three namespaces - “questions”, “programs” and “answers”, which contain respective token to integer mappings.