Automated Question Generation and Answer Verification using Visual Data

Link to full Journal Paper

This paper delineates the automation of question generation as an extension to existing Visual Question Answering (VQA) systems. Through our research, we have been able to build a system that can generate questions and answer pairs on images. It consists of two separate modules - Visual Question Generation (VQG) which generates questions based on the image, and a Visual Question Answering (VQA) module that produces a befitting answer that the VQG module generates. Through our approach, we not only generate questions but evaluate the questions generated by using a question answering system. Moreover, with our methodology, we can generate question-answer pairs as well as improve the performance of VQA models. It eliminates the need for human intervention in dataset annotation and also finds applications in the field of the educational sector, where the requirement of human input for textual questions has been essential till now. Using our system, we aim to provide an interactive interface which helps the learning process among young children.

Received a grant from the University of Mumbai

• natural language processing
• computer vision
• visual question answering
• visual question generation
• recurrent neural network
• vqa dataset
• image feature extraction
• long short term memory
• gated recurrent unit
• convolution neural network
• attention module
• e-learning system