A question-generation system that turns any source material into a question bank

Client Industry

Education technology · learning platform

Key Challenge

Manual question writing from mixed-format source material

Core Tech Stack

NLP · Transformers · OCR · ASR · Python

Deployment Model

Multimodal ingestion pipeline behind a web tool

SoftmaxAI developed an automated question-generation system capable of turning any kind of source material into a structured and assessable set of questions. Text, images, audio, and video are converted into clear passages using OCR and speech-to-text; afterwards, a language processing pipeline detects the testable statements in each passage and produces multiple-choice, true/false, fill-in-the-blank, and short-answer questions based on them. The system uses rule-based linguistic analysis to determine what kinds of questions can be generated, thus ensuring that each question is based on a particular sentence from the original source rather than on what the model happens to know.

• Assessment authoring did not scale. For each new passage the teacher had to write out the questions, the answers, and the plausible incorrect options — a procedure which took up preparation time but did not enhance the material.
• Source material arrived in every format. The course material was just as often found in scanned handouts, slide photographs, recorded lectures, and video as it was in editable text, meaning that a text-only tool would have only covered a small part of it.
• One passage needs several question types. Recall, comprehension, and recognition are tested in different ways; a system which produced only multiple choice questions would have dealt with one aspect of the job and left the other aspects untouched.
• Plausible wrong answers are the hard part. Multiple choice is only useful when the distractors are tempting but genuinely incorrect — clearly wrong options are of no educational value, and those that happen to be correct undermine confidence in the tool.

The system operates in a left-to-right pipeline format; the mixed-format source material is converted into clean text, analyzed to extract testable propositions, expanded into questions and distractors using the generation models, verified against the source, and then presented as a shareable question bank.

• Spec before code. Before carrying out the development, we worked with the client to define the inputs that were supported, the formats of the questions, and the acceptance criteria for each of them — including specifying what the system must refuse to generate rather than attempt to guess.
• Normalize every input to one representation. The OCR process is used for photographed and scanned pages, speech-to-text is used for audio and video, and all these methods end up using the same cleaned passage structure — that is why the generation logic is written only once rather than four times.
• Linguistics constrains the model. By using rule-based parsing, the entities, relations, and factual propositions actually contained in a passage are identified and questions are then generated based on these. It is because of this that the output remains tied to the source rather than to the model's prior knowledge.
• Human-owned quality bar. The reviewers checked the generated sets against the actual course material and made adjustments to both the distractor generation and the answer-grounding checks until the output could be used by a teacher without the need to examine each question.

• Question banks from any source. Whether it is a handout that has been photographed, a lecture that has been recorded, or a paragraph that has been pasted, the result is always the same kind of structured output, which means that existing course material can be used without having to retype it.
• Four question formats from one passage. For a single upload, multiple choice questions, true or false questions, fill-in-the-blank questions, and short answer questions are all produced together, covering both recall and comprehension.
• Self-assessment without a teacher in the loop. The learner uploads some material, selects the concepts they want to test, and then goes through a quiz that is generated — thus turning what would have been passive reading into active recall, all according to their own schedule.
• Every answer traceable to the source. Since questions include a reference back to the sentence which provides the answer, a disputed point can be resolved by referring to the passage rather than by relying on the generator.
• Reusable, shareable output. Instead of regenerating them, generated sets are saved and then shared, which causes the question bank to build up as a durable asset throughout the course.