On-Device AI Proposal Scorer

Too much material to review by hand

At Edison, I worked on a proposal-review problem involving 11 contractor responses, roughly a thousand pages each, and 40 scoring criteria. The documents included tables and scans, and the actual proposal files needed to stay on the device.

Building the pipeline

I built the workflow in Python and ran Qwen3-VL 8B locally through llama.cpp. I used OCR and table preprocessing to make the inputs usable, split the documents into chunks, and retrieved the passages relevant to each rubric question.

I separated ingestion, retrieval, evaluation, and export so I could see which stage caused a bad result. Reranking helped bring the most relevant passages forward before scoring.

Making the output easier to trust

I redesigned the rubric into explicit questions and point logic. Preflight checks caught missing inputs, and each score linked back to the material behind it. The reviewer could inspect the evidence instead of accepting an answer without context.

The screenshots use demonstration data to show the workflow while keeping contractor documents private.

Result

Estimated review effort dropped from 126 hours to 8. In the project comparison, the system’s scores were within ±7% of the human review baseline.

What I learned

Give the reviewer a way to check the answer. A score is much more useful when the supporting passage is one click away.

Project context

The time reduction is an estimate from this workflow. The interface screenshots use demonstration data so the actual contractor proposals stay private.