Private Medical Document Vault

Outcomes

Context

Clinical paperwork is sensitive by default — architecture must assume on-prem or self-hosted inference.

Challenge

Families accumulate lab results, referrals, and scans across formats and languages. Cloud OCR and public LLM APIs were unacceptable for medical documents. The product needed upload, extraction, structured conversion, and summaries while keeping all processing on controlled infrastructure.

Approach

We built a multi-service stack: React frontend, FastAPI application API, dedicated OCR service with PaddleOCR and vision-model correction, Ollama for local LLM summaries and timelines, and PostgreSQL for family/member/document models. GPU Docker Compose made OCR and LLM practical on owned hardware.

Architecture

React + TypeScript UI, FastAPI backends, PaddleOCR + vision models for layout-aware OCR, Ollama LLM runtime, PostgreSQL, Docker Compose with NVIDIA GPU support.

Scope

• —Family accounts
• —Document upload
• —Local OCR
• —Markdown structuring
• —LLM summaries and timelines
• —Multilingual document support

Constraints

• —No cloud OCR/LLM dependency
• —GPU required for practical throughput
• —Romanian-first language needs
• —Mixed PDF and image inputs

Solution highlights

• —OCR as a separate service from the product API
• —Vision models for layout understanding, not only raw text
• —Ollama as the private analysis runtime
• —Compose + NVIDIA toolkit for reproducible GPU stacks

Technologies

Results

• —End-to-end private path from scan upload to searchable structured text
• —OCR pipeline with bounding-box preservation and vision-assisted correction
• —Local LLM summaries and timelines without public API calls
• —Family/member account model suited to household medical history

Lessons learned

• —Medical AI products are infrastructure products first.
• —Separate OCR from chat summarization or both become unmaintainable.
• —Language coverage belongs in the OCR and LLM selection, not only the UI.