Ingestion
PDF validation, bounded uploads, filenames, and local storage.
Loading
01 / Flagship system
Document processing / Engineering experience
Follow one real document through job handling, parsing, transformation, export, and packaging. Every result shown below comes from a local RelayWorks run.
01 / 06
The problem
Turning a PDF into useful artifacts required coordination across upload handling, long-running work, normalization, review, and packaging.
Document-heavy AI work repeatedly starts with the same glue code before any retrieval or model workflow can begin.
Observed constraint
A real five-page NIST sample spent 120.76 seconds in conversion. The interface could not treat that work like a quick form submission.
PDF validation, bounded uploads, filenames, and local storage.
A conversion can outlive a practical HTTP request and still needs honest status.
Parser output must become reviewable Markdown, HTML, JSON, text, and goal-oriented content.
Images, reports, and generated files need predictable names, downloads, and one complete package.
02 / 06
Follow a document
Run the representative sequence, then select any stage to inspect the work and the observable signal behind it.
NIST.AI.600-1-sample.pdf
5 pages · AI Knowledge Base goal
03 / 06
Architecture
Select a node to inspect its responsibility, why it exists, the nearby alternative, and the tradeoff the implementation accepts.
Node 01 / 07
04 / 06
Engineering decisions
These choices are visible in the current implementation, its history, and its documented limitations.
Why move conversion out of the request-response cycle?
The real five-page sample took 120.76 seconds in Marker. The current endpoint saves the upload, creates a job, returns HTTP 202, and lets the frontend poll honest lifecycle stages.
Accepted tradeoff
The local JSON store and thread worker are understandable, but they are not a durable distributed queue. A restart marks active work interrupted rather than resuming it.
Why keep Markdown, JSON, HTML, and text?
Markdown is readable and model-friendly, JSON preserves programmatic structure, HTML supports rendered review, and text provides a low-friction search or ingestion format. All four come from the same normalized document.
Accepted tradeoff
More artifacts increase testing and packaging work, and none of the formats can correct errors already present in the parser output.
Why keep documents in the buyer's environment?
The source PDF, job metadata, generated outputs, and packages stay under the operator's control. The current workflow requires no API keys and does not depend on hosted storage.
Accepted tradeoff
The operator owns Python, Node.js, Marker, filesystem permissions, capacity, upgrades, and cleanup. This is not a hosted multi-user service.
Why package the result instead of returning one file?
The same conversion can feed review, retrieval, display, search, or archival work. A fixed package layout makes the handoff predictable while individual download routes remain available.
Accepted tradeoff
The package adds another stored artifact and may include outputs a particular downstream workflow does not need.
05 / 06
Inspect outputs
Explore sanitized excerpts and extracted images from the real five-page sample package.
Authentic sample package
NIST.AI.600-1-sample.pdf
# Artificial Intelligence Risk Management Framework
## Generative Artificial Intelligence Profile
This publication is available free of charge from:
https://doi.org/10.6028/NIST.AI.600-1
July 2024These excerpts and metrics come from a real local RelayWorks package. Private paths and job identifiers are intentionally omitted.
06 / 06
Reflection
The useful lessons are in the gaps between a working local kit and a system ready for broader operating conditions.
What surprised me
The real sample completed successfully, but its extracted text still contains spelling artifacts and its table structure needs review. Honest output inspection matters as much as job success.
What held up
Markdown, HTML, JSON, text, goal output, and the report all come from the same parse. That keeps the artifacts related and avoids paying the parsing cost repeatedly.
What I would redesign
For multiple machines or resumable work, the JSON job store and local thread worker would need to become a durable queue plus database-backed state. The current version deliberately favors inspectability on one machine.
Operational lesson
Marker runs as a separately installed CLI. Python, Node.js, filesystem permissions, timeouts, and process recovery are not setup trivia—they shape whether the workflow is dependable.
Next improvements
The next useful work is testing more representative PDFs, improving OCR and complex-layout handling, defining retention cleanup, and making interrupted work resumable—not adding decorative features.
Walkthrough complete