Experience
Velric
An AI hiring platform that generates, sabotages, and grades code at scale.

Velric is an AI-driven technical-hiring platform, it generates coding missions, sabotages them per candidate, and grades the results at scale. I came in as one of two lead engineers and owned the systems that keep it honest and fast: the generation pipeline, the sandboxed execution and grading engine, and the mission economy underneath it.
Stack
The systems I built
05 distinct products under one engagement, built end to end.
Mission generation & admission control
The pipeline that turns survey and résumé data into personalized missions, with a Postgres-table job queue and two-tier backpressure so six web instances never double-fire.
Postgres-as-queue admission control, the generation_jobs table IS the queue; jobs are claimed with an atomic optimistic lock so any web instance or the dispatcher can fire safely, capped by a global concurrency limit.
Two-tier backpressure with load-shedding, a fair dispatcher plus a hard VM-side semaphore, a max-queue-depth that sheds load with 503s, and a stale-slot sweeper that reclaims jobs stuck past a 10-minute timeout.
Opportunistic + autonomous drain, dispatches inline on new requests, and a 10-second cron sweep drains the backlog FIFO when traffic goes quiet.
Background generation UX, kicks generation off on dashboard mount to hide a 30s+ blocking wait that previously caused production 504s.
Warm mission pool & sabotage engine
A gold-blueprint economy: curated blueprints are AI-mutated into personalized, deliberately-bugged debugging missions, served instantly from a warm pre-generated pool.
AI code-sabotage engine, GPT-4o injects targeted bugs into blueprint code and personalizes the story to the candidate, running personalization and sabotage concurrently.
Closed-loop sabotage validation, executes the sabotaged code against the blueprint's tests to confirm the bug actually breaks the suite, retrying and cross-checking the produced error before it's ever served.
Warm mission pool, matches pre-generated missions by type, difficulty and category with Postgres array-overlap, clones one per candidate for an instant serve, and retires a mission after a set number of serves.
Difficulty-adaptive tuning, adjusts time limits and hint verbosity from the gap between blueprint difficulty and the candidate's stated experience.
Sandboxed execution & grading engine
A server-side engine that compiles and runs candidate submissions across JS/TS, Python and SQL in isolated workspaces, then grades them by real test execution plus an AI reviewer.
pgTAP SQL grading in throwaway databases, spins a uniquely-named temp Postgres DB per run, installs pgTAP, runs the candidate's setup and test SQL, parses the TAP output, and drops the database in a finally cleanup.
Multi-language execution, JS/TS on server-side Jest and Python on pytest, each in its own per-run temp workspace, normalized into one result shape with per-step timeouts and output caps.
AI code-review grader on Claude, scores submissions across code quality, efficiency, security and best-practices, returning a letter grade, strengths, improvements and a feedback paragraph.
Honest, process-level isolation, ephemeral databases, temp workspaces, OS process boundaries and hard timeouts, torn down after every run.
Async AI worker fleet
A fleet of BullMQ workers on Redis, run under PM2 across Azure VMs, offloading every heavy AI task from the request path.
Seven-queue BullMQ topology, matching, assessment, grading, tech-grading, candidate-summary, job-description and doc-verify, each with dedicated worker processes.
Resilient job defaults, three attempts with exponential backoff, completed jobs retained a day and failed jobs a week for debugging, all Sentry-instrumented.
PM2 cluster orchestration, runs the web app at six cluster instances (deliberately below the box's core count for headroom) plus dedicated processes per AI worker and the generation dispatcher.
Self-hosted ops, a two-VM split (prod app and workers vs the agentic FastAPI service), both managed through PM2.
Anti-fraud: DocVerify & rubric grading
An AI trust layer: vision-based transcript verification with tamper and duplicate detection, a validated résumé-parsing pipeline, and a rubric grader that probes for ghost-written work.
Vision-based DocVerify, extracts name, institution and GPA from PDFs and images with GPT-4o vision, runs an authenticity check for tampering, and does soft fuzzy name-matching plus duplicate-profile detection.
Résumé-parsing pipeline, PDF/DOCX extraction → GPT-4o structuring → schema validation with retries → sanitized JSON safe for Postgres jsonb.
Anti-bluffing rubric grader, GPT-4o grades submissions across five weighted dimensions, then generates verification questions from the submission and re-grades the answers to adjust the tier.
Behavioral anti-cheat, folds tab-switch counts, attempts and time into the final score with explicit penalty deductions.
Next
Sendcoins