.env files.PromptVault treats prompts as production code — version history, character-level diffs, structural checks before you ship, and alerts when OpenAI model metadata changes.
Built for solo developers running 2–3 production prompts. No signup required to explore the workflow.
Customer Support Reply
gpt-4o-mini · v3
Code Review Assistant
gpt-4o-mini · v2
Release Notes Generator
gpt-4o-mini · v2
The problem
Most teams patch prompts in docs and config files. When something breaks, the post-mortem starts with: “what changed?” PromptVault connects all three fixes in one workflow.
You edited a prompt last Tuesday. Something broke on Friday. You have no idea what changed.
Teams iterate on prompts constantly. Without version control, every change is destructive. No diff, no rollback, no audit trail. When behavior regresses, the post-mortem is guesswork.
A provider deprecates or updates model metadata. You only find out when production breaks.
Providers change model lists, version metadata, and deprecations without a reliable changelog for every alias. PromptVault polls provider metadata daily, re-runs your structural test suite, and alerts you when a watched model changes or disappears.
Something broke in production. Nobody knows who owns the prompt, who changed it last, or why.
Prompts live in Notion pages, spreadsheets, and Slack threads. Nobody is accountable. No approval trail. When something goes wrong, the blame radius is the entire team.
How it works
Most tools bolt features together. PromptVault is built around a single causal chain: every node feeds the next. Remove any one and the chain breaks.
01
Save a new version
Every change is tracked. Diff shows exactly what moved between versions.
02
Structural checks run automatically
Your prompt-contract tests fire on every save. Text regressions surface immediately — without calling a model.
03
Provider metadata changes are detected
Daily polling catches model list updates and deprecations for watched models.
04
Owner is notified
If structural pass rate drops after a metadata change — or a model is deprecated — you get an email with the score diff and a link to results.
Feature 01 — Prompt Vault
Every save creates a new immutable version. Nothing is overwritten. Compare any two versions side-by-side with character-level diffs — exactly like a code review.
Variables are detected automatically. Write {{user_name}} or {{context}} and PromptVault highlights them, lets you set test values for each, and tracks when they change between versions.
Every version carries metadata: the model it was written for, the author, a commit message, and an optional label.
{{variable}} detection and labellingVersions (4 / 20)
Comparing v2 → v3
v2 — before
You are a customer support agent for Acme Corp. Reply professionally to:
v3 — after
You are a senior customer support agent for Acme Corp. Reply professionally and empathetically to:
Test cases (3 / 10)
v4 scored lower than v3 (87%). Review the diff before promoting to production.
Feature 02 — Test Suite
Attach a test suite to any prompt. Each test case defines input variables and an expected string. Every time you save a new version, the suite runs automatically.
Contains and Exact match validate the rendered prompt text — zero API calls. LLM-as-judge runs manual output evaluation with your own key via a server action. PromptVault does not store your key.
If v4 passes 6/10 structural checks and v3 passed 9/10, that regression is flagged visually before you promote the version.
contains — pass if rendered prompt includes expected stringexact_match — pass if rendered prompt matches exactlyllm_judge — manual output grading with a custom rubric (requires API key at run time)Feature 03 — Model Watch
Every LLM provider updates model lists and deprecates aliases. Sometimes the changelog reaches you. Often it does not.
Enable Model Watch on any prompt, set a structural pass-rate threshold (default 80%), and go back to work. Every day, PromptVault polls provider model metadata endpoints and compares the current snapshot against the last known state.
When metadata changes or a model is deprecated, your structural test suite runs automatically. If pass rate drops below threshold — or the model disappears — you get an email with what changed.
Model Watch monitors metadata and availability — not silent behavioural changes behind an unchanged alias.
Last checked 2h ago
Recent events
Subject: [PromptVault] Model change detected — your prompt may be affected
A change was detected in gpt-4o (OpenAI).
Your prompt “Customer Support Reply” structural pass rate dropped from 90% → 62% (threshold: 80%).
Pricing
Model Watch is free. Test suites are free. Version control is free. The upgrade is about scale, not access.
$0 / month
Team
No credit card required. No API key needed to start. You only need an API key for manual llm_judge runs — transmitted to our server action for that request and not stored.
FAQ
No. Sign up with your email, create a prompt, and start versioning. Version control and the contains/exact match evaluators work immediately — no API keys, no configuration.
When you run llm_judge manually, you paste an OpenAI API key into the Run tests form. The key is sent to a Next.js server action for that request only, used to call OpenAI, and is not stored in the database or written to logs.
Model Watch polls provider model metadata endpoints — free APIs that return available models and version fingerprints. No tokens consumed. If the fingerprint changes or a model is deprecated, we trigger your structural test suite. Contains and exact match tests run automatically. LLM-as-judge tests are skipped in automated runs because they require a user-supplied key.
Once per day on the free tier. Hourly polling is available with a Vercel Pro deployment or by using an external cron service.
Nothing bad. The poll is skipped and retried the next day. You will never receive a false alert because a provider was temporarily unreachable.
Saving a version automatically triggers your structural test suite. A provider metadata change automatically re-runs that suite. A threshold breach or deprecation automatically emails the prompt owner. Remove any link and the chain breaks.
Because it only works if you have test cases, and test cases only matter if you have versions. The alert email a user forwards to their team is the acquisition loop. Gating it would kill that loop.
Free for solo developers. Version, check, and monitor the prompts your app depends on.
Already running prompts in production? You likely have prompt drift right now.