The Step in My Routine That Is Not Safe to Run Twice
Suneet Malhotra
May 20, 2026
A cron entry on my machine fires this blog routine at 06:00 PT every weekday. It pulls a repository, drafts an essay, writes a new object into a TypeScript file, commits, and pushes. By the time I am awake the post is live.
The thing about a cron entry is that it does not retry. If the 06:00 run fails, cron does not try again at 06:01. The post simply does not exist. I find out, and I run the routine by hand. That manual rerun is the interesting case, because I run it blind. I do not know how far the failed run got before it died. I am assuming it did nothing. Sometimes it did something.
The question that matters for any routine on a schedule is not what happens when it fails. It is what happens when it half-succeeds and then gets run again.
The failure that is actually dangerous
Total failure is benign. On April 22 the cron fired, the claude binary was not on the cron environment's PATH, and the whole run died before it produced anything. That is a clean failure. Nothing landed. I fixed the PATH, reran it, no harm done.
The dangerous failure is partial. This routine is a dozen numbered steps, and four of them have side effects: it pulls a repo, it writes a file, it commits and pushes, and it appends a line to a separate log. Suppose a run gets through the file write and the commit, and then the push fails because the network blipped. Four side effects, two landed, two did not. Now I rerun it blind. If the rerun does not know the file was already written, it writes the new entry again. The array now holds two objects with the same id. The site router renders whichever it reaches first. The other is a silent duplicate that nothing flags.
That is the bug. It is not caused by the failure. It is caused by the rerun on top of the failure.
The three properties I check
Before I trust a routine on a cron, I walk every step that changes state and ask three questions in order. The first yes wins.
Is it idempotent by construction? Running it twice produces the same result as running it once, with no special handling. The repo pull is the example. The routine does not do a polite git pull. It does a hard reset onto origin/main. That discards whatever local state exists, from this morning or from a run three days ago, and re-derives the working tree from origin. Run it once, run it ten times, the tree is identical afterward. Idempotency by demolition. It is the cheapest trick available, and I use it anywhere the local state is disposable.
If not, is it guarded by a pre-check? The step looks for evidence of its own prior output before it acts. The file write is this case. Before the routine prepends a new blog object, it checks whether an object with today's slug already exists in the array. If one does, the routine stops. It does not overwrite, it does not append a second copy, it raises the duplicate and exits. The guard is the thing that makes a blind rerun safe: the second run sees the first run's output and refuses to double it.
If not, is it at least atomic? It either lands completely or not at all, with no partial state in between. The commit and push are atomic for free, because git made them atomic. A commit is one object. Origin either has it or does not. There is no such thing as half a commit on the remote. This is also why the order of the side effects matters. The file write is guarded, and the commit that publishes it is atomic, so the worst realistic case is a clean retry, not a corrupted array.
A step that is none of the three is a rerun bug with a date on it. You do not know when the date is. You know there is one.
The step that fails all three
The fourth side effect appends a line to a log file in a different repository. It records what was published and when. It is not idempotent: append twice, two lines. It is not guarded: nothing checks whether today's line is already there. It is not atomic in any sense that protects me, because a second append just quietly succeeds.
I have known about this for a while. The reason it has not bitten me is positional. The log append is the last side effect in the routine, so for it to run twice the entire routine has to run twice, and a full second run gets caught earlier by the duplicate-slug guard on the file write. The guard upstream covers the unguarded step downstream. That is not a property of the log step. It is luck wearing the costume of a design.
The fix is one line. The log entry already contains the slug. Before appending, grep the log for that slug and skip if it is present. That converts the step from unguarded to guarded, and then it stops depending on the step above it to stay safe. It is on next week's list.
The point that generalizes
If you put an agent on a schedule, you are not designing for the case where it runs. You are designing for the case where it runs, fails halfway, and gets run again by someone who assumes it did nothing. That someone is usually you, and you rerun blind because the whole appeal of a scheduled job is not having to hold its internal state in your head.
So for every step that changes state, get to a yes. Idempotent, or guarded, or atomic. Write down which one, step by step. The act of writing it down is the audit. The first step you cannot answer for is the line that will duplicate something in production, and you will have found it before it did.
Share this post
You Might Also Like
Test AI Latency from First Token to Finished Answer
A fast first token can hide a late or invalid answer. Test queueing, streaming, and completion with a synthetic workload before accepting a model-serving change.
AI & AutomationTest the Trace Before You Trust the Dashboard
A correct AI answer can hide a broken trace. Use a synthetic request and a deliberately stripped context header to test whether your debugging evidence survives service boundaries.
Career & Best PracticesThe Cheapest Test Is the One You Run Before You Build
The most expensive engineering mistakes in my systems were not hard bugs. They were reasonable implementations of questions nobody had defined.
Quantitative TradingA Signal Is Not a Decision
A signal says that conditions are interesting. It does not say that the portfolio should trade. The missing layer is the decision record between detection and execution.
Latest Blog Posts
Test AI Latency from First Token to Finished Answer
A fast first token can hide a late or invalid answer. Test queueing, streaming, and completion with a synthetic workload before accepting a model-serving change.
Test the Trace Before You Trust the Dashboard
A correct AI answer can hide a broken trace. Use a synthetic request and a deliberately stripped context header to test whether your debugging evidence survives service boundaries.
The Evaluator Needs an Abstain State
A judge that must choose pass or fail will turn uncertainty into false confidence. Add an abstain state, calibrate it, and test the cases where the evaluator should refuse to decide.
Related Tools & Demos
The QA Field Manual to Language Models
A free 24-chapter book. Start at “what is AI, really?” and finish with a small language model you built yourself — one that reads a failing Playwright test and proposes a fix you can run. Read it in your browser, or download the PDF or the Mac app.
View Source Code →Multi-Model LLM Harness
One interface to call any AI model — capability routing, fallback chains, budgets, circuit breakers, and a quality feedback loop. A practical architecture pattern write-up.
Automated Trading System
Multi-engine trading platform with real-time risk management, regime-based strategy selection, and automated order execution.
View Source Code →
Stay in the Loop
Get weekly insights on AI-driven QA, engineering leadership, and automation strategies.
No spam, ever. Unsubscribe anytime.