Career & Best Practices5 min read

Four Silent Failures in My Own Stack — A Friday Postmortem

S

Suneet Malhotra

Apr 24, 2026

1 views
Four Silent Failures in My Own Stack — A Friday Postmortem - Career & Best Practices blog post

At 06:45 Pacific on Tuesday, the cron job that was supposed to publish my daily blog post tried to run, immediately exited with the error "claude: command not found," and wrote a single line to a log file I wasn't watching. At 06:45 the next day, the exact same thing happened. The scheduled automation I had just spent two weeks building wasn't running, and I didn't notice until I sat down that evening and pulled up the post log to see two empty slots where there should have been entries.

That was the first of four silent failures this week in a content automation stack I thought I understood. None of them crashed loudly. None of them paged me. They all just stopped working, in the particular way that hobby-project infrastructure stops working: quietly, and in a place no one was watching.

Here is each one, what the logs actually said, and what I changed.

1. The cron that could not find its own interpreter

The failure mode was obvious in retrospect. I had installed the Claude CLI under a path that is on my interactive shell PATH because my zshrc puts it there. launchd does not source zshrc. When the scheduler spawned the routine, the process had a different PATH, and the invocation of the CLI failed before the routine ever read its first line of memory.

I had technically already fixed this two weeks ago by adding a CLAUDE_BIN environment variable to the plist and patching the routine to call the binary by absolute path. The fix worked. It stopped working when a system update rebuilt the CLI at a new path and quietly invalidated the hardcoded CLAUDE_BIN I had set.

The right fix is what I did after I caught it: resolve the absolute path of the interpreter at routine startup, cache it, and fail loudly if it is not executable. Belt and suspenders. If the scheduler cannot find its interpreter, I want a Telegram push that says so, not a silent non-zero exit code written to a log nobody tails.

2. The broadcast fan-out with a partial publish

The content pipeline fans a single draft out to four channels. On Tuesday's midday run, two of them published cleanly. A third returned an empty error string. A fourth threw an ImportError for a function I had refactored locally but never pushed to the deploy path. The draft had been half-published and the pipeline reported success.

Two separate bugs, stacked. The empty error was a rate-limit response with a stripped body; my code caught the exception, logged the empty string, and moved on. The ImportError only surfaced at runtime because the import chain was lazy and the deploy tree was stale.

Both are fixable with boring diligence. The more interesting thing is the shape of the failure: the pipeline returned success because two branches succeeded, and there was no fan-out-level check for "did all four channels actually receive the write." I have added one. The contract is now: if any branch fails, the whole run is a partial failure, and Telegram gets a diff of which channels hit and which didn't.

3. The fourteen-day audience check that told me nothing was landing

Late that evening I pulled the fourteen-day engagement metrics from one of those channels. Twenty-four posts. One total like. The math is ugly: 0.04 likes per post, well below the rate at which you can attribute anything to content quality. It is not even signal.

There are two honest readings. The first is that my audience on that channel is effectively zero and cross-posting to it is costing me nothing but also buying me nothing. The second is that I have not done the active-engagement work, replying and following and joining threads, that any social channel needs in its first year. Both can be true.

The engineering response to a metric like that is to either invest in the channel properly or deprioritize it. Posting into a void is zero ROI. I have not decided yet. But I have stopped pretending the fan-out is "distribution" until the channels actually distribute.

4. The routine that wakes up every day just to fail

The fourth failure is the cleanest. I have a routine that searches for high-signal accounts to reply to on one particular platform. Every step of it uses that platform's API. That API has been disabled on my account for weeks because I am not on the paid tier. The routine has been running on schedule, failing fast, and logging a skip line, every day.

It was not hurting anything. It was also not doing anything. The correct answer is to either rewrite it to target the platforms where I actually have write access, or disable it in the scheduler until the original path comes back. A routine that wakes up, fails, and goes back to sleep is a schedule slot that is not being used for something valuable. This week I disabled it.

The through line

Four failures, one pattern. Every one of them was something ordinary observability would have caught in a production system: PATH hygiene in scheduled jobs, end-to-end success checks on fan-out writes, an engagement-rate threshold below which a channel is not worth feeding, and an audit of which scheduled jobs are actually producing output. The reason they all slipped was that the system was mine, hobby-scale, and I had not given it the monitoring I would have demanded from anything more serious.

The lesson I am writing down for next week: hobby projects deserve observability budgets too. A personal automation that runs every day is a system. If it does not tell you when it breaks, it is already broken, you just haven't noticed yet.

Share this post

You Might Also Like

Stay in the Loop

Get weekly insights on AI-driven QA, engineering leadership, and automation strategies.

No spam, ever. Unsubscribe anytime.