Scheduler & Watchdog: Ops Automation | Toss Auto-Trading Dev Journal #18

By the time Phase 5’s alerting and monitoring had been validated in real use, only one thing was left on the roadmap: Phase 6, operational automation.

A quick note on context: Toss Securities

For readers outside Korea — Toss Securities is a Korean brokerage that exposes a REST-based Open API, similar in spirit to Alpaca or Interactive Brokers’ API but scoped to the Korean market and KRW-denominated trading. This series documents building a personal auto-trading bot on top of it.

The problem: automation actually meant two different things

“Operational automation” turned out to split into two separate problems. One was running a data-collection script on a recurring schedule. The other was keeping always-on scripts like auto_trader.py and bot_server.py alive — starting them on boot and restarting them if they crash. The first is solvable purely in Python; the second is really an OS-level (Windows) concern, so the two needed different tools entirely.

Part 1: Scheduled data collection with APScheduler

I built scheduler.py using APScheduler’s BlockingScheduler to run price-snapshot collection every 30 minutes (configurable). Along the way I realized toss_portfolio.py‘s existing display_portfolio() function was built for a human reading a console table — not for unattended execution — so I split out a separate collect_price_snapshots() function that writes to the database without any console output.

The scheduler is defensive: if authentication fails or an unexpected exception hits, it doesn’t die — it just retries on the next cycle. I verified this by running the scheduler at a 1-second interval briefly and confirming it fired exactly on schedule.

Part 2: Auto-restart with a watchdog

watchdog.py restarts always-on scripts if they die from accumulated network errors or unexpected bugs. The rule I cared most about: a clean exit (code 0) should never trigger a restart. If auto_trader.py finishes buying every target symbol and exits on its own, force-restarting it would be actively dangerous.

I tested this against a real subprocess — a dummy script rigged to fail — to verify restart counts, a max-restart cap, and that clean exits are correctly left alone. Even after a restart, config.py is re-read from scratch each time, so the triple safety-guard values (DRY_RUN, target price, etc.) always come back unchanged.

Docker: deliberately skipped

Given the current setup — a personal Windows PC with a Google Drive–synced folder — containerizing with Docker would add complexity without a clear payoff right now. It’s noted in the roadmap for reconsideration if this ever moves to a cloud server.

Documentation: the deployment/ops checklist

I consolidated scattered operational knowledge into one file, 배포_운영_체크리스트.md: a table of how to run each script, watchdog usage, the steps to register auto-start on Windows Task Scheduler, a mandatory safety-guard test pass before enabling automation, and how to monitor things via the web dashboard, Telegram bot, and log files.

Phase 6 adds scheduled data collection via APScheduler and process auto-restart via a watchdog script, with a strict rule that clean exits are never restarted — verified with real tests confirming the triple safety guards stay intact across restarts.

Results

The full pytest suite passed all 118 tests (109 existing + 9 new), with no changes to any safety-guard values. Both the local environment and the project repo are in sync. That closes out Phase 0 through Phase 6 on the roadmap.

FAQ

Q1. APScheduler vs. Windows Task Scheduler — which one should I use?

APScheduler is convenient when you want to manage recurring logic inside your Python code. Windows Task Scheduler is needed when you need to control execution itself — like auto-starting on boot. Phase 6 uses both, each for what it’s good at.

Q2. What stops the watchdog from restarting forever?

A hard cap on restart count. Once it’s hit, the watchdog stops restarting and just sends an alert instead — unbounded restarts just burn resources while hiding the real problem.

Q3. Why skip Docker in this phase?

On a personal Windows PC with a Google Drive–synced folder, Docker adds complexity without enough upside right now. It’s on the list to revisit if the setup ever moves to a cloud server.

Q4. What’s next?

Only Phase 7 (live-trading readiness) remains, and it’s deliberately gated behind explicit user approval before any progress is made.