One central lesson keeps reappearing while building production AI workflows: if a task is repeatable and the operational steps are known, never ask an AI agent to execute the entire routine through natural language on every run. Use the model to write the deterministic code once, and then run that code repeatedly.
This architectural shift saves money, runs substantially faster, is easier to test, and produces predictable outcomes. The goal is never to avoid AI. The goal is to deploy artificial intelligence exclusively where genuine judgment is required.
Code Handles Predictable Mechanics, Models Supply Judgment
Consider a standard automation task: generating a daily digest from multiple Slack channels. The operational workflow is completely straightforward:
- Fetch channel messages from the preceding 24 hours.
- Identify threads that received replies.
- Retrieve all messages within each active thread.
- Filter out automated bot notices and duplicate messages.
- Summarize each meaningful conversation thread.
- Publish the formatted digest to an executive channel.
An engineer could prompt a frontier model with an open-ended directive: “Go through these Slack channels, collect all messages and threads from the last 24 hours, summarize them, and post the digest here.” While the agent may complete the task, using an expensive LLM to calculate timestamps, manage API pagination, loop across threads, and strip duplicates is wasteful.
These operations are basic software engineering problems. A disciplined developer uses an AI model to generate a Python service that interacts directly with Slack APIs for data retrieval and cleaning. The system then passes only the sanitized conversation text to a smaller model for summarization. The frontier model builds the workflow once rather than executing basic plumbing every single day.
Why the Code Boundary Matters for Speed, Cost, and Testing
Relying on agents to orchestrate standard plumbing introduces compounding penalties across three operational dimensions:
- Token cost: Fully agentic pipelines force models to ingest tool definitions, API schemas, pagination payloads, raw JSON responses, and intermediate routing instructions on every loop. In production, the overwhelming majority of consumed tokens manage process mechanics rather than evaluating semantic meaning.
- Latency: An agent must receive an API response, deliberate over the payload, pick the next tool, and trigger another call sequentially. Compiled code fetches multiple channels and threads concurrently in milliseconds without waiting on an inference round-trip.
- Predictability: Natural-language prompts create edge-case ambiguity. Questions about bot filtering, duplicate handling, and run limits are solved cleanly in code using explicit start times, bot user IDs, and idempotent database checks.
This boundary transforms verification. When an entire process relies on conversational prompts, teams are forced to build fragile LLM evaluations to verify whether the agent obeyed basic instructions like pagination or date filtering. Replacing those steps with code allows deterministic unit tests to validate timestamp bounds, pagination loops, and deduplication logic instantly. Unit tests cost nothing, run in milliseconds, and give teams total confidence in pipeline mechanics, leaving expensive model evaluations focused entirely on summarization quality.
Enforcing the Boundary: Where AI Belongs in the Pipeline
Software should do software work. When operational steps, filtering logic, and data schemas are known in advance, write deterministic code to handle them.
Reserve language models for the narrow stages where semantic comprehension, synthesis, classification, or human judgment is indispensable:
- Filtering and ingestion: Code fetches the payload, normalizes fields, and discards noise.
- Intelligence: The model ingests the clean excerpt and generates the summary.
- Routing: Routine requests follow standard code branches or small, focused models. Frontier reasoning models are reserved strictly for rare, complex exceptions.
Without this boundary, teams inadvertently turn their most expensive frontier models into brittle, slow HTTP clients and loop counters.
Hard Budgets Force Disciplined Architecture
Engineering organizations must establish strict monthly operating budgets for AI pipelines. When a team operates under a firm $2,000 monthly ceiling, developers naturally ask the questions that produce sound software architecture:
- Can this operational step run as compiled code?
- Why are we passing raw metadata to the context window?
- Can we filter or normalize payloads before inference?
- Can we cache stable intermediate representations?
- Can a lightweight, specialized model handle this classification?
Budget constraints prevent teams from treating frontier models as a universal crutch for lazy architecture. When resources are unlimited, the easiest response is dumping an unstructured dump into a model and hoping for the best. Enforcing hard limits forces teams to build durable systems where AI automates code generation, and deterministic software delivers repeatable execution.