SHIFT, five months on: the story of a fleet that became something else

Today, if you open a pull request in almost any repository at CERC, an agent will have left comments on it within minutes. Nobody summoned it. It has a name, a configured “soul,” a model of its own — and a spot in a pixel-art office where you can watch it, live, sit down at a desk, type, wander off for coffee when it’s idle, and throw confetti when it’s done.

In February, none of this existed. SHIFT was a single container running on someone’s laptop that could barely open a PR without tripping over itself.

The distance between those two sentences is the story of this post. And the interesting part is that almost none of it was in the plan.

The dream we shipped

The original idea was clean and, we’ll admit, a little romantic: you describe a task in plain language, an agent understands it, clones the repo, writes the code, and hands you a pull request ready for review. The hero of the product was the agent that writes. That’s what the demo showed, that’s what kept us up at night with excitement.

Underneath it was a more serious bet, one we called the shift-left mindset: for an agent to deliver something good, the person describing the task has to think better — decompose the problem, be clear about intent, anticipate dependencies. The promise wasn’t just to automate code. It was that, by learning to talk to the machine, the engineer would become more of an engineer.

That was the script. What followed was the thing learning to walk on its own and, along the way, choosing to be far bigger than we imagined.

The timeline, in six jumps

Six months, around thirty thousand tasks, dozens of people, hundreds of repositories. But the numbers only mean something once you tell what happened between them.

A cast, not an agent

For a long time, all the agent’s behavior lived in a single script of about five hundred lines. It worked, but it was a monolith — changing one thing risked breaking three. Instead of patching it forever, we rewrote it as a modular system. And alongside it, the concept that changed how we think about the platform: the Shifties.

A Shifty is an agent’s “soul” — a name, a prompt, a model, a set of tools, and a way of delivering. Instead of one generic agent, we now had a cast. And the cast grew fast:

• a developer that implements features and fixes bugs by opening PRs — in one repository or several at once;
• a bench of reviewers — from generalist to domain specialist, plus one focused on QA and testing;
• a multi-repo documentation generator that spans several repositories and turns contract tests into browsable, unified docs;
• a Jira triager that reads and classifies tickets for a business domain without even cloning a repo;
• a Kubernetes workload fixer that tunes resources and HPA from KRR recommendations, and a multi-repo analyzer that can see a change spanning several services;
• a security remediation specialist that scans repositories for insecure configurations — missing or misconfigured SSL/TLS policy in Helm charts, and missing Subresource Integrity (SRI) on external scripts and stylesheets — and opens PRs with the fix: computing the SRI hashes, forcing HTTPS, and enabling the SSL policy where it’s absent.

Two traits run through the cast. Almost all are multi-repo — a single task can clone and work across several repositories at once, something the documentation generator leans on constantly to build a unified view from code scattered across services. And some are harness-specific: they bring their own toolset and configuration, cloned before the task, to specialize even further.

Today there are around fifteen of these, and the list keeps growing. Some were born in the UI, used by one team, worked, and became part of the catalog. From a single anonymous worker, SHIFT became a team with specialties.

And because a team deserves a place to work, we built The Office — a 2D, top-down, pixel-art office, hand-rendered on a canvas. It’s not decoration: it’s observability dressed up as a toy. Each active agent walks in through the door, sits at a desk, and types while it works; floats a little thought bubble when it reasons; wanders over to the couch, the coffee, or the water cooler when it’s idle; slumps when it errors; throws confetti when it finishes. Each one’s appearance is deterministic, so you learn to recognize your “coworkers” from across the room. On a busy day, a lead glances at the office and understands the state of the whole operation in a second. (Yes, it has a World Cup mode with the Brazilian flag and, in season, it got Carnival decorations. Working with agents doesn’t have to be grim.)

Wired into the ecosystem

The jump from a few hundred to thousands of tasks a month didn’t come from more people clicking “create task.” It came from SHIFT ceasing to be a website you visit and becoming a service other systems talk to.

Today a task can be born in many places:

• from the web interface, when a person describes what they want;
• from Azure DevOps webhooks, on every pull request opened;
• from Jira webhooks, on every ticket that needs triage;
• from direct calls to our internal API by other services — for example, the datagov service account (data governance) firing tasks, or the SRE portal, which automatically triggers workload fixes;
• exposed, in a controlled way, through Apigee, our API gateway, so integrations outside the cluster can reach the platform securely;
• from schedules, for whatever needs to run on its own from time to time.

The agent stopped being something you use and became something that participates. It reacts to events, is called by machines as much as by people, and fits inside the delivery flow instead of standing beside it.

In the same “integrate with everything” spirit, SHIFT also learned to route Claude through the Databricks AI Gateway, using the infrastructure’s own identity on GCP — with no static secret stored anywhere. Where it makes sense, the agents’ reasoning engine runs through the data platform the company already uses, instead of a parallel integration.

And to make sense of all of it, we moved the analytical side to BigQuery. Not for fashion: once a fleet passes thousands of runs a month, you want to ask things like “how much did each team spend,” “which Shifty runs most,” “how is volume trending” — and get the answer in a single query, over the whole history, instead of scanning collection by collection. The dashboards behind this entire post come from there.

Governance: freedom with guardrails

A platform anyone — and any service — can trigger, running models that cost real money against production repositories, can’t be a free-for-all. A good chunk of the last few months was less glamorous than building agents: it was building the guardrails that make it safe to let the fleet grow.

None of this is decoration. At a registrar, control and traceability are part of the product: knowing who spent what, on which model, in which repository is what lets you hand out autonomy without losing your footing. Governance doesn’t slow adoption down — it’s what lets us speed up without fear.

The review nobody asked for

Every pull request opened at CERC goes through an automated review. Not a sample, not an opt-in: 100% of PRs, within minutes of being opened, without anyone having to ask.

And with no friction in the day-to-day. The agent reads the PR in parallel, leaves its comments, and disappears — the author keeps their normal flow, now with an extra pair of eyes that never tires and never forgets. The effect is twofold: the quality bar rises for all of engineering, for free, and problems get caught early — often before they ever reach production. We won’t get into the details of what it finds, but the value isn’t subtle.

And it isn’t a shallow, one-shot pass. It’s iterative and context-aware: on every new revision of the PR, the agent looks at the diff between the branches — what actually changed — and reads the full history of prior comments and reviews on that PR. It doesn’t repeat what’s already been said, it tracks whether earlier points were actually addressed, and it adjusts the conversation as the code evolves. It’s closer to a human reviewer who follows a PR from start to finish than to a linter that fires and forgets.

And here’s the biggest surprise of the five months: the hero we designed was the agent that writes code. The one that delivers most, in the end, is the one that reviews — quiet, tireless, on every pull request.

The cheaper model that got better

Here’s a lesson that went against our instincts.

Early on, the default was to use the strongest — and most expensive — model for everything, on the comfortable assumption that quality came from the model. As volume grew, the bill grew with it, and it became clear that assumption needed testing. We made Haiku 4.5 the default for most tasks, bracing for the worst: a quality drop in exchange for savings.

That’s not what happened. Quality went up.

The reason is a little embarrassing: to make a smaller model shine, we were forced to finally write decent prompts — clearer instructions, better structure, tighter execution rules. We’d spent months paying for the horsepower of an expensive model to cover for lazy prompting. Once the prompt improved, the cheap model delivered just as well — in many cases better, because sharp instructions produce more consistent reviews than a powerful model reading a vague request.

The result showed up straight on the bill: cost per task fell about 45% while volume kept climbing. The cheapest lever was never the model. It was always the prompt — we just found that out when the budget forced us to look.

The real shift is in the mind

If there’s one thing five months taught us, it’s that the hard part was never the technology. The model, in the end, is the stable piece of the equation — the one that gave us the least trouble. The hard part, the one we’re still in the middle of, is the change in mindset.

Putting an agent on a task that already exists is the obvious move — and the least interesting. Automating the old process just gives you the old process, slightly faster. The question SHIFT forces us to ask is a different one: if an agent can take a part of this, how would I redesign the whole job? It’s not slotting an agent into yesterday’s flow. It’s looking at yesterday’s flow and asking whether it should still exist.

When that really happens, the bottleneck moves. It stops being typing code and becomes thinking clearly — decomposing the problem, defining the right thing, writing the spec an agent (or a human) can execute without guessing. It’s the shift-left mindset stepping off the slide and into the routine, by a path we hadn’t predicted: it wasn’t the engineer who changed to talk to the machine; it was the work that had to be rethought to fit a world with capable machines in it.

That’s why our bet, going forward, is as much about culture as about code. The platform is the easy part. The hard part is the rest: educating people to use AI well, creating the safety to question the usual way, and pushing — gently, but pushing — every team to reimagine their flows instead of just speeding them up. The competitive edge won’t belong to whoever has the best model; everyone will have the same models. It’ll belong to whoever has the nerve to redesign how they work.

And that’s uncomfortable, on purpose. Rethinking how you work means giving up habits that used to work. But that’s exactly where the value is — and it’s where we’re putting CERC’s energy: less into magic tools, more into people and culture, which is where real change lives.

What we actually learned

Building the agent was the weekend. Raising the fleet was the job. What actually consumed the engineering wasn’t the model — it was everything around it: giving each agent a personality and a purpose, wiring them into every corner of the ecosystem, seeing the whole operation at a glance, paying for tens of thousands of runs without the bill scaling with them, and — the hardest part — helping people rethink their own work.

We designed a pull-request factory and operate an ecosystem. We thought the hero would be the one who writes; it was the one who reviews, tireless. We assumed quality came from the more expensive model; it came from the clearer prompt. And we imagined a tool that would amplify people; what it amplifies most, in the end, is the need to rethink how people work.

The fleet keeps growing, faster than we planned. The work from here is the same as ever in platform engineering — keep it cheap, integrated, and reliable — only now with pixel-art coworkers, and with the certainty that the next big evolution will be less about the machine and more about us.

This post was written by: Allan Martins | COE - Architecture.