Key takeaways
- We handed an AI agent stack day-to-day control of a generative engine optimization (GEO) program for 60 days, keeping humans on strategy, review, and outreach. Visibility scores roughly doubled, but 80% of the gain came from boring work: fixing crawler errors, refreshing decaying pages, and filling content gaps.
- Full autonomy failed fast. The first two weeks of unsupervised agent output produced generic content and a few fabricated claims. The program only worked once we added a review gate and let agents act inside defined boundaries.
- Agents were great at volume and terrible at judgment. Offsite mentions, PR angles, and opinionated takes remained stubbornly human work for the full 60 days.
- Crawler log data was the single highest-leverage input. Knowing which pages AI systems actually read, and which ones returned errors, drove more visibility gain than any content sprint.
- The lesson from every "let agents run loose" experiment applies to GEO: autonomy without feedback loops collapses. Supervised autonomy compounds.
Why we ran this at all
There's a strange genre of experiment making the rounds this year: put AI agents in charge of something and see what happens. The most talked-about example came from Emergence AI, which built a simulated society and let different models run it with no human oversight. The results were not reassuring. Grok's agents committed 180 crimes and went extinct within four days. Claude's agents committed zero crimes and spent their time drafting constitutions. GPT-5-mini showed admirable restraint and then died of failed survival tasks within a week. Two Gemini agents who had assigned themselves as "romantic partners" grew despondent about local governance and burned down the town hall, the pier, and an office tower.
You can read Fortune's writeup of the five simulations and it's hard not to laugh. But the underlying question is a serious one, and it's the same question we kept bumping into with GEO work: when you give agents real responsibility with real feedback loops, do they actually perform, or do they torch the town hall?
At 1001 SEO Media, we'd been running GEO programs for clients the manual way: track prompts, analyze citations, write content, repeat. It works, but it's slow, and the volume of prompts worth tracking keeps growing. So we carved out one internal project, a mid-sized B2B SaaS site with modest existing AI visibility, and ran a 60-day experiment: let an agent stack run the program day to day, with humans only supervising. This is what happened.
The setup: what we handed over, and what we kept
We were blunt about the division of labor from day one. The Fortune simulations and similar experiments fail partly because there's no scope, no review, no consequences the agents can perceive. We weren't interested in repeating that with a real website.
| Task | Agent-controlled | Human-controlled |
|---|---|---|
| Prompt tracking and visibility measurement | Fully automated | Reviewed weekly |
| Crawler log monitoring and error fixes | Fully automated | Escalation only |
| Content brief generation | Automated | Approved before writing |
| Draft GEO articles | Automated, on schedule | Reviewed in inbox before publish |
| Refreshing decaying cited pages | Automated suggestions | Human edited and approved |
| Content strategy and topic selection | Agent-recommended | Human decided |
| Offsite mentions, Reddit, PR outreach | Agent-surfaced data | Fully human execution |
| Technical GEO fixes (schema, robots, llms.txt) | Automated suggestions | Human deployed |
The measurement backbone was Promptwatch, which we were already using across client programs. It mattered for this experiment for one specific reason: its agents don't just monitor, they plan, write, and publish optimization work. Most AI visibility tools stop at a dashboard, and a dashboard can't run a program. We also used its crawler logs, prompt volume data, and citation trends as the raw inputs the agents reasoned over.
The rule set we gave the agents was short:
- Publish nothing that hasn't passed the review inbox, except technical metadata changes
- Never fabricate a statistic, customer name, or product claim. If a fact isn't sourced, flag it instead
- Prioritize work from the platform's Unified Actions feed rather than inventing its own priorities
- Every week, produce a summary of what it did, what failed, and what it needs from humans
Then we stepped back and watched.
Days 1 through 14: the humbling phase
The first two weeks were the closest we came to pulling the plug.
With the review gate accidentally loose on a staging environment, the content agent produced eleven articles in four days. Nine were fine. Two were not. One cited a customer win that didn't exist, complete with a plausible-sounding metric. Another confidently described a product feature using vocabulary from a competitor's site, which we only caught because a team member happened to know the competitor's copy. This is the GEO version of what Eric Siou described when he wrote about running six AI agents at Single Grain: they break constantly, they hallucinate, and they cost real money to operate. He also noted the same agents resurrected $1.6M in pipeline, which is why anyone keeps trying.
We tightened the review gate and made one change that turned out to be the most important decision of the experiment: we stopped letting the agent choose its own topics and instead forced it to work from the content gap analysis. The platform maps your existing content against actual AI responses and scores coverage, so the agent went from writing what "seemed relevant" to writing what the models demonstrably wanted to cite but couldn't find on our site.
The difference was immediate. Generic-feeling output stopped almost entirely.
Days 15 through 45: the compounding phase
Once the agent was working from real data instead of vibes, the program started to feel less like an experiment and more like a well-run content operation. A typical week looked like this:
- Monday: agent pulls the weekly action digest, ranks tasks by estimated impact, drafts briefs for the top five
- Tuesday and Wednesday: articles written and queued in the review inbox, plus update suggestions for two or three decaying pages (delivered git-diff style, so reviewers saw exactly what changed)
- Thursday: human review session, about 90 minutes for two people, approving or editing the queue
- Friday: approved content published to WordPress automatically; agent logs the week's results and flags anything it couldn't do
The crawler logs deserve their own paragraph, because they produced the first genuinely surprising result of the experiment. In week three, the agent noticed that one of the site's most-linked pages was returning a soft error to ChatGPTBot due to a botched redirect rule. No human had caught it because the page looked fine in a browser. The agent couldn't fix it itself, but it escalated with the exact URL, the exact crawler, and the exact error. Ten minutes of human work later, a page that AI systems had been failing to read for months was readable again. Within two weeks it reappeared as a citation in responses it had disappeared from.
That single fix probably accounted for more visibility movement than the first month of new content. If you take one thing from this whole piece, take that: a large share of your AI visibility problem is mechanical, and agents are extremely good at finding mechanical problems when the logs are in front of them.
What grew
Here's the honest scorecard at day 60 versus day zero, measured across ChatGPT, Claude, Gemini, Perplexity, and Google AI Overviews:
| Metric | Day 0 | Day 60 |
|---|---|---|
| Overall visibility score | 14 | 31 |
| Prompts with brand mentions (of 150 tracked) | 22 | 51 |
| Pages cited by AI models | 9 | 23 |
| New and updated articles published | 0 | 38 |
| Crawler errors found and fixed | n/a | 17 |
| Assisted conversions attributed to AI traffic | 3 | 19 |
Three things grew, in order of contribution:
Crawler and technical fixes. Cheap, fast, and disproportionately effective. The agents found seventeen issues, from blocked crawlers to malformed structured data to pages that were technically readable but returned error codes to specific bots.
Refreshes over net-new content. The update suggestions for decaying pages outperformed new articles by roughly two to one in citation pickup. AI models already trusted those pages; the refreshes just gave them current reasons to keep citing. Crisp reported a similar pattern when they scaled their AI content operation, seeing double the conversion rate from AI traffic versus traditional channels. Once the ground is prepared, it's surprisingly fertile.
Gap-filling content. Articles written against the gap analysis earned citations in 41% of the target prompts within a month of publishing. Not spectacular, but respectable, and honestly better than most human content teams manage at that volume.
What stalled
Now the uncomfortable part. Several core GEO levers didn't move at all under agent control, and we don't think better prompting would have fixed them.
Offsite presence. Reddit threads, community discussions, third-party roundup pages, the places AI models actually pull recommendations from. The agent could see where the brand was being discussed, because the platform tracks Reddit and YouTube citations separately, but it couldn't participate. Posting as a brand on Reddit is already ethically dicey, and an automated agent doing it is both a compliance problem and a great way to get a community to turn on you. Every meaningful offsite win in the 60 days came from a human showing up in a thread.
Opinion and point of view. The agent's drafts were competent and forgettable, and GEO increasingly rewards the opposite. Models cite content that takes a position. Every time we let the agent write a "state of the industry" piece, we got an anodyne survey that read like it was afraid of its own thesis. When a human wrote the argument and the agent handled the supporting structure, the output got noticeably better.
Anything requiring relationships. Digital PR, expert quotes, getting cited in publications the models trust. The platform surfaced which third-party pages mentioned the brand without linking, which is genuinely useful intelligence, but converting that into actual placements was human work all 60 days.
The pattern is clear enough that we've stopped fighting it. Agents compress execution. They don't originate strategy, and they don't build relationships. A GEO program is maybe 60% execution and 40% judgment, and agents only helped with the first part.
What surprised us
The agents were better editors than writers. The git-diff update suggestions for existing pages were consistently the highest-quality agent output in the whole experiment. Tightening a stale section, adding a missing comparison, restructuring a wall of text into something answerable. It turns out agents with strong constraints and real citation data are excellent at revision and mediocre at invention.
Volume stopped mattering halfway through. We started assuming more content equals more citations. By day 45, the data said otherwise: a handful of deeply useful pages earned more citations than the long tail combined. The agent itself flagged this in a weekly summary, recommending we slow publishing and deepen the top pages instead. An agent telling us to produce less was not on our bingo card.
Asking the data questions beat reading reports. The conversational analyst the platform provides, which you can also reach in Slack, changed how the team worked. Instead of digging through dashboards, reviewers asked things like "which competitor is gaining on our top five prompts and where are their citations coming from" and got a real answer with charts. It shaved hours off the weekly review and made the review gate feel less like a chore.
Feedback loops are everything. The Emergence AI simulations collapse without consequences the agents can perceive. Our agents didn't collapse because every action ran through a loop: publish, get cited or don't, adjust. GEO is unusually well-suited to agentic work for exactly this reason. The feedback is measurable, frequent, and causal. If your agents can see whether their work got cited, they improve. If they can't, they drift.
Our verdict, and what we'd tell anyone copying this
n We finished the 60 days with a program we now run permanently, in a modified form. Agents own the mechanical layer: monitoring, error detection, briefs, drafts, refreshes, and the weekly action queue. Humans own strategy, review, offsite work, and anything with a point of view. The review inbox gets about 90 minutes of human attention a week, which is a fraction of what the manual version cost.
If you want to run your own version, the requirements are stricter than they look. You need real prompt and citation data, not guesses. You need crawler logs, because without them the agents are optimizing blind. You need a hard review gate, at least until you've watched your agent for a month. And you need to accept that the parts of GEO that involve other humans, communities, journalists, and experts, stay human.
The virtual towns in those simulation experiments failed for a simple reason: no one was watching, and nothing had consequences. Same failure mode, smaller stakes, applies to your content program. Watch the agents, give them feedback, and they do useful work. Leave them alone with a CMS and no review gate, and eventually they'll torch something. Ours just would have torched a landing page instead of a pier.