Every skill I've written about so far operates on material somebody handed it. The attribute fill-rate audit gets a listing export. The main image lab gets a listing URL and a library of 52 tactics to choose from. The premium one-pager gets a brief. In all of them the interesting engineering is about what the model is allowed to say — don't invent a certification, don't touch the hero, choose from the list instead of making something up.
This one is different, and the difference changed what the guardrails are for.
amazon-review-audit doesn't get handed anything. It gets an ASIN, opens a browser, and goes to find the evidence itself. It's the only skill in my library that operates a live session on a website I don't own, and once you do that, "don't invent things" stops being the hard problem. Two new ones show up: don't touch anything, and don't confuse a page you couldn't read with a page that was empty.
The problem, and what it costs to do by hand
Every piece of creative work I've published this year eventually points at the same input. The A+ copywriting post says the only qualification that predicts good A+ copy is whether the writer has read your reviews. The return-reason framework says your buyers already ran a merchandising audit on your image stack and filed it. The advice is correct and almost nobody follows it, and the reason isn't laziness.
Doing it properly is an hour to ninety minutes per ASIN. Open the detail page. Transcribe the title, the bullets, the A+ headings, the product details, so you know what the listing currently claims. Open the reviews. Then — and this is the part that gets skipped — filter to five stars, four, three, two, one, and read a real sample from each band, because the themes are completely different in each and the ones that matter for creative are almost never in the five-star pile.
On one SKU, fine. Do it. On a 20-ASIN catalog that's a week of a strategist's time, which is why brands do it once for the hero product at launch and then never again, and why the reviews on the other nineteen listings sit there for three years describing a problem nobody has read.
The build
Three files. SKILL.md is 678 words. references/browser-js-patterns.md is 389 words across 122 lines. The agent manifest is 43 words. It lives in ~/.codex/skills/amazon-review-audit/ and runs on the Codex in-app browser — no scraping service, no proxy pool, no API key, just the browser I already have open and the Amazon session I'm already logged into.
The workflow is seven steps, and step two is the one doing the real work:
2. Determine the input type:
- Current Amazon product page: use it.
- Amazon product URL: open it.
- ASIN: open https://www.amazon.com/dp/<ASIN>.
- Search term: open Amazon, search the term, and choose a strong
organic product listing with a high review count unless the user
names another selection criterion.
That last branch is the difference between a tool and a workflow. "Audit the top result for stainless steel water bottle" is the question a strategist actually asks when they're working a category they don't own yet, and the skill resolves it without me having to go find an ASIN first.
The extraction itself is plain JavaScript run in the page. Nothing clever:
title: text("#productTitle"),
bullets: texts("#feature-bullets ul li span.a-list-item")
.filter((t) => !/^Make sure/i.test(t)),
aplusHeadings: texts("#aplus h1, #aplus h2, #aplus h3, ..."),
That filter on the bullets is my favourite line in the file and it took a wasted afternoon to earn. Amazon injects its own "Make sure this fits by entering your model number" line into the feature-bullets container on a lot of listings. Without the filter, that boilerplate comes back as one of the brand's five bullets, and then the model dutifully analyses it, and you get a report telling a client that one of their bullets is about model number compatibility when they never wrote it. Nothing errored. The output looked fine. It was just wrong in a way you'd only catch if you knew the listing.
Stratified sampling is the whole idea
The part of this skill that actually makes it worth having is four lines of URL construction:
https://www.amazon.com/product-reviews/<ASIN>?...&sortBy=recent&filterByStar=five_star
...&sortBy=recent&filterByStar=four_star
...&sortBy=recent&filterByStar=three_star
...&sortBy=recent&filterByStar=two_star
...&sortBy=recent&filterByStar=one_star
Two deliberate decisions in there, and both of them are merchandising decisions, not engineering ones.
Filtering by star band, rather than reading "the reviews." Hand a model a review page and tell it to summarise, and it reads what's on the page. What's on the page is Amazon's default view, which is weighted toward helpfulness, which means it is overwhelmingly positive and overwhelmingly old. You get back a summary of the listing's greatest hits. It is pleasant, it is accurate, and it is worthless for creative work, because the frames you need to build are the ones that answer the three-star review — the buyer who liked it enough to keep it and still had to explain something the page should have explained first. Three-star and two-star bands are where the expectation gaps live. They're also the bands with the fewest reviews, which means they're the ones a naive sample misses entirely.
sortBy=recent, not top-rated. A product changes. Formulations get reformulated, suppliers get switched, a pack size moves, packaging gets redesigned to save on freight. The reviews from 2023 describe a product that may not be the one currently in the box. If you're briefing a shot list off review data, you want the data that describes what's shipping now.
The default is up to ten visible reviews per band, and the skill is told to state the sample size in the report. That sentence is there because a report that doesn't say how many reviews it read looks exactly like a report that read all of them. An audit that quietly sampled 12 reviews and presented five confident themes is worse than no audit, because somebody will brief a photoshoot off it.
The guardrails changed class
In the creative skill library I audited a couple of weeks ago, the constraint sentences were all about fabrication: don't invent a country of origin, don't build a concept that depends on an unsupported claim, don't generate finals. Every one of them protects against the model saying something that isn't true.
A browser skill needs those too, but it needs two more kinds that a text pipeline doesn't.
An action boundary. Verbatim from the file:
Use the user's logged-in Amazon session only for reading publicly available listing/review content. Do not add to cart, buy, change account settings, submit reviews, vote, or otherwise mutate account state.
That sentence exists because the skill is driving a browser that is authenticated as a real person with a real payment method and a real reviewer identity. Nothing about a review audit requires writing. But the model has hands now, and a model with hands that hits an unexpected page will improvise. Enumerating the specific mutations — cart, purchase, settings, review submission, helpful votes — is more useful than a general "be careful," because "be careful" is not a rule, it's a mood. This is the same principle as writing only into the generator line: the value is in the specificity, not the sentiment.
A stop instead of a workaround. Two of them, actually:
If Amazon requires sign-in to view reviews, pause and ask the user to log in manually in the in-app browser, then continue.
Do not use broad web search as the primary source unless Amazon blocks access.
The second one is the important one and it's the failure I'd have bet on. Give an agent a job and a wall, and it will find a way around the wall, and the way around this particular wall is a third-party site that has summarised the reviews already. It will come back with a report in the correct format, with themes and complaints and expectation gaps, assembled from somebody's SEO page about the product. It will be plausible. It will not be your reviews. The rule doesn't ban the fallback — sometimes Amazon genuinely blocks and a directional read is better than nothing — it demotes it and forces it to be declared.
A UGC rule:
Do not paste large volumes of raw review text. Reviews are user-generated content; summarize themes and use short snippets only when necessary.
Partly hygiene, partly output quality. A report that dumps 40 reviews back at you hasn't done the job. The job is the synthesis.
What broke: the selector problem, which is the old problem wearing a new costume
I'm not going to invent a dated war story. What I can tell you is the shape of the failure, because it's the same one I named in the attribute fill-rate build log and it has a mechanical cause here instead of a cognitive one.
Amazon's DOM belongs to Amazon. #productDescription, #aplus p, [data-hook='review-body'] — every one of those is a promise somebody else can break on a Tuesday without telling me. When a selector stops matching, document.querySelector returns null, clean() turns null into an empty string, and the extraction object comes back with productDescription: "".
Then the model reads that object and writes a report. And unless something stops it, the report says the listing has no product description.
That is the exact failure I built a whole sentence into the attribute skill to prevent: the model could not distinguish "I can't see it" from "it isn't there." In a text pipeline that failure comes from a model over-reaching. Here it comes from a CSS selector, which means it can happen silently, at scale, on every ASIN in a batch, and produce a beautifully formatted recommendation to "add a product description" to a listing that has one.
The mitigations are unglamorous. The report format has an explicit line for provenance — every audit has to state whether findings came from the product page, the all-reviews page, star-filtered samples, or a limited sample due to access problems. The extraction captures pageText alongside the parsed fields, so an empty structured field sitting next to a page full of text is visible rather than invisible. And there's a variant note in the file:
If a product page aggregates parent/child ASIN reviews, note that the review page or filters may include related variants.
That one has bitten real audits. On a parent listing, the review pool is pooled. A brand gets handed a complaint theme about a colour they don't sell in that ASIN, panics, and rebuilds a frame to fix a problem belonging to a sibling SKU.
Cost, and the number I'm not going to give you
The workflow was an evening. The extraction patterns took a couple of sessions of watching things come back empty and going to look at the page. Running it costs a few dollars per ASIN — it's a browser session and a few thousand tokens of synthesis, not a fine-tune.
There is no cron job. No daemon. No unattended write path anywhere in this thing. A human invokes it, a human reads the output, and if it turns into a creative brief a human writes the brief.
I'm not going to publish an hours-saved-per-month figure across a nine-person team. I'd be making it up, and a made-up efficiency number is exactly the thing I'd call out in somebody else's post. The narrow, true version: getting from "here is an ASIN" to "here is what the listing currently claims, what buyers actually say across all five star bands, and where those two things disagree" goes from an hour-plus of skilled attention to a few minutes of it. The judgment about which disagreement is worth a photoshoot is unchanged, and that's the part I'm paid for.
The justifying number is on the other side of the ledger anyway. The most common creative-attributable return in every category I work in is "smaller than I expected," and it is sitting in the three-star reviews of listings whose owners have never read them. One frame that prevents that return pattern on one SKU is worth more than every minute this skill has ever saved.
What an operator could replicate
Six things, in order of how much they'd change your output:
-
Sample across star bands, not across "reviews." This is the entire value and it requires no automation at all. Open five URLs, read ten reviews in each. If you do nothing else in this post, do this.
-
Sort by recent. Your product changed. Your reviews from 2023 are describing something else.
-
Write the action boundary before the workflow. If your agent can authenticate as you, enumerate the specific things it may not do. "Don't mutate state" is a mood. "Do not add to cart, buy, change account settings, submit reviews, vote" is a rule.
-
Give it a stop, not a workaround. Any agent that fetches will substitute when fetching fails. Name the fallback, demote it, and require it to be declared in the output.
-
Separate can't-see from isn't-there, mechanically. In a browser pipeline this means capturing the raw page text next to the parsed fields, so an empty field with a full page behind it is detectable. An audit that confidently reports an absence is worse than one that reports nothing.
-
Make it state its own sample size. The cheapest honesty mechanism there is. A report that tells you it read 41 reviews across five bands is a document you can argue with. One that doesn't is a vibe.
The pattern across all of these, and across the last three build logs: the model's job is to go look, report what it saw, and be explicit about what it couldn't see. Everything expensive that has ever come out of one of these workflows came from an output that was confident about something it never actually looked at.