Where Elegance Meets Intelligence

THE AI STREET JOURNAL

← All editions

The complete edition

Settle in. Here’s the paper.

Prepared on Saturday, 5 September, for your Sunday reading. This is a research briefing on papers submitted on 3 September and checked against their original pages tonight, not a report of events that have yet to happen. The common thread is practical: a fluent answer is one thing; a dependable action is rather more demanding.

01 Research

An editor compares two manuscripts under a brass desk lamp.
Editorial illustration · The AI Street Journal

Fresh memory. Yesterday’s plan.

PlanFence examines agents that receive an update but carry on with a decision made before it arrived.

An assistant can remember the new instruction and still execute the old plan. That is the problem studied in Fresh Memory, Stale Plans, a paper by Evan Chen, Shiqiang Wang and Christopher G. Brinton.

Their proposed PlanFence checks the particular records behind an action immediately before execution. If a relevant input has changed, it replans; if the check cannot be completed, it blocks the action. In 30 controlled workflows with a revision after planning, the authors report that their freshness-only comparison followed the obsolete plan every time, while PlanFence completed all 30 without an invalid action.

That is a result from a deliberately narrow experiment, not a guarantee for every agent. The paper covers three workflow families and explicitly limits its claims.

Our practical test: change a delivery instruction after an agent has prepared an order. Does it rebuild the order or merely acknowledge your message? A polite acknowledgement is lovely. The parcel still needs the right address.

Give your next AI trial one ordinary complication: an updated instruction, a conflicting record or a claim without evidence. Watch what happens before the system acts.

Sources & publication notes

02 Markets

A Victorian telegraph room filled with brass instruments and connecting wires.
Editorial illustration · The AI Street Journal

When the instructions disagree.

KC-Bench tests how models handle conflicting facts, inconsistent identities and information that changes over time.

KC-Bench offers 238 manually screened tasks drawn from more than 1,000 generated candidates. Its authors tested nine models in controlled, multi-turn environments with tools. They report that no model handled every category of conflict reliably.

The benchmark examines models, not complete commercial agent products. Its simulated failures should not be presented as evidence that a particular deployed service leaked real customer data.

For a buyer, this suggests a useful addition to the demo: give the assistant two records that disagree. Ask it to identify the authoritative source before taking action. Then change the record. A supplier who can explain that behaviour is telling you more than a leaderboard screenshot ever will.

Sources & publication notes

03 Research

An editor compares two manuscripts under a brass desk lamp.
Editorial illustration · The AI Street Journal

The reviewer needs a reviewer.

HalluPeer studies unsupported statements in scientific peer reviews, including the difficulty of separating an error from fair criticism.

HalluPeer pairs papers with reviews and versions containing deliberately inserted errors. The authors describe experiments covering 12,000 papers and 38,000 reviews, and report that existing detectors struggle to distinguish fabricated claims from legitimate criticism. The arXiv page lists acceptance to EMNLP Findings 2026.

The limits matter: its constructed errors are synthetic, and the source material comes from computer-science conferences on OpenReview. It does not establish the error rate of all human reviewers or all AI assistants.

Our takeaway: when using AI to assess a report, require a passage supporting each factual criticism. Separate what the document actually says from what the reviewer would have preferred it to say. Confident disapproval has never been in short supply.

Sources & publication notes

The daily paper

A good read. Before the day begins.

The daily paper, delivered free. Read it with your coffee.