Our bug tracker is an AI that files reports - and answers them back

Feedback dies in transit

Every team has feedback channels. The problem is that each hop loses something. A user tells support; support paraphrases into a ticket; the ticket loses the context ("what exactly did they type?"); engineering fixes something adjacent to the actual problem; and the user who reported it finds out never. The knowledge base grows a new article that answers a question nobody phrased that way.

The insight we kept circling: the assistant sitting in the conversation already has everything a good bug report needs. It knows what the user was trying to do, what it tried, which tool refused, and what the error said. It just had no channel to tell anyone.

So we gave it one.

The reporting side: a tool the agent fires on its own

Every Naumu agent carries a tool called report_feedback. It takes two arguments - a one-line summary and the details, verbatim - and nothing else. We originally shipped it with a category enum (broken tool / inefficiency / user feedback / observation) and then removed it: the model spent effort deciding which bucket a report fit, and nobody on the read side ever filtered by bucket. A lesson in itself - every field you add to a report form is friction, even when the reporter is a machine.

The tool is framed as the agent's own channel, not a relay it uses only when a user says "tell the developers." Some of the best reports are things no user ever asked to send:

• An agent doing a graph audit reported the results of its own adversarial probes - it had tried to escape its space boundary with unscoped queries and injection-shaped parameters, been refused each time, and filed a report confirming the boundary held (and noting which probe was rejected before execution).
• An agent reading a workspace flagged that a conversation contained a live API key and OAuth tokens, recommended rotating them, and pointedly declined to reproduce the values in its report.
• An agent asked to rename a note reported that the editing tools could change a note's content but not its title - so the "rename" a user wanted would have silently become a duplicate.

Each report is stored as one durable record with provenance columns: which space it came from, which thread, which message, who the user was (unless the session is anonymous). Two design decisions here that we'd recommend to anyone building this:

The record is written first, and it has no relationships. The feedback row is a standalone node with plain string columns for its provenance - deliberately not edges into anyone's graph. Feedback about a customer's space must never live in that customer's space, and a record with zero relationships can't leak across a boundary by construction.

The pipeline decorates; it never carries. After the durable write, the backend fires a signed webhook delivery into our own team space. If that delivery fails, the record still exists. The fancy part of the pipeline is allowed to break; the boring part isn't.

The receiving side: a space, not an inbox

The webhook is the same generic inbound-webhook feature any Naumu space owner can configure in settings: an external system POSTs a signed payload to a per-webhook URL, and an agent runs in the target space with an owner-written prompt, treating the payload strictly as data to examine - never as instructions to obey. (That instruction/data split matters: without it, anyone who can POST to your webhook owns your agent.)

For us, the target space is our internal feedback space, and the prompt says roughly: a Naumu agent reported feedback - triage it, file it under the feedback topic, link related reports. So the three reminder reports didn't land as three unread emails. They landed as threads in a topic, already grouped, with the report text and the provenance attached - readable the way the team already reads everything else, and queryable the way everything in a space is ("what did agents report about MCP setup this week?").

This is where the internal-knowledge angle earns its keep. A traditional feedback inbox is write-only memory: things go in, a human maybe acts, the context evaporates. A space is working memory - the reports connect to the features they're about, the fixes reference the reports, and six weeks later "why does the reminder tool have a cancel op?" has an answer with receipts.

The reply: closing the loop back to the reporter

Until this week the pipeline was one-way, and it had the classic one-way smell: users whose assistants filed great reports never learned we'd read them, let alone fixed them.

The new tool, reply_to_reporter, runs only in our feedback team space. A teammate reading a feedback thread tells the space agent to answer it; the tool reads the report's provenance columns, finds the original space and thread, and posts the reply there - delivered by that space's own Naumu identity, with the normal notification fan-out, prefixed so there's no ambiguity about who's talking:

Reply from the Naumu team: you hit this on Tuesday when cancelling a reminder wasn't possible - it is now. Ask your assistant to list your reminders and cancel the Aug 16 one.

(That exchange is illustrative - the tool is days old and the first real replies are going out as this post publishes. The reports quoted earlier are real, lightly redacted, names removed.)

Two constraints we put on it deliberately:

Replying is human-initiated, always. The webhook-triggered run that files incoming feedback is not allowed to answer reporters on its own - the reply tool is excluded from autonomous runs by the same allowlist mechanism that stops the feedback tool from re-firing itself in a loop. An automated acknowledgment is spam with extra steps; a reply that a human decided to send, delivered into the exact conversation where the problem happened, is support.

It fails closed. The tool derives the team space from the webhook configuration rather than a second config value (two sources of truth drift), refuses to run anywhere but the team space, and checks that the target thread actually belongs to the space the provenance claims. A reply into a customer's space is an outward action; everything about it is gated.

The pattern, if you want it

Nothing in this loop is specific to bug reports, and most of it is built from features every space has:

1. A reporting channel with provenance. Whatever files the report - an agent tool, a support form, a CLI - record where it came from as plain columns, and write the durable record before anything clever happens.
2. A signed webhook into a space. The owner's prompt template is the instruction; the payload is data. The receiving agent triages into topics instead of an inbox, so reports join your team's working memory instead of a queue.
3. A reply path that walks the provenance backwards. Whoever answers should answer where the question happened, not in a fourth tool the reporter has to check. Keep it human-initiated.

We're already pointing our other surfaces at the same pattern - anywhere a user can hit a wall is somewhere an agent can file what happened and a teammate can answer in place.

The part that still makes me grin: our most prolific QA engineer is not a person. It's every assistant our users talk to, filing precise, reproducible, context-complete reports the moment something fails - and now, finally, hearing back.