Chat insights: observability & self-healing for your guest AI

Most chatbots are fire-and-forget: you paste in some blanket info, switch them on, and never learn which questions they fumbled. Haven treats your guest AI chat as a full-cycle tool. Every reply is graded, gaps are explained and grouped, the fix is one click, and the loop keeps running on its own — so you can automate guest messaging without going blind.

Everything lives in Home → Performance → Chats on your Host Dashboard.

The loop at a glance

1. Detect — every guest question is paired with the assistant's reply and graded: Answered, Low confidence, or Couldn't answer, with a one-line reason when it struggled.
2. Explain — flagged replies are grouped into topic insights (Parking, Check-in / Access, EV charging, Pets…) with counts, the properties they came from, and every cited conversation.
3. Fix — click Add context: the dialog is prefilled with the suggested wording, the right property or collection, and the right visibility. Save it to one site or all sites. It is live for the next chat.
4. Verify — the next reply on that topic is graded like every other. Insights re-run automatically as new chats arrive, so a fixed gap drops off the list by itself.
5. Ask — ask your chats anything in plain English and jump straight into the cited conversations.
6. Escalate — when a human is needed, the full transcript is already on your Messages thread.

The Chats tab

Search and filters

A structured search bar at the top works like the session-replay search: type free text, or add tokens for property, collection, surface (Property site / Multi-unit site / Guidebook), topic, signal, guest name, or email. A period picker sits inline. The table header shows the total for the period plus signal pills — Couldn't answer, Low confidence, Errored reply — click one to filter to just those chats. An All / Property site / Multi-unit / Guidebook toggle narrows by surface.

Every row shows the site's thumbnail, who chatted (name/email when known), the surface, the first question, and the reply grade.

Transcript drawer

Click a row to open the full conversation. Every flagged reply shows an amber marker with the reason it struggled and a gold Add context button that opens the Add-context dialog prefilled for that exact gap. The header pill summarises how many replies in the chat "could use more context". When a session replay exists for the same visitor, Watch session takes you to it.

Chat insights panel

Below the table, Chat insights lists one row per topic detected across your chats for the period. Each row shows the topic, how many chats cited it, and — when the assistant struggled — "N couldn't answer, N low confidence" as links. Insights are two kinds:

• Gap — at least one cited reply was partial or unanswerable. This is what to fix.
• Question theme — guests ask about it a lot, and the assistant handled it. Useful for what to put on your listing page or in your FAQs.

Click a row (or a tier link) to open the evidence sheet: every cited conversation with site and surface chips, Answered / Low confidence / Couldn't answer badges, filter chips, and expand-to-transcript with the flagged reply highlighted. Add AI context is one click away here too, prefilled and scoped to the sites where the gap was detected.

Insights are computed automatically in the background (roughly every few hours as new chats come in) — you never trigger a report.

Ask your chats

The slim input above the table takes plain-English questions — "Are people asking about parking?", "What do guests ask before booking?", "What couldn't the assistant answer this week?", "Do guests ask about pets?". The answer cites the exact conversations; click a citation to open that chat scrolled to the cited message. The same ability is available to the Haven assistant in your dashboard, so you can ask it there too.

Add context: fix once, everywhere

Wherever you see Add context (transcript drawer, insight row, evidence sheet), the dialog opens prefilled:

• Context — the suggested wording the grader thinks would have answered the question. Edit it freely.
• Applies to — defaults to the property or collection where the gap was detected when there is exactly one; otherwise All chats. Detected-on sites are listed first.
• Visibility — Private (guests-only) is the default when every flagged chat came from a guidebook; otherwise Public.

Save, and the entry lands in Messages → AI knowledge alongside everything else you have taught the assistant. It is live for the next chat — no redeploy, no re-training. See AI guest chat for scopes, knowledge sets, and public vs. private.

Train it by talking to it

Your own conversations count. Open your property site, multi-unit site, or a guidebook as a guest would, ask what a guest would ask, then come back to Chats: your session is there, graded, with a reason for anything it struggled on. Add context, ask again, and watch the grade flip. It is the quickest way to tune the assistant before guests arrive — on each surface, because each has a different knowledge boundary.

What is graded (and what isn't)

• Grades are per reply, based on that reply's own quality with the nearby questions as context — a guest asking two things in a row does not confuse it.
• Two flagged tiers only: Couldn't answer (unanswerable with the knowledge it had) and Low confidence (partial or hedged). Everything else is Answered.
• Errored reply is a separate signal for a reply that failed to generate (network, aborted stream); it is not a knowledge gap.
• Insights show at most one row per topic for a period, so a busy week does not produce a wall of near-duplicates.

Privacy

Chats are keyed to your account and to the specific property or collection. Guests are identified by the name/email they gave (guidebook gate, booking, or form) or shown as Guest; anonymous visitors are never merged with each other. Booking-only knowledge never reaches the public site chat — see AI guest chat.

Related articles

• AI guest chat — surfaces, public vs. private, AI knowledge scopes and knowledge sets
• Performance analytics — the Analytics and Session replays views next door
• Guest communication — where hand-offs land
• Guest Portal