# KHAO Data Schema

KHAO publishes a daily AI & technology news digest in multiple machine-readable formats. This document describes each format's fields so external tools, RAG pipelines, and AI agents can ingest the data without guesswork.

---

## 1. JSON Feed (`/feed.json`)

Format: [JSON Feed 1.1](https://jsonfeed.org/version/1.1)

Top 40 stories from the most recent digest, sorted by importance score descending.

### Feed-level fields

| Field | Type | Description |
|---|---|---|
| `version` | string | Always `"https://jsonfeed.org/version/1.1"` |
| `title` | string | `"KHAO · Daily AI Signal"` |
| `home_page_url` | string | Canonical site URL |
| `feed_url` | string | Self-referencing feed URL |
| `description` | string | One-line description |
| `language` | string | Always `"en"` |
| `items` | array | Array of story objects (see below) |

### Item-level fields

| Field | Type | Description |
|---|---|---|
| `id` | string | Canonical URL (story page or original article) |
| `url` | string | Same as `id` |
| `title` | string | Story headline, emoji-stripped |
| `content_text` | string | Summary + key facts, plain text, ≤500 chars |
| `date_published` | string | ISO 8601 timestamp |
| `tags` | array | Category + topic tags (e.g. `["models", "openai"]`) |
| `_importance` | int | KHAO importance score, 0–100 (see scoring below) |
| `_source` | string | Publishing outlet name (e.g. `"Anthropic"`) |

---

## 2. RSS Feed (`/feed.xml`)

Format: RSS 2.0 with Atom and KHAO namespace extensions.

Namespace: `xmlns:khao="https://khao.ai/ns/1.0"`

### Standard item elements

| Element | Description |
|---|---|
| `<title>` | Story headline |
| `<link>` | Canonical URL |
| `<guid>` | Same as link, `isPermaLink="true"` |
| `<pubDate>` | RFC 822 timestamp |
| `<category>` | Category (Models, Research, Business, etc.) |
| `<description>` | Summary text, ≤500 chars |

### KHAO extension elements

| Element | Description |
|---|---|
| `<khao:importance>` | Importance score 0–100 (integer) |
| `<khao:source>` | Publishing outlet name |

---

## 3. Daily Markdown Digests (`/archive/YYYY-MM-DD.md`)

One file per day. YAML frontmatter followed by story blocks.

### YAML Frontmatter

```yaml
---
title: "KHAO — Monday, April 21, 2026"
issue_date: 2026-04-21
kind: daily-digest
story_count: 30
source_count: 18
generated_at: "2026-04-21T06:12:00Z"
---
```

### Story Block Format

```markdown
## Section · Models  (8)

## ★★ 82 · Story Title Here
**Source Name · 2026-04-21T09:30:00Z**
Two-sentence summary of the story. Second sentence with key detail.
Tags: `tag1`, `tag2`
Read: <https://original-article-url>

### Body
Full article body text, paragraphs separated by blank lines.

---
```

### Story Block Fields

| Field | Description |
|---|---|
| `★★ 82` | Stars (visual) + importance score (0–100) |
| Title | Headline, cleaned of emoji and attribution kickers |
| Source line | `**Outlet · ISO-timestamp**` |
| Summary | 1–3 sentences, plain English |
| Tags | Comma-separated topic tags in backticks |
| Read | Original article URL |
| Body | Full article body (extractive, 40–60% of original length) |

---

## 4. Archive Index (`/archive/index.json`)

Machine-readable catalogue of all available daily digests.

```json
{
  "digests": [
    {
      "date": "2026-04-21",
      "issue": 388,
      "story_count": 30,
      "md_url": "https://khao.ai/archive/2026-04-21.md",
      "html_url": "https://khao.ai/archive.html#2026-04-21"
    }
  ]
}
```

| Field | Type | Description |
|---|---|---|
| `date` | string | ISO 8601 date |
| `issue` | int | Sequential issue number (starts at 100) |
| `story_count` | int | Number of stories in that digest |
| `md_url` | string | Direct URL to the Markdown digest file |
| `html_url` | string | Deep link to the archive page anchored to that date |

---

## 5. Search Index (`/search-index.json`)

Flat array of all stories across all digests. Used by the archive page's client-side search. Also suitable for RAG ingestion.

```json
[
  {
    "t": "Story headline",
    "s": "Source Name",
    "x": "Two-sentence excerpt.",
    "d": "2026-04-21",
    "i": 82,
    "u": "https://original-url",
    "c": "models"
  }
]
```

| Key | Field | Type | Description |
|---|---|---|---|
| `t` | title | string | Story headline |
| `s` | source | string | Publishing outlet |
| `x` | excerpt | string | Summary text (1–2 sentences) |
| `d` | date | string | Issue date (YYYY-MM-DD) |
| `i` | importance | int | Importance score 0–100 |
| `u` | url | string | Original article URL |
| `c` | category | string | Category slug (see below) |

---

## 6. Importance Score (0–100)

KHAO scores every story on a 0–100 scale combining 7 factors:

| Score range | Meaning |
|---|---|
| 0–19 | Trivial or ultra-niche: vendor notes, minor version bumps |
| 20–39 | Low signal: incremental updates, regional news |
| 40–59 | Moderate: notable but not urgent |
| 60–74 | High signal: major model releases, significant research |
| 75–89 | Front-page: landmark results, widely covered events |
| 90–100 | Historic: paradigm-shifting breakthroughs, major regulation |

Scoring factors (weights vary slightly day-to-day):
- **Editorial merit** (35%) — Claude's standalone assessment of the story
- **Cross-source agreement** (20%) — how many independent outlets covered it
- **Source strength** (10%) — originator vs. rewriter prior
- **Velocity** (10%) — cluster size relative to day's biggest cluster
- **Entity prominence** (10%) — coverage of tracked major entities
- **Topic rarity** (10%) — inverse tag frequency (rare topics score higher)
- **Recency decay** (5%) — exponential decay, half-life 18h

---

## 7. Category Taxonomy

| Slug | Description |
|---|---|
| `models` | New model releases, capabilities, benchmarks |
| `research` | Papers, findings, safety research, alignment |
| `business` | Funding, M&A, earnings, policy, regulation, infrastructure |
| `tech` | Developer tools, hardware, cloud platforms |
| `world` | International AI news, geopolitics, non-US coverage |

---

## 8. Public JSON API (`/api/v1/` and `/th/api/v1/`)

Pre-baked, CORS-enabled, static. `generated_at` is derived from the latest
archive input, not from build time, so an unchanged archive yields
byte-identical output.

### English (`/api/v1/`)

| Endpoint | Contents |
|---|---|
| `index.json` | Manifest of every endpoint + aggregate counts |
| `entities.json` | Every recurring entity (slim record) |
| `entities/<slug>.json` | Per-entity detail with full timeline |
| `events.json` | Confirmed, evidence-backed entity events |
| `relationships.json` | The whole typed knowledge graph (see below) |

### Thai (`/th/api/v1/`)

| Endpoint | Contents |
|---|---|
| `index.json` | Manifest of the Thai endpoints |
| `entities.json` | Every Thai entity with a public dossier (slim record) |
| `entities/<slug>.json` | Per-entity detail (fields below) |

### Thai per-entity fields

| Field | Meaning |
|---|---|
| `mention_count` | Stories naming this entity across the archive |
| `windows` | `{d7, d30, all}` — trailing counts. `all == mention_count`. Anchored on the newest day in the archive, not the wall clock, so these match the numbers on `/th/insight` exactly |
| `daily[]` | 90 contiguous days of `{date, n}`, zero-filled |
| `outlets[]` | `{source, n, share, index}`, most-covered first |
| `co_occurring[]` | Top entities named in the same stories |
| `declared[]` | Typed knowledge-graph edges touching this entity |
| `events[]` | Confirmed events naming this entity |
| `spikes[]` | Detected coverage spikes with the day's top story |

### What the numbers do and do not claim

Three of these fields are routinely misread, so they are stated plainly:

- **`declared[]` is an assertion. `co_occurring[]` is not.** A declared edge
  is a curated, typed claim carrying a validity range and, where available,
  an evidence URL. A co-occurrence means only that two entities were named in
  the same story. Never render the second as a relationship.
- **`outlets[].index` is publication volume, not editorial intent.** It is
  the outlet's share of this entity's coverage divided by that outlet's share
  of all coverage in the same window. `1.0` means exactly in proportion.
  `null` means the sample was too small to report a figure — render it as
  "—", never as a number, and never describe the index as bias or slant.
- **`spikes[]` is share-based, not count-based.** A spike is a day when the
  entity's share of coverage exceeded its own 28-day trailing share by the
  configured multiple, from at least 3 distinct outlets. Raw counts would
  make every entity "spike" on a day when the archive itself was unusually
  busy, such as a backfill.

There is deliberately **no sentiment or tone field**. KHAO does not score
Thai coverage as positive or negative; mention counts, outlet breadth and
disproportion are reported instead.

### `relationships.json`

The knowledge graph is language-agnostic (shared EN + TH slug space) and
ships once, from `/api/v1/relationships.json`.

| Field | Meaning |
|---|---|
| `subj` / `pred` / `obj` | Subject slug, predicate, object slug |
| `dir` | `directed` or `symmetric` |
| `valid_from` / `valid_to` | ISO `YYYY[-MM[-DD]]`. `valid_to: null` = current |
| `evidence[]` | `{url, date}` source citations, where available |
| `source` | `curated` (human) — mined edges do not ship |
| `confidence` | 0–1 |

Predicates come from a closed 16-term vocabulary listed in the file's
`predicates` array. **Dates keep their source precision**: a `valid_from` of
`"2025"` means the year is all that is known — it is not January 1st. Not
every edge carries evidence; those that do not are curated registry
structure, and the site renders them under a distinct provenance tier.

---

## Questions / API access

Contact: contact@khao.ai

See also: [llms.txt](/llms.txt) for AI crawler guidance.