Methodology
“public-community experience index, not a benchmark”
What this index measures
The experience score is an observational index of how users in public communities describe their experience with each AI model family — positive vs negative experience signals extracted from real comments. It is not a capability benchmark and says nothing about what a model can or cannot do.
Data sources
We continuously analyze public comments from Reddit, Hacker News, and Chinese communities (Zhihu and others). Each comment is classified by an LLM pipeline into experience signals per model and per dimension (coding, reasoning, safety refusals, speed…). The global score pools every valid opinion equally; platform is retained only for source breakdowns and audit.
Windows and thresholds
The main score uses a 7-day rolling window; the pulse uses 24 hours. Model families or dimensions below the minimum sample threshold are shown as “not enough data” instead of a score. Every number on this site carries its sample size.
Version-level scores
A version score uses only comments that explicitly name that version. Generic mentions such as “ChatGPT” or “Claude” stay in the family score, so family sample sizes are larger. Versions below the minimum threshold show mention counts without a score.
Your direct reports
The one-tap report widget feeds a separate first-party signal. We deduplicate by hashed IP per model per day, never store raw IP addresses, and only display aggregate counts. Direct reports are cross-checked against community signals to detect manipulation; they do not enter the experience score itself.
Honesty rules
Changes are reported as user-perceived experience, never as claims about model capability. Anomaly annotations distinguish strong, weak, or no match with public events. When we don't know, we say so.