User Model: Always-On Preference Enforcement

The problem

In the distill-benchmark competitive analysis, aura-distill scored 3.17/5 on user model — placing 5th out of 7 systems. This was the lowest-performing category, dragging down overall results despite strong showings in knowledge retrieval and bias resistance.

Four scenarios exposed specific failure modes:

Benchmark results

Full 7-system comparison on the user model category:

The 1.16-point gap between knowledge-graph (1st) and aura-distill (5th) was the largest category deficit in the benchmark. On overall scores, aura-distill placed competitively — but user model was the clear drag.

Root cause analysis

The failure decomposed into three layers:

1. Load-time gap

User preferences in aura-distill were stored in profile files behind a lazy-load gate. The SPINE had to match a domain keyword, trigger a file read, and then the model had to attend to the preference. For U3 (concise answers) and U4 (fish shell), there was no domain keyword to trigger loading — these are cross-cutting preferences, not domain-specific knowledge.

2. Application gap

Even when preferences loaded, they were stated as descriptive facts: user prefers concise answers. This phrasing invites the model to weigh it against its own tendencies. The winning system used enforcement language: give concise answers, no preamble.

3. Coverage gap

The user profile was thin on output-format preferences. Shell preference, verbosity calibration, and expertise level were either missing or buried in domain-specific files where cross-cutting scenarios couldn't find them.

Academic context

The user model problem is well-studied in recent LLM personalization research. Seven papers provide direct context for the always-on preference mechanism:

AlpsBench (Li et al., 2025) benchmarks models on latent user traits — preferences that must be inferred from behavior rather than stated explicitly. Models struggle most when traits conflict with default behavior, exactly the failure mode in U3 and U4.

PersistBench (Kovac et al., 2025) finds a 97% failure rate on memory-induced sycophancy: when a memory says "user prefers X" but X is wrong for the current context, models comply with the memory rather than pushing back. This informed the contradiction-checking mechanism in always-on preferences.

PLUS (Chen et al., 2025) demonstrates that text summaries outperform embedding-based retrieval for user preference matching. This validates the always-on approach: a concise text block in the system prompt beats a vector-search pipeline for preference recall.

Memory as Metabolism (Xu et al., 2025) distinguishes two types of personalization: mirroring style (match how the user communicates) and compensating substance (fill gaps in what they know). Always-on preferences handle style; domain knowledge handles substance.

ProfiLLM (Gupta et al., 2025) shows a 55-65% gap reduction in personalization after a single profile-enriched prompt. The effect is immediate and does not require multi-turn interaction. This confirms that always-on inlining is sufficient — no iterative refinement needed.

STALE (Jang et al., 2025) achieves only 55.2% accuracy on detecting stale memories — nearly a coin flip. This motivated the confidence lifecycle in always-on preferences: only validated or hardened preferences get promoted to always-on, reducing the risk of stale enforcement.

SteeM (Hu et al., 2025) introduces user-controlled memory reliance, letting users decide how much the model should depend on stored preferences vs. in-context signals. The always-on compliance checklist draws on this principle: users can override any always-on preference in-session.

The fix

Always-On User Preferences — a max 15-line section added to rules/distill.md that loads every session with zero retrieval cost.

Design principles:

1. Inlined, not retrieved. Preferences live in the rules file, not behind a SPINE trigger. They are visible to the model before any user message.
2. Enforcement language. "Give concise answers" not "user prefers concise answers." Imperative phrasing reduces model hedging.
3. Auto-synced. The /distill command evaluates profile data and writes the always-on section automatically. No manual curation required.

Example always-on section:

# Always-On User Preferences (auto-synced by /distill)
# Do not edit manually — will be overwritten on next distillation.

- Use fish shell syntax in all shell examples. Never label as bash.
- Give concise answers. No preamble, no "Great question!", no restating.
- Calibrate to senior engineer (15yr). Skip tutorials, explain trade-offs.
- Commit messages: imperative mood, no period, max 72 chars.
- TypeScript strict mode. No `any` unless explicitly allowed.
- Prefer composition over inheritance.
- Test names: describe behavior, not implementation.

## Compliance checklist
1. Before responding: re-read the preferences above.
2. If a preference conflicts with the user's current request, follow the request.
3. If uncertain whether a preference applies, apply it — the user will correct.

How it works

Preference lifecycle

Not every preference reaches always-on. The confidence ladder filters noise:

Only validated and hardened preferences are promoted to the always-on section. This prevents one-off corrections from becoming permanent enforcement rules.

Sync process

When /distill runs, it:

1. Reads the current user profile and all session signals
2. Evaluates each preference against the confidence ladder
3. Ranks validated/hardened preferences by impact (cross-cutting preferences score higher)
4. Formats the top preferences into imperative rules
5. Writes the always-on section to rules/distill.md, replacing the previous version

The 15-line limit forces prioritization. If a user has 30 hardened preferences, only the 15 most cross-cutting survive. Domain-specific preferences stay in their domain files.

Contradiction detection

Every distillation checks for contradictions between:

• New session signals and existing always-on preferences
• Always-on preferences and domain-specific knowledge
• User behavior and stated preferences (say-do gap)

When a contradiction is detected on a hardened preference, it triggers a paradigm alarm rather than a silent update — because something the user confirmed 5+ times is now being contradicted.

References

1. Li, Z. et al. (2025). AlpsBench: Benchmarking LLM Alignment with Latent Personality and Social Traits. arXiv: 2603.26680
2. Kovac, G. et al. (2025). PersistBench: Evaluating Memory-Induced Sycophancy in Conversational AI. arXiv: 2602.01146
3. Chen, Y. et al. (2025). PLUS: Personalized LLM User Profiling via Text Summaries. arXiv: 2507.13579
4. Xu, H. et al. (2025). Memory as Metabolism: Mirroring Style, Compensating Substance. arXiv: 2604.12034
5. Gupta, A. et al. (2025). ProfiLLM: Single-Prompt Personalization via User Profiles. arXiv: 2506.13980
6. Jang, J. et al. (2025). STALE: Detecting Stale Memories in Long-Term LLM Interactions. arXiv: 2605.06527
7. Hu, S. et al. (2025). SteeM: Steerable Memory Reliance in Conversational Agents. arXiv: 2601.05107