# Different models, different roles

Author: Daniel Concepcion
Published: 2026-08-22
Canonical: https://danielconcepcion.com/thinking/different-models-different-roles/

> Separating execution and review improves focus; using a different model may add useful difference, but task-specific evaluations decide which model belongs in each role.

One spec in my personal agentic AI project passed 4 rounds of AI review and was approved for implementation. Then a model from another provider found 24 more issues in it.

That left me with a question I couldn't answer: why had the earlier reviews missed them?

I mentioned this in my first post: recurrent issues had survived one model doing the work and another reviewing it.

At first I thought splitting the roles was the important change: one agent to execute, another to challenge the result against the evidence.

- That separated the jobs. Model choice was still its own decision.

- The same model can do two different jobs when the instructions and acceptance criteria change. A different model may bring different strengths and blind spots.

So I looked for research that might explain part of it.

- A recent study found self-preference even with verifiable rubrics. Some judges were more than 50% more likely to mark a failed criterion as satisfied when reviewing their own output.

- An Apple study counted both what review fixed and what it broke. Results changed with the reviewer model and prompt. Its GPT-4o and o3-mini comparison is already dated.

I didn't find a definitive answer. The studies gave me hypotheses. (I left their links in the first comment)

For this project, the multi-provider setup is now part of the foundation. Reviews have been better when the executor and reviewer use models from different providers. That is still my current feeling. I haven't run a controlled comparison yet. But I can change the model behind a role without rebuilding the workflow.

In the day-to-day work on this personal project, I use several coding agents: Claude Code, Codex and Pi, with OpenRouter as a gateway to open-weight models such as DeepSeek, GLM and Kimi. I follow the same pattern by passing work between models.

My next step is to test that observation with data. I need evaluations built from the work I actually do. Coding and research may need different model pairings and evidence.

I expect model rankings to move as versions change. I want the role to stay stable while I test and replace the model behind it.

How are you deciding which model gets which role?

#AgenticAI #LLMEvaluation #ModelSelection #LearningAIByUsingAI

## Sources and links

- [Self-Preference Bias in Rubric-Based Evaluation of Large Language Models](https://arxiv.org/abs/2604.06996)
- [Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents](https://aclanthology.org/2026.gem-main.13/)
- [An Empirical Study of LLM-as-a-Judge](https://aclanthology.org/2026.gem-main.19/)
- [All Prompts Are Created Equal?](https://aclanthology.org/2026.findings-acl.1929/)


## Companies, products and research

- [Claude Code](https://www.anthropic.com/claude-code)
- [Codex](https://openai.com/codex/)
- [OpenRouter](https://openrouter.ai/)


## More Thinking

- Newer: [After the laptop closes: building a persistent AI agent](https://danielconcepcion.com/thinking/persistent-ai-agent/)
- Older: [Where AI lives](https://danielconcepcion.com/thinking/where-ai-lives/)