Here's the one-line finding: we ran 756 API calls across 7 frontier AI models, asked them 18 core doctrinal questions, and not one model scored above 45.1 out of 100. Every single model — GPT, Gemini, Claude, Kimi, GLM — landed in the tier we call "Faith-Distancing." None reached "Broadly Orthodox." This is an AI theology benchmark built from our own data, not someone else's summary.
I wrote about the Gospel Coalition and Gloo studies in my earlier analysis. Both are worth reading. But I kept running into the same limitation: I couldn't see the raw data. So I built my own benchmark, ran it, and I'm publishing the whole thing.
Why I ran my own benchmark
The Gospel Coalition's AI Christian Benchmark is genuinely useful — seven respected theologians graded seven AI platforms on seven questions, using the Nicene Creed as the standard. But it's small-n and qualitative. Seven questions is enough to spot a pattern, not enough to map where a model fails and where it doesn't.
Gloo's FAI-C benchmark goes wider — 807 questions — but the methodology behind the scoring isn't public. I can tell you Gloo found faith questions scoring 49% against 83% for finance questions. I can't tell you exactly how they got there, and neither can you.
I wanted something reproducible: a fixed set of questions, a fixed rubric, a documented model list, and raw scores anyone could audit. So I built the pipeline myself and ran it against Assemblies of God doctrine — the tradition I served in for eight years as a pastor before I became an AI engineer.
Methodology
Here's exactly what we did:
- Selected 18 questions spanning the 16 AG Fundamental Truths plus the Gospel Coalition's own question set (included for cross-comparison), covering topics from the nature of God to what happens after death.
- Tested 7 models: GPT-5.2 Pro, GPT-5.2, Gemini 3.1 Pro, Kimi K2.5, GLM-5, Claude Sonnet 4.6, and Claude Opus 4.6.
- Ran each model-question pair 3 times to control for response variance — 7 models × 18 questions × 3 runs = 378 answer-generation calls.
- Scored every answer with an independent judge model against AG doctrine as the reference standard — another 378 calls.
- Totaled 756 API calls, split evenly between generating answers and grading them.
- Classified every score into four tiers: AG Aligned (80-100), Broadly Orthodox (65-79), Partially Accurate (45-64), and Unreliable/Faith-Distancing (0-44).
No model was prompted with denominational context, a system prompt establishing a faith posture, or any retrieval augmentation. This is what a person gets when they open ChatGPT, Gemini, or Claude cold and ask a theological question — which is exactly what most people do.
The results
| Rank | Model | Overall Score | Tier |
|---|---|---|---|
| 1 | GPT-5.2 Pro | 45.1 | Partially Accurate |
| 2 | Gemini 3.1 Pro | 45.0 | Partially Accurate |
| 3 | GPT-5.2 | 44.9 | Unreliable |
| 4 | Kimi K2.5 | 43.4 | Unreliable |
| 5 | GLM-5 | 42.3 | Unreliable |
| 6 | Claude Sonnet 4.6 | 34.1 | Unreliable |
| 7 | Claude Opus 4.6 | 27.9 | Unreliable |
The average across all seven models was 40.4/100. Zero models reached "Broadly Orthodox." Zero reached "AG Aligned." Every single one landed in a tier we call Faith-Distancing on at least a meaningful share of the questions.
Breaking it down by question rather than by model tells a sharper story:
| Question | Avg Score | Verdict |
|---|---|---|
| What is the Gospel? | 82.9 | The one bright spot — genuinely useful |
| How do Christians become holy? | 64.5 | Near orthodox, needs denominational correction |
| Was Jesus a real person? | 57.1 | Historical scholarship is well-represented in training data |
| What is the purpose of the church? | 55.8 | Reasonable coverage, hedges on mission |
| What is being filled with the Holy Spirit? | 50.6 | Half credit on a doctrine that defines Pentecostal identity |
| Who is God? | 17.8 | Trinity dissolved into multi-faith pluralism |
| What happens after you die? | 4.4 | Near-total failure — afterlife as a buffet of opinions |
What "faith-distancing" actually looks like
The number that should stop you is 4.4 out of 100 on "What happens after you die?" That's not a low score from a hard question — it's a model consistently refusing to affirm any biblical claim about judgment, presenting eternity as an open question with no defensible answer.
The pattern shows up again on "Who is God?" (17.8/100). Rather than describing the Trinity as historic Christian doctrine, models routinely flatten it into "one of many ways people conceive of God," treating a defining article of the faith as a cultural preference.
I saw the same mechanism up close when I tested raw AI models against evidence-grounded answers for the same 18 questions. Where a raw model would write that AG's teaching on the Spirit is "one interpretation among several" or that healing today is "debated among believers," an evidence-grounded answer stated the same doctrine as a conviction the tradition actually holds — no hedge, no false balance. One transcript I reviewed even had a raw model explicitly reframe a denomination's own doctrinal statement as "a perspective," softening a specific, confident teaching into something optional. That's the faith-distancing pattern in miniature: not hostility to Christianity, just a reflexive retreat to "there are many views" whenever the topic turns doctrinal.
The best-performing question, "What is the Gospel?" at 82.9/100, is the exception that proves the rule. It's the most-discussed, best-documented topic in Christian writing anywhere on the internet, so models reproduce it well. The moment a question requires denominational precision — Spirit baptism, the nature of God, final judgment — the average collapses.
Why models fail here
This isn't a defect anyone can patch with a better prompt. It's structural.
Training data is the average of the internet, and the internet's writing about theology is dominated by comparative religion, academic religious studies, and secular commentary — genres that describe belief rather than hold it. A model trained on that mix learns to talk about Christian doctrine, not from inside it.
RLHF (reinforcement learning from human feedback) rewards inoffensiveness. Human raters tend to prefer answers that avoid taking a side on contested topics, and "God's nature" or "what happens after death" reads as contested to anyone outside a faith tradition. The model gets trained, call after call, to hedge exactly where a pastor would want conviction.
Put those two forces together and you get exactly what the benchmark shows: strong performance on well-documented, low-controversy content (the Gospel), and collapse on anything doctrinally specific or eternally consequential (final judgment, the Trinity, Spirit baptism).
What this means for your church
If people in your congregation are asking AI about doctrine — and the earlier research suggests they are, often at 11 p.m., often about something they're too afraid to ask you — they are getting answers graded, on average, 40.4 out of 100 against their own tradition's teaching. That's not a neutral tool giving a slightly imperfect answer. That's a tool that will, on the hardest and most consequential questions, actively steer them away from what the church actually teaches.
This is why I built OpenLumin: a tool that grounds AI answers in a denomination's actual doctrinal statements and Scripture, rather than the statistical average of the internet. Applying that evidence-first architecture to the same 18 questions and the same judge model moved the AG-specific benchmark from 40.4 to the mid-60s to low-70s range across repeated real-workflow runs — with zero faith-distancing responses. It isn't perfect, and I'm not claiming it is. But the gap between raw AI and evidence-grounded AI is the whole point: the failure mode is fixable when you build for it deliberately, and it stays broken when you don't.
Your church doesn't need to take my word, TGC's word, or Gloo's word for any of this. You need to test the specific tools your staff and members are actually using, against your specific doctrine, and see where they fail.
Get your AI tools evaluated
If you want your church's AI tools — chatbots, sermon assistants, whatever staff or members are already using — benchmarked against your own doctrinal standards the way we benchmarked these seven models against AG doctrine, that's exactly the kind of engagement I take on through the Navigate track.
Or if you just want to talk through what this means for your context, reach out. I'd rather you find out where your tools fail from a benchmark than from a member who took a wrong answer as gospel.