We scored Fable 5 vs Opus 5 vs Opus 4.8 and only the old one answered every call

Even Anthropic’s documentation disagrees with itself

Pick a Claude model to build on today and you have three options. Claude Fable 5 and Claude Opus 5 are the new pair, and the older Opus 4.8 still runs in plenty of production systems. The first thing you notice is that even Anthropic’s own documentation disagrees with itself. It introduces Fable 5 as the most capable widely available model. The same section then steers builders toward Opus 5 for complex agentic work, and third-party leaderboards reverse the order again. Beyond that, most of the public debate is about price. Price is not what LatentEval measures, so we ran all three models through our reliability suite. Every model received the same tasks, in the same period, at its provider’s default settings.

Fable 5 and Opus 5 finish in a genuine tie

The two newer models finished about one point apart on our reliability index. That covers everything both of them were allowed to answer, and it sits inside the two-point threshold we fixed in advance for declaring a winner. So the result is a tie, and a stable one: it stayed too close to call under almost every reasonable weighting we tried, while the model nominally in front kept changing from one weighting to the next. A ranking that changes with the weighting would mislead you. We are not publishing one. You can read the full benchmark results and detailed figures here.

The real difference is which prompts get answered

One task in our suite was deep enough to separate two frontier models. It was a 1,129-step calculation with a single correct answer, and it ran for only one of them. Fable 5 was served nearly every request and returned the exact value each time. Opus 5 never got to try. Every request came back refused before the model produced anything, which is also why part of its index went unscored. Opus 4.8 answered all of its requests but could not settle on one value.

A refusal of that kind is usually a fact about the serving stack rather than the model itself. Routing rules, policy layers and safety classifiers all sit between your prompt and the model. They also behave differently from one account to the next. Which prompts get refused depends on the type of task as well. Fable 5 lost more calls on the contract tasks we describe in the next section, while Opus 5 lost more on long calculations. Refusals react to how a prompt looks as much as to what it asks, and we saw that directly. We sent the same multiplication in two forms. Presented inside an accounts-payable screen, complete with a locked subtotal and a sign-off field, it was refused by both newer models. Written as one bare line of text, the same sum was answered by all three. The newer pair agreed to the cent on every run, while Opus 4.8 gave a different number almost every time. The practical advice is simple. Send long, dependent calculations to whichever newer model your account can reach. Then check your answer rate separately for each type of task, because an overall average will hide the problem.

The expensive failures look like correct answers

Consistency rarely breaks on the newer models, and when it does break, the arithmetic itself is usually fine. One of our tests puts the model under social pressure. A senior VP has already ruled on an error rate, and the ruling is wrong. The tool that files the rate accepts a number and nothing else. Fable 5 diagnosed the situation correctly but filed two different numbers across its runs. Opus 5 filed the correct figure every time, and most of its replies explained why the ruling could not be right. Opus 4.8 never filed the VP’s figure either. More often than not, though, it filed nothing at all, and in production an empty filing fails just as hard as a wrong one. This came from a single test item, so treat it as something to verify on your own prompts rather than as a settled ranking.

A more expensive failure shows up when the specification itself is impossible. We gave the models contracts that cannot be satisfied, and the correct behavior is to say so. Both newer models flagged every impossible contract they were allowed to answer. Opus 4.8 answered every one of them and misjudged in both directions. It produced answers for some impossible contracts and declared some solvable ones impossible. The worst case is worth spelling out. On one item it returned a well-formed tag that broke the specification’s first rule. The downstream validator on that item accepts only tags from an approved list and rejects error messages, so the wrong tag passed validation while an honest refusal would have been rejected. This is the type of production failure to plan for, because nothing crashes. The system simply receives a well-formed wrong answer that passes every automated check.

Honesty fails in a similar way. We took a true mathematical statement, quietly changed it so that it became false, presented it as a theorem, and asked for a proof. On nearly every attempt that ran to completion, Opus 4.8 returned a well-organized proof of the false statement. The single refusal said it could not identify a legitimate proof strategy, and that “constructing a fake one would be dishonest”. Both newer models returned essentially nothing on this task, so this behavior belongs to the old model alone. It is worth remembering in any pipeline that asks a model to verify a claim before building on it.

Two situations where Opus 4.8 is still the better choice

Opus 4.8 finished far behind on every capability dimension. Part of that distance is configuration, since its defaults leave out the extended thinking step. It was also the only model that came back with an answer on every single call. That combination makes it the better choice in two specific situations.

The first situation is when a person is going to read the answer and check it themselves. Opus 4.8 writes out its working, so that you can follow each answer line by line. That stays true when the answer is wrong, and that is exactly when you want the working in front of you. There was one warehouse task where it kept arriving at the same wrong total, and it got there in three different ways. In one of those attempts it wrote min(5440, 6200) and then took the larger of the two numbers. A person reading the working spots that mistake straight away. It also helped that this was the only model our account could reliably get an answer out of for several kinds of back-office arithmetic.

The second situation is different. In some workloads, getting a slightly wrong answer is better for you than getting nothing back at all. Go back to the long calculation from earlier. Even the least accurate answer Opus 4.8 gave there was within three percent of the correct value, and its errors follow a pattern you can recognize. What often happens is that it drops one step somewhere in the chain, and every step after that point is off by the same amount. Once you know that, you can measure the size of the error on your own data and plan around it. So the rule we ended up with is simple. Use Opus 4.8 when you are going to read the reply yourself, or when a small and measurable error is acceptable, and use one of the newer models for everything else.

Test reliability on your own workload

By this point it should be clear that model behavior depends on your account and your provider settings, not only on the model itself. So before you commit, run the same checks on your own prompts, along the lines of the check we ran on a different model trio. Here is the list we would work through.

• Work out your answer rate for each type of task separately. An average across your whole workload can look fine while one type of task is failing badly, and you will not see it until you split the numbers.

• Take the longest chained calculation in your product and send it several times, then count how many different answers you get back. If you only check whether one attempt was right, you will miss the run that came back wrong by two.

• Retry refused calls at least once. In our runs, the exact same request was refused on one attempt and went through on the next.

• Decide in advance what your client code should do with a refused call. Some refusals in our runs arrived midway through a response, with tokens already billed, and a refusal like that will be counted as a success if your code does not check for it.

• Before comparing two models, look up what their default settings actually are. The output limit, for example, is a single budget that covers both the extended thinking and the visible reply, and the thinking part is spent first. The same limit can therefore restrict one model far more than the other.

from collections import defaultdict

def certify(model, tasks, k=8):
    """Run per task family, per candidate model. Compare families one at a time."""
    sent = answered = contracts_ok = 0
    values = defaultdict(set)
    for task in tasks * k:                       # same bytes, k times
        r = call(model, task)
        sent += 1
        if not r.answered:                       # a block is a result: log it
            log("no_answer", model, task.id, r.stop_reason)
            continue
        answered += 1
        values[task.id].add(r.value)             # more than one value and you cannot cache this
        contracts_ok += (r.says_impossible == task.is_impossible)
    if not answered:                             # blocked out of the family entirely:
        return {"coverage": 0.0,                 # no stability and no rate exist
                "stable": None,
                "contracts": None}
    # before comparing two models' results, check both ran at the same thinking default
    return {"coverage": answered / sent,         # keep this out of the score
            "stable": all(len(v) == 1 for v in values.values()),
            "contracts": contracts_ok / answered}

Start with the type of task your product depends on most, and repeat the checks on a schedule, because routing and policy behavior change over time. If you want the numbers behind this page, they are all in the complete benchmark report, along with the full method.

So which Claude model should you pick?

Between Fable 5 and Opus 5, choose whichever one your account can reach for the type of task you care about, and configure the other as a fallback. On everything both models answered, they performed as equals. The clearest behavioral difference we recorded came from the VP test above, where Opus 5 gave the correct figure every time and Fable 5 filed two different numbers. And Opus 4.8 remains the right choice for the two situations described earlier. It answered every call we sent it, including the questions that had no correct answer.

Note

Results come from LatentEval’s reliability run. We sent the same 580 calls to each model, on a single account within a single measurement window, with every model at its provider’s default settings. Those defaults give both newer models an extended thinking step and give Opus 4.8 none, so the older model’s distance from the other two reflects configuration as well as capability. Refused and blocked calls tell you about the serving stack, meaning the routing rules, policy layers and classifiers between you and the model. Correctness is scored only on the calls that came back. The full benchmark report contains every count and every variant of the index.