Benchmarks
Measured speed, prompt caching and MMLU-Pro accuracy for Janus 1.0 Flash and Janus 1.0 Pro, Kaer's proprietary models on the Kaer API. Every figure is generated from per-request records taken on api.kaer.ai's production serving path at default settings.
Measured on 20–21 November 2026 (UTC).
How fast are Janus 1.0 Flash and Janus 1.0 Pro?
At the median, Janus 1.0 Flash returned its first token after 1.02 s and then streamed 52.4 tokens a second. Janus 1.0 Pro took 1.49 s and streamed 44.9 tokens a second.
| Measure | Janus 1.0 Flash | Janus 1.0 Pro |
|---|---|---|
| Time to first tokenMedian | 1.02 s | 1.49 s |
| Time to first token90th percentile | 1.62 s | 2.92 s |
| Output speedMedian, tokens per second after the first token | 52.4 | 44.9 |
| End-to-end latencyMedian | 6.04 s | 7.19 s |
| End-to-end latency90th percentile | 13.50 s | 23.93 s |
| Output lengthMedian, tokens | 273 | 330 |
| ErrorsFailed or broken streams | 0 of 400.0% | 0 of 400.0% |
40 streamed requests per model, one in flight per model: 12 short answers, 12 explanations of about 250 words, 8 long-form pieces and 8 tasks over pasted documents.
Prompts of 165 to 1,488 tokens; answers of 23 to 872 tokens.
Measured 20 November 2026, 23:32–23:40 UTC.
How much of a repeated prompt is served from cache?
When an identical long prompt was sent again, 92.6% of Janus 1.0 Flash's prompt tokens and 95.9% of Janus 1.0 Pro's were read from cache. None were cached on the first call with a new prompt.
| Measure | Janus 1.0 Flash | Janus 1.0 Pro |
|---|---|---|
| Prompt sizeMedian, tokens | 3,594 | 3,604 |
| Cached on the first callShare of prompt tokens | 0.0% | 0.0% |
| Cached on repeat callsShare of prompt tokens | 92.6%92.4% to 92.7% per call | 95.9%92.4% to 99.4% per call |
| Time to first tokenFirst call, median | 1.54 s | 1.86 s |
| Time to first tokenRepeat calls, median | 1.41 s | 1.58 s |
| CallsPrompts × calls each | 2 × 60 errors | 2 × 60 errors |
Each prompt was a system message holding meeting notes, a policy, an email thread and a product catalogue, opened by a unique ID so the first call could not reuse an earlier cache. Every call then asked a different one-sentence question.
Cached share is usage.prompt_tokens_details.cached_tokens over usage.prompt_tokens. Cached input is billed at a lower rate than uncached input: API pricing.
Measured 20 November 2026, 23:40:16–23:40:50 UTC.
How accurate are they on MMLU-Pro?
On 700 MMLU-Pro questions, 50 from each of 14 subjects, Janus 1.0 Flash answered 85.7% correctly and Janus 1.0 Pro 85.1%. MMLU-Pro asks multiple-choice questions with up to ten options across academic and professional subjects.
| Measure | Janus 1.0 Flash | Janus 1.0 Pro |
|---|---|---|
| All subjects700 questions | 85.7%95% interval 82.9 to 88.1 | 85.1%95% interval 82.3 to 87.6 |
| Biology | 98.0% | 98.0% |
| Business | 98.0% | 98.0% |
| Chemistry | 88.0% | 84.0% |
| Computer science | 88.0% | 88.0% |
| Economics | 94.0% | 92.0% |
| Engineering | 80.0% | 84.0% |
| Health | 72.0% | 76.0% |
| History | 80.0% | 74.0% |
| Law | 74.0% | 70.0% |
| Math | 92.0% | 92.0% |
| Other | 76.0% | 78.0% |
| Philosophy | 84.0% | 78.0% |
| Physics | 88.0% | 92.0% |
| Psychology | 88.0% | 88.0% |
| No answer foundScored wrong | 0 | 0 |
| ErrorsFailed after retries | 0 | 0 |
TIGER-Lab/MMLU-Pro, test split, 5-shot chain-of-thought prompt, one answer per question at the API's default settings. Per-subject figures rest on 50 questions each, so they move more than the overall score.
Measured 20 November 2026, 23:40 UTC to 21 November 2026, 00:04 UTC.
How were these numbers measured?
Where. Every request went through api.kaer.ai's production serving path, the models and settings the public API serves today. The calls ran inside Kaer's network on an internal route that skips the billing gateway, so no customer was billed. Times therefore leave out the internet round trip between your client and api.kaer.ai.
When. Performance: 20 November 2026, 23:32–23:40 UTC. Prompt caching: 20 November 2026, 23:40:16–23:40:50 UTC. MMLU-Pro: 20 November 2026, 23:40 UTC to 21 November 2026, 00:04 UTC. No figure comes from an earlier run.
Settings. Each request body held only model, messages, stream: true and a request ID. Every other field, temperature, max_tokens and reasoning_effort included, kept its API default, listed in the API reference.
Speed. Time to first token runs from sending the request to the first streamed content. The API streams content in chunks of several tokens, so this is the time to the first chunk. Output speed is a request's completion tokens, from its usage block, divided by the time from its first to its last streamed chunk. End-to-end latency runs to the end of the stream. Medians and 90th percentiles interpolate linearly between the ordered per-request values. Prompts asked for free prose: explanations, letters, stories and summaries. None asked for predictable text such as counting, which streams faster than real answers.
Caching. Per model, 2 new prompts, each sent 6 times in a row a few seconds apart. The first call with each prompt counts as the first call; the rest are repeat calls.
MMLU-Pro. TIGER-Lab/MMLU-Pro, test split, revision b189ec7. The questions are the first 50 of each subject in a fixed random selection (seed 20261005), which is itself a uniform stratified sample. Each question used the benchmark's official 5-shot chain-of-thought prompt as one user message. Answers were graded by the official extraction rules; a response with no extractable letter counts as wrong. Intervals are 95% Wilson score intervals. The benchmark's own script samples at temperature 0; these runs used the API default. A request the API refused or broke off was retried up to three times; 4 of 1,400 needed a retry, and every question has an answer on record.
Scope. Kaer has not measured other vendors' models this way, so this page makes no comparisons. A script generates every figure here from the stored per-request records; none is typed by hand.