A Synthetic Benchmark for Font Family Identification
Words in known fonts, printed or painted on real surfaces and in real scenes, with the exact font, the text and the position of every letter.
GitHub1,059 sans-serif, 1,000 serif, 500 slab serif, 300 script and 141 monospaced, from Adobe Fonts, Creative Fabrica, Google Fonts and DaFont (commercial-use licenses only).
20 per font, with the same three types and three levels as v1.0: a text printed on a close-up surface, painted in a real scene, or on a printed object.
The length and the case of every text follow what people type when they search on WhatFontIs: from a single letter to a whole word, mostly in capitals. Letter-level annotations as in v1.0.
In v1.0 every image holds one word of 7-12 letters. Real searches are shorter: a logo, a few capitals, a single letter. In v2.0 the length and the case of the text follow the statistics of real searches on WhatFontIs, so the set measures the case that matters most in practice: identifying a font from only a few letters.
| Type | Easy | Medium | Hard | Per font | In the set |
|---|---|---|---|---|---|
| texture: the text printed or painted straight on a close-up photo of a real surface (wood, wall, concrete, brick, metal, fabric, leather, paper, cardboard) | 3 | 2 | 5 | 10 | 30,000 |
| scene: the text painted inside a real photo of a place | 1 | 2 | 3 | 6 | 18,000 |
| object: the text on a printed object (label, box, book cover, poster, menu, shop sign, packaging, business card, painted wall) | 1 | 1 | 2 | 4 | 12,000 |
As in v1.0, the images are frontal; texture and scene images add only a small camera deviation (corners moved at most 1.5 %, rotation at most 1°). Text contrast is at least 70 grey levels (50 on the hard level), no letter is clipped, and the long side is never over 2000 px. The texts are strings from COCO-Text v2, with their case changed to match the search statistics.
The WhatFontIs API on all 60,000 images of v2.0, October 2026, in its two modes: the default one and mode=pro. As for v1.0, the API searches its whole catalogue of over 1.2 million fonts, and a result is correct when it is the right font family, in any weight or from any source.
Top-1 font family accuracy: 88.1 % with mode=pro, 82.2 % in the default mode.
| Default mode | mode=pro | ||||||
|---|---|---|---|---|---|---|---|
| Images | Correct at 1 | In top 5 | In top 20 | Correct at 1 | In top 5 | In top 20 | |
| All images | 60,000 | 82.2 % | 88.0 % | 89.9 % | 88.1 % | 94.3 % | 95.6 % |
| texture | 30,000 | 82.4 % | 88.1 % | 89.9 % | 88.9 % | 95.0 % | 96.1 % |
| scene | 18,000 | 79.6 % | 85.6 % | 87.7 % | 86.0 % | 92.4 % | 94.0 % |
| object | 12,000 | 85.7 % | 91.4 % | 93.0 % | 89.2 % | 95.4 % | 96.8 % |
| easy | 15,000 | 83.0 % | 88.1 % | 89.7 % | 89.2 % | 95.1 % | 96.5 % |
| medium | 15,000 | 81.4 % | 86.9 % | 88.6 % | 87.6 % | 94.0 % | 95.3 % |
| hard | 30,000 | 82.3 % | 88.5 % | 90.6 % | 87.8 % | 94.0 % | 95.3 % |
| sans-serif | 21,180 | 76.7 % | 83.1 % | 86.0 % | 84.1 % | 90.5 % | 92.5 % |
| serif | 20,000 | 84.7 % | 90.7 % | 92.1 % | 88.9 % | 95.7 % | 96.9 % |
| slab serif | 10,000 | 86.4 % | 91.1 % | 92.0 % | 91.2 % | 97.3 % | 97.9 % |
| script | 6,000 | 86.8 % | 91.5 % | 92.3 % | 93.7 % | 97.4 % | 97.9 % |
| monospaced | 2,820 | 81.8 % | 87.5 % | 91.0 % | 89.4 % | 94.9 % | 96.8 % |
| Median time per image | 4.4 s | 9.0 s | |||||
This is where v2.0 differs from v1.0. With five or more different letters the API is at least as accurate as on the long words of v1.0. With fewer letters there is less of the font to tell it apart from similar ones: with a single letter, accuracy falls to about half.
| Default mode | mode=pro | ||||||
|---|---|---|---|---|---|---|---|
| Different letters | Images | Correct at 1 | In top 5 | In top 20 | Correct at 1 | In top 5 | In top 20 |
| 1 letter | 2,697 | 43.2 % | 53.5 % | 59.9 % | 48.2 % | 62.4 % | 69.1 % |
| 2 letters | 3,713 | 64.7 % | 73.7 % | 78.6 % | 73.8 % | 85.9 % | 89.8 % |
| 3 letters | 8,695 | 77.3 % | 83.6 % | 86.1 % | 86.2 % | 93.4 % | 95.1 % |
| 4 letters | 9,625 | 82.6 % | 88.5 % | 90.4 % | 89.6 % | 95.7 % | 96.9 % |
| 5-6 letters | 19,056 | 86.8 % | 92.0 % | 93.2 % | 92.0 % | 97.2 % | 97.8 % |
| 7 or more letters | 16,214 | 89.8 % | 94.4 % | 95.2 % | 93.5 % | 97.8 % | 98.2 % |
Images without an answer count as misses: 280 in the default mode, where no letters were found, and 50 with mode=pro (37 where the request failed, 13 where no letters were found). Times are the median per image on the live API, one request at a time, measured on 100 of the images.
Annotations, font list and tools are in the GitHub repository. The images of v2.0 are in the release v2.0, as 15 zip files:
| File | Size | Images |
|---|---|---|
| WhatFontIs-Bench_v2.0_images_part01.zip | 477 MB | 00000-04002 |
| WhatFontIs-Bench_v2.0_images_part02.zip | 525 MB | 04003-08008 |
| WhatFontIs-Bench_v2.0_images_part03.zip | 455 MB | 08009-12013 |
| WhatFontIs-Bench_v2.0_images_part04.zip | 455 MB | 12014-16019 |
| WhatFontIs-Bench_v2.0_images_part05.zip | 455 MB | 16020-20024 |
| WhatFontIs-Bench_v2.0_images_part06.zip | 481 MB | 20025-24027 |
| WhatFontIs-Bench_v2.0_images_part07.zip | 478 MB | 24028-28031 |
| WhatFontIs-Bench_v2.0_images_part08.zip | 460 MB | 28032-32038 |
| WhatFontIs-Bench_v2.0_images_part09.zip | 455 MB | 32039-36044 |
| WhatFontIs-Bench_v2.0_images_part10.zip | 433 MB | 36045-40055 |
| WhatFontIs-Bench_v2.0_images_part11.zip | 455 MB | 40056-44068 |
| WhatFontIs-Bench_v2.0_images_part12.zip | 493 MB | 44069-48081 |
| WhatFontIs-Bench_v2.0_images_part13.zip | 791 MB | 48082-52081 |
| WhatFontIs-Bench_v2.0_images_part14.zip | 795 MB | 52082-56081 |
| WhatFontIs-Bench_v2.0_images_part15.zip | 787 MB | 56082-60081 |
With the GitHub CLI:
gh release download v2.0 --repo whatfontis/WhatFontIs-Bench
Unzip all of them into v2/; they create v2/scenes/NNNNN.jpg. The text crops (the text cut out
of each image, unscaled) come from python tools/make_crops.py v2. Font files are not included.
| File | What it holds |
|---|---|
v2/labels.jsonl | one line per image, with the same fields as v1.0, plus query_spec (the number of different characters and the case the text was drawn with) |
v2/annotations.json | the same set in COCO format: images, categories (one per font), annotations (word box, text, letters) |
v2/fonts.json | the 3,000 fonts: title, family, source, category, capitals-only flag, WhatFontIs page, number of images |
v2/backgrounds.json | the background photos, with source page, author and license |
v2/text_pools.json | the strings the texts were drawn from, by case and number of different characters |
v2/quick1200.txt | 1,200 images, one font each, that give about the same score as the whole set: a quick first check for a new model |
tools/make_crops.py | cuts the text crops out of the images |
200 sans-serif, 200 serif, 100 slab serif and 100 monospaced, from Adobe Fonts, Creative Fabrica, Google Fonts and DaFont (commercial-use licenses only).
20 per font: a word printed on a close-up surface, painted in a real scene, or on a printed object - in three levels, easy, medium and hard.
For every image: the exact font, the text, the word box and the four corners of every letter, plus how it was made (size, blur, noise, light). Also in COCO format.
20 example images from 20 different fonts, over all types, levels and categories, scaled to 1200 px. Green: the word; blue: each letter. All 11,995 images, full size, are in the release.
Ten sheets show all 600 fonts of v1.0, 60 per sheet. Fonts that only have capitals (107) are written in capitals.
| Font | Family | Category | Source | Images |
|---|
| Type | Easy | Medium | Hard | Per font |
|---|---|---|---|---|
| texture: the word printed or painted straight on a close-up photo of a real surface (wood, plaster, concrete, brick, metal, fabric, leather, paper, cardboard) | 3 | 2 | 5 | 10 |
| scene: the word painted on a wall inside a real photo (rooms, shops, facades, workshops) | 1 | 2 | 3 | 6 |
| object: the word on a printed object (label, box, book cover, poster, menu, shop sign, packaging, business card, painted wall) | 1 | 1 | 2 | 4 |
The images are frontal, with no slant or perspective, so an upright font never looks italic; texture and scene images add only a small camera deviation (corners moved at most 1.5 %, rotation at most 1°). Text contrast is at least 70 grey levels (50 on the hard level), no letter is clipped, and the long side is never over 2000 px. One word per image, 7-12 letters, taken from COCO-Text v2.
The WhatFontIs API on all 11,995 images of v1.0, October 2026, in its two modes: the default one and mode=pro, which takes about two and a half times as long per image and finds more. The API searches its whole catalogue of over 1.2 million fonts, not only the 600 fonts of the set. A result is correct when it is the right font family, in any weight or from any source: an image in Roboto Bold answered with Roboto Regular is correct; answered with Arial Bold it is wrong, however close it looks.
Top-1 font family accuracy: 89.6 % with mode=pro, 83.2 % in the default mode.
| Default mode | mode=pro | ||||||
|---|---|---|---|---|---|---|---|
| Images | Correct at 1 | In top 5 | In top 20 | Correct at 1 | In top 5 | In top 20 | |
| All images | 11,995 | 83.2 % | 92.2 % | 94.6 % | 89.6 % | 97.1 % | 98.5 % |
| texture | 5,997 | 83.1 % | 92.3 % | 94.4 % | 90.1 % | 97.4 % | 98.8 % |
| scene | 3,599 | 81.2 % | 90.2 % | 93.0 % | 87.6 % | 95.6 % | 97.5 % |
| object | 2,399 | 86.6 % | 95.2 % | 97.2 % | 91.5 % | 98.5 % | 99.4 % |
| easy | 2,995 | 86.0 % | 94.1 % | 95.8 % | 91.3 % | 97.9 % | 99.1 % |
| medium | 3,000 | 82.8 % | 91.1 % | 93.3 % | 89.8 % | 97.0 % | 98.4 % |
| hard | 6,000 | 82.1 % | 91.9 % | 94.5 % | 88.7 % | 96.7 % | 98.3 % |
| sans-serif | 4,000 | 75.2 % | 87.5 % | 91.6 % | 84.1 % | 94.7 % | 97.3 % |
| serif | 4,000 | 83.8 % | 94.3 % | 96.0 % | 88.8 % | 97.6 % | 99.0 % |
| slab serif | 1,995 | 92.6 % | 95.2 % | 95.8 % | 98.1 % | 99.4 % | 99.5 % |
| monospaced | 2,000 | 89.0 % | 94.7 % | 96.3 % | 93.9 % | 98.4 % | 99.2 % |
| Median time per image | 5.2 s | 12.8 s | |||||
Images without an answer count as misses: 20 in the default mode, where no letters were found, and 31 with mode=pro, where the request failed. Times are the median per image on the live API, measured on 1,200 of the images. The previous measurement, September 2026, had a single mode: 83.7 % correct at 1, 93.3 % in top 5, 96.5 % in top 20.
Annotations, font list and tools are in the GitHub repository. The images of v1.0 are in the release v1.0, as four zip files:
| File | Size | Images |
|---|---|---|
| WhatFontIs-Bench_v1.0_images_part1.zip | 466 MB | 00000-02998 |
| WhatFontIs-Bench_v1.0_images_part2.zip | 470 MB | 02999-05997 |
| WhatFontIs-Bench_v1.0_images_part3.zip | 460 MB | 05998-08996 |
| WhatFontIs-Bench_v1.0_images_part4.zip | 604 MB | 08997-11994 |
With the GitHub CLI:
gh release download v1.0 --repo whatfontis/WhatFontIs-Bench
Unzip all four into v1/; they create v1/scenes/NNNNN.jpg. The text crops (the word cut out
of each image, unscaled) come from python tools/make_crops.py v1. Font files are not included.
| File | What it holds |
|---|---|
v1/labels.jsonl | one line per image: text, font (title, family, source, category, WhatFontIs page), type, level, crop box, word and letter corners, how it was made (cap height, font size, ink colour, contrast, blur, noise, JPEG quality, effects, camera transform) and the background |
v1/annotations.json | the same set in COCO format: images, categories (one per font), annotations (word box, text, letters) |
v1/fonts.json | the fonts: title, family, source, category, capitals-only flag, WhatFontIs page, number of images |
v1/backgrounds.json | the background photos, with source page, author and license |
words.txt | the word list the texts were drawn from (shared by all versions) |
v1/specimens/ | ten sheets showing all 600 fonts, 60 per sheet |
tools/make_crops.py | cuts the text crops out of the images |
backgrounds.json.Questions about the set: contact us.