In order to provide an unbiased accuracy comparison of our Language Identification API, we decided to test it against the Europarl corpus. It is a corpus that contains 21 languages each having 1000 texts, randomly sampled from the Europarl corpus. It is the same corpus used by Mike McCandless to compare the accuracy of CLD (Compact Language Detector), Tika and language-detection. For completeness, we also included langid.py in our comparison test.

Tika is missing Bulgarian (bg), Czech (cs), Lithuanian (lt) and Latvian (lv), for our comparison we will only test on the remaining 17 languages that all language detectors support.

da	Danish	de	German	el	Greek
en	English	es	Spanish	et	Estonian
fi	Finnish	fr	French	hu	Hungarian
it	Italian	nl	Dutch	pl	Polish
pt	Portuguese	ro	Romanian	sk	Slovak
sl	Slovene	sv	Swedish

This test is challenging, as many of the texts are very short: the shortest is 25 bytes, and 290 (1.7%) of the 17000 are 30 bytes or less.

In addition to the challenges of the corpora, there is also the difference in the number of languages each language detector can identify. WhatLanguage.net detects 110 languages, langid.py detects 97 languages, CLD detects at least 76 languages, while language-detection detects 53 and Tika detects 27. This biases slightly against our WhatLanguage.net API and langid.py, since the classification task becomes harder with more supported languages.

The results of CLD, Tika and language-detection are taken from the comparison test of Mike McCandless. We used version 1.1.3 of langid.py.

Below are the test results:

WhatLanguage.net API results (total 99.79% = 16965 / 17000):

da	98.5%	da=985	nb=15
de	99.9%	de=999	sv=1
el	100.0%	el=1000
en	100.0%	en=1000
es	99.5%	es=995	gl=3	af=1	ia=1
et	100.0%	et=1000
fi	99.9%	fi=999	et=1
fr	99.6%	fr=996	lb=1	mg=1	pt=1	vo=1
hu	100.0%	hu=1000
it	99.9%	it=999	bg=1
nl	99.7%	nl=997	li=2	fy=1
pl	100.0%	pl=1000
pt	99.8%	pt=998	gl=2
ro	100.0%	ro=1000
sk	99.7%	sk=997	cs=3
sl	100.0%	sl=1000
sv	100.0%	sv=1000

For completeness, the results for the four languages (Bulgarian (bg), Czech (cs), Lithuanian (lt) and Latvian (lv)) excluded from this test:

bg	99.8%	bg=998	uk=1	mk=1
cs	100.0%	cs=1000
lt	100.0%	lt=1000
lv	99.9%	lv=999	pt=1

langid.py results (total 99.16% = 16858 / 17000):

da	95.4%	da=954	no=29	nb=14	nl=2	pt=1
de	99.8%	de=998	en=1	fi=1
el	100.0%	el=1000
en	99.9%	en=999	es=1
es	99.4%	es=994	gl=5	it=1
et	98.6%	et=986	fi=7	it=2	en=1	id=1	pt=1	se=1	sv=1
fi	99.7%	fi=997	da=1	de=1	no=1
fr	99.5%	fr=995	es=2	ku=1	lb=1	oc=1
hu	99.9%	hu=999	sv=1
it	99.8%	it=998	de=1	eu=1
nl	99.9%	nl=999	af=1
pl	99.9%	pl=999	en=1
pt	99.3%	pt=993	gl=5	es=1	oc=1
ro	99.9%	ro=999	fr=1
sk	98.4%	sk=984	cs=11	sl=2	ga=1	lt=1	pt=1
sl	97.3%	sl=973	hr=17	bs=5	it=2	hu=1	lt=1	sv=1
sv	99.1%	sv=991	no=8	nb=1

CLD results (total 98.82% = 16800 / 17000):

da	93.4%	da=934	nb=54	sv=5	fr=2	eu=2	is=1	hr=1	en=1
de	99.6%	de=996	en=2	ga=1	cy=1
el	100.0%	el=1000
en	100.0%	en=1000
es	98.3%	es=983	pt=4	gl=3	en=3	it=2	eu=2	id=1	fi=1	da=1
et	99.6%	et=996	ro=1	id=1	fi=1	en=1
fi	100.0%	fi=1000
fr	99.2%	fr=992	en=4	sq=2	de=1	ca=1
hu	99.9%	hu=999	it=1
it	99.5%	it=995	ro=1	mt=1	id=1	fr=1	eu=1
nl	99.5%	nl=995	af=3	sv=1	et=1
pl	99.6%	pl=996	tr=1	sw=1	nb=1	en=1
pt	98.7%	pt=987	gl=4	es=3	mt=1	it=1	is=1	ht=1	fi=1	en=1
ro	99.8%	ro=998	da=1	ca=1
sk	98.8%	sk=988	cs=9	en=2	de=1
sl	95.1%	sl=951	hr=32	sr=8	sk=5	en=2	id=1	cs=1
sv	99.0%	sv=990	nb=9	en=1

Tika results (total 97.12% = 16510 / 17000):

da

87.6%

da=876

no=112

nl=4

sv=3

it=1

fr=1

et=1

en=1

de=1

de

98.5%

de=985

nl=3

it=3

da=3

sv=2

fr=2

sl=1

ca=1

el

100.0%

el=1000

en

96.9%

en=969

no=10

it=6

ro=4

sk=3

fr=3

hu=2

et=2

sv=1

es

89.8%

es=898

gl=47

pt=22

ca=15

it=6

eo=4

fr=3

fi=2

sk=1

nl=1

et=1

et

99.1%

et=991

fi=4

fr=2

sl=1

no=1

ca=1

fi

99.4%

fi=994

et=5

hu=1

fr

98.0%

fr=980

sl=6

eo=3

et=2

sk=1

ro=1

no=1

it=1

gl=1

fi=1

es=1

de=1

ca=1

hu

99.9%

hu=999

ca=1

it

99.4%

it=994

eo=4

pt=1

fr=1

nl

97.8%

nl=978

no=8

de=3

da=3

sl=2

ro=2

pl=1

it=1

gl=1

et=1

pl

99.1%

pl=991

sl=3

sk=2

ro=1

it=1

hu=1

fi=1

pt

94.4%

pt=944

gl=48

hu=2

ca=2

it=1

et=1

es=1

en=1

ro

99.3%

ro=993

is=2

sl=1

pl=1

it=1

hu=1

fr=1

sk

96.2%

sk=962

sl=21

pl=13

it=2

ro=1

et=1

sl

98.5%

sl=985

sk=7

et=4

it=2

pt=1

no=1

sv

97.1%

sv=971

no=15

nl=6

da=6

de=1

ca=1

Language-detection results (total 99.22% = 16868 / 17000):

da	97.1%	da=971	no=28	en=1
de	99.8%	de=998	da=1	af=1
el	100.0%	el=1000
en	99.7%	en=997	nl=1	fr=1	af=1
es	99.5%	es=995	pt=4	en=1
et	99.6%	et=996	fi=2	de=1	af=1
fi	99.8%	fi=998	et=2
fr	99.8%	fr=998	sv=1	it=1
hu	99.9%	hu=999	id=1
it	99.8%	it=998	es=2
nl	97.7%	nl=977	af=21	sv=1	de=1
pl	99.9%	pl=999	nl=1
pt	99.4%	pt=994	es=3	it=1	hu=1	en=1
ro	99.9%	ro=999	fr=1
sk	98.7%	sk=987	cs=8	sl=2	ro=1	lt=1	et=1
sl	97.2%	sl=972	hr=27	en=1
sv	99.0%	sv=990	no=8	da=2

Conclusion

Our Language Identification API has the best accuracy. Closely followed by the language-detection and langid.py libraries. So if you need accurate language identification in your application, you should test our API without any risk and see for yourself. Besides being very accurate, our API web service also detects 110 languages and can be easily called by any programming language due to its JSON or XML formatted output.