Criterion keeps its Greek ending, and that is the whole point of it as an example. The Greek kriterion meant a means for judging, from krites, "a judge," from a root meaning "to sieve" and hence to discriminate or distinguish. English used it as a frankly Greek word from the 1610s and had naturalised it enough to write it in Latin letters by the 1660s — but it never gave up the inflection.
That is why the plural is criteria and not criterions. English keeps the Greek neuter plural in a small set of words taken from the page rather than from speech: phenomenon and phenomena, schema and schemata. The pattern is so unfamiliar that criteria and phenomena are constantly used as singulars, which is what usually happens to a foreign inflection given enough time. Data has already lost the fight.
criterion (n.) "a standard of judgment or criticism, rule by which opinion or conduct can be tested," 1660s, from Latinized form of Greek kriterion "means for judging, standard," from krites "judge," from PIE root *krei- "to sieve," thus "discriminate, distinguish." Used in English as a Greek word from 1610s. also from 1660s