langreiter.com plain, simple: 2006-08-04-gigaGram

LANGREITER.COM plain, simple

START

INDEX

2006-08-04-gigaGram

CREATED BY chris • LAST EDITED BY chris 6713 days AGO

[create N-Gram]s to the people: "We processed 1,011,582,453,213 words of running text and are publishing the counts for all 1,146,580,664 five-word sequences that appear at least 40 times. There are 13,653,070 unique words, after discarding words that appear less than 200 times."

"[...] [T]here's no data like more data [...]"

And in that spirit^*, Greg Linden points to even more research data being made available from AOL Research (who'd have known ...).

^* Misguided as, in an infinity of non-tongue-in-cheek cases, it is; about the best outcome of that particular data release would be that it raises awareness of what an incredibly detailed picture of their users search companies have ...