5 commits

Author	SHA1	Message	Date
Daoud Clarke	2844c1df75	Index common crawl data	2021-12-13 11:23:01 +00:00
Daoud Clarke	65b366d30d	Add spacy	2021-12-12 20:58:44 +00:00
Daoud Clarke	c46257c6d1	Use our own filesystem-based queue	2021-12-11 16:57:17 +00:00
Daoud Clarke	14817d7657	Optimise imports	2021-12-05 20:38:05 +00:00
Daoud Clarke	312f32bf61	Add common crawl extract script and dependency management with poetry	2021-12-05 20:31:49 +00:00