Commit graph

13 commits

Author SHA1 Message Date
Daoud Clarke
b5b37629ce Clean unicode when formatting result 2023-05-20 22:11:51 +01:00
Rishabh Singh Ahluwalia
e9dfd40ecb
Merge pull request #98 from mwmbl/rishabh-fix-trim-data
Fix trimming page size logic while adding to a page
2023-03-28 08:18:53 -07:00
Rishabh Singh Ahluwalia
8e197a09f9 Fix trimming page size logic while adding to a page 2023-03-26 10:04:05 -07:00
Daoud Clarke
7d0c55c015 Fix broken test 2023-02-25 18:18:09 +00:00
Daoud Clarke
e5c08e0d24 Fix big with other URLs 2023-02-25 16:48:59 +00:00
Daoud Clarke
bc6be8b6d5 Merge branch 'master' into update-urls-queue-quickly 2023-02-24 21:37:54 +00:00
Daoud Clarke
a03b76e5cc Fix broken test 2023-02-24 21:37:32 +00:00
Rishabh Singh Ahluwalia
30aff3b920 Add pytest, unit tests for completer,gh actions ci 2023-02-22 21:37:10 -08:00
Daoud Clarke
2b36f2ccc1 Try and balance URLs before adding to queue 2023-01-19 21:56:40 +00:00
Daoud Clarke
6209382d76 Index batches in memory 2022-07-24 15:44:01 +01:00
Daoud Clarke
680fe1ca0c Fix unicode encoding error 2022-07-16 10:54:25 +01:00
Daoud Clarke
326f7e3d7f Use JSON instead of struct to store metadata 2022-02-18 22:22:47 +00:00
Daoud Clarke
e6273c7f76 WIP: include metadata in index - using struct approach 2022-02-18 22:12:22 +00:00