Commit graph

Select branches

Hide pull requests

add-history

add-term-info-to-index

allow-extra-results

fastapi-users

improve-crawling

local

main

monolith

rishabh-add-launch-json

rishabh-port-configuration

rishabh-unit-testing-with-ci

unique-urls-in-queue

update-index

update-readme-for-new-crawler

#100

#109

#109

#11

#110

#112

#113

#114

#115

#116

#117

#118

#119

#120

#121

#122

#123

#124

#125

#126

#127

#128

#129

#13

#130

#17

#18

#21

#23

#24

#25

#31

#34

#36

#37

#38

#39

#41

#42

#46

#47

#5

#51

#53

#54

#55

#56

#57

#58

#6

#61

#66

#68

#69

#70

#71

#72

#73

#74

#78

#82

#83

#86

#89

#90

#91

#92

#93

#94

#95

#96

#97

#98

Mono Color

• 1620322118 Use try_files instead of root Daoud Clarke 2023-10-13 20:41:39 +0100
• a409c240e1 Missed semicolongs Daoud Clarke 2023-10-13 13:48:56 +0100
• c30fd607d4 Special cases for home and stats pages Daoud Clarke 2023-10-13 13:41:17 +0100
• 973fe8198c There may not be files there, so use -rf Daoud Clarke 2023-10-12 22:12:58 +0100
• b182e92b75 Extra comme Daoud Clarke 2023-10-12 22:06:57 +0100
• bd496aeab2 Add the app.json in Dockerfile Daoud Clarke 2023-10-12 22:02:27 +0100
• ef6b87e9cd Use a predeploy script instead Daoud Clarke 2023-10-12 21:44:34 +0100
• 64f0d0a04a typo Daoud Clarke 2023-10-12 21:32:16 +0100
• 09de4a918e Copy to the mounted volume in python code Daoud Clarke 2023-10-12 21:21:30 +0100
• 88c3437456
  Merge pull request #117 from mwmbl/include-front-end Daoud Clarke 2023-10-12 20:53:40 +0100
• 2e4456a338 Build the front end #117 Daoud Clarke 2023-10-12 20:52:51 +0100
• 013c007bb0 Serve front end Daoud Clarke 2023-10-12 17:38:19 +0100
• 536487b3d2 Add front end Daoud Clarke 2023-10-12 17:17:42 +0100
• 0b380b50bf Move the static spec down Daoud Clarke 2023-10-11 21:15:23 +0100
• 040921836c Add static hosting Daoud Clarke 2023-10-11 21:09:58 +0100
• 8494b746e7 Add api.mwmbl.org to allowed hosts Daoud Clarke 2023-10-10 20:39:31 +0100
• 8f0061f0c3 Fix port Daoud Clarke 2023-10-10 20:35:30 +0100
• c3fd328237
  Merge pull request #116 from mwmbl/django-rewrite-fixes Daoud Clarke 2023-10-10 20:23:15 +0100
• 1227ae33c8 Run poetry lock #116 Daoud Clarke 2023-10-10 20:21:37 +0100
• c6d9e6ebb0 Fix some paths, use prod settings in Dockerfile Daoud Clarke 2023-10-10 20:18:43 +0100
• 213bdaa365
  Merge pull request #115 from mwmbl/django-rewrite Daoud Clarke 2023-10-10 16:25:36 +0100
• 918eaa8709 Rename django app to mwmbl #115 Daoud Clarke 2023-10-10 13:51:06 +0100
• fab5e5c782 Use different dev and prod settings Daoud Clarke 2023-10-08 21:42:04 +0100
• a1d6fd8bb1 Start background processes Daoud Clarke 2023-10-08 21:20:32 +0100
• b6fd27352b Add crawler router Daoud Clarke 2023-10-08 14:13:38 +0100
• ed64ca6c91 Merge branch 'main' into django-rewrite Daoud Clarke 2023-10-07 19:19:34 +0100
• d716cb347f
  Merge pull request #114 from mwmbl/exclude-domains-by-keyword Daoud Clarke 2023-10-04 20:20:51 +0100
• 41061a695b Add tests #114 Daoud Clarke 2023-10-04 20:19:42 +0100
• 593c71f689 Exclude domains by keyword Daoud Clarke 2023-10-04 19:51:33 +0100
• a77dc3eb4c
  Merge pull request #113 from mwmbl/more-stats Daoud Clarke 2023-10-02 22:19:20 +0100
• 988f3fd2a9 Add more stats #113 Daoud Clarke 2023-10-02 22:19:02 +0100
• 7c3aea5ca0 Temporary just select some URLs at random for initialization Daoud Clarke 2023-09-29 22:32:31 +0100
• 826d3d6ba9
  Merge pull request #112 from mwmbl/stats Daoud Clarke 2023-09-29 21:49:28 +0100
• ab527c4b58 Use stats manager from redis URL #112 Daoud Clarke 2023-09-29 21:48:36 +0100
• 0d795b7c64 Fix bugs with date method Daoud Clarke 2023-09-29 21:27:32 +0100
• e1bf423e69 Get stats Daoud Clarke 2023-09-29 13:58:26 +0100
• a55a027107 Store stats in redis Daoud Clarke 2023-09-28 17:48:29 +0100
• db658daa88 Store stats in redis Daoud Clarke 2023-09-28 17:48:29 +0100
• 86a6524f0a WIP add search API to Django Daoud Clarke 2023-09-24 08:09:18 +0100
• 177324353f Merge branch 'main' into django-rewrite Daoud Clarke 2023-09-23 12:56:22 +0100
• bec00cdab5 Exclude additional domain Daoud Clarke 2023-09-22 23:06:04 +0100
• 7e054d0854 Better blacklist Daoud Clarke 2023-09-22 23:04:37 +0100
• ed96386f05
  Merge pull request #110 from mwmbl/update-blacklist Daoud Clarke 2023-09-22 21:54:12 +0100
• 019095a4c1 Exclude blacklisted domains #110 Daoud Clarke 2023-09-22 21:53:53 +0100
• 19cc196e34 Add django ninja Daoud Clarke 2023-09-22 19:56:42 +0100
• d29cf2acbb Optimize Dockerfile by cleaning up apt cache #109 Peter Dave Hello 2023-09-19 20:32:49 +0800
• 4aefc48716 Add django Daoud Clarke 2023-08-27 07:37:15 +0100
• e75b7121b0 Try fastapi users fastapi-users Daoud Clarke 2023-08-27 07:28:56 +0100
• 18dc760a34 Temp disable CORS Daoud Clarke 2023-05-20 23:23:43 +0100
• 01bf4c21df Temporarily disable lemmy as connection is refused Daoud Clarke 2023-05-20 22:26:33 +0100
• 8851d86ff4 Justext is not extra Daoud Clarke 2023-05-20 22:17:23 +0100
• b5b37629ce Clean unicode when formatting result Daoud Clarke 2023-05-20 22:11:51 +0100
• dec7c4853d Whitespace fix Daoud Clarke 2023-05-20 21:52:33 +0100
• 3e08c6e804 Check response status; provide an answer when registering Daoud Clarke 2023-05-20 21:51:57 +0100
• 60980a6bc7
  Merge pull request #100 from mwmbl/user-registration Daoud Clarke 2023-04-30 20:31:09 +0100
• 8d64af4f1b Keep track of curated couments #100 Daoud Clarke 2023-04-30 18:25:48 +0100
• f0592f99df Require a curated boolean flag Daoud Clarke 2023-04-13 06:27:51 +0100
• 759dbf07b9 Revert index Daoud Clarke 2023-04-13 05:37:43 +0100
• 00b5438492 Track curated items in the index Daoud Clarke 2023-04-09 06:26:23 +0100
• a87d3d6def Store curated pages in the index Daoud Clarke 2023-04-09 05:31:23 +0100
• 61cdd4dd71 Merge branch 'main' into user-registration Daoud Clarke 2023-04-01 07:17:29 +0100
• 3e1f5da28e Off by one error with page size Daoud Clarke 2023-04-01 06:40:03 +0100
• 91269d5100 Handle a bad batch Daoud Clarke 2023-04-01 06:35:44 +0100
• e9dfd40ecb
  Merge pull request #98 from mwmbl/rishabh-fix-trim-data Rishabh Singh Ahluwalia 2023-03-28 08:18:53 -0700
• f232badd67 fix comma formatting #98 Rishabh Singh Ahluwalia 2023-03-27 22:18:10 -0700
• 8e197a09f9 Fix trimming page size logic while adding to a page Rishabh Singh Ahluwalia 2023-03-26 10:04:05 -0700
• 23688bd3ad Merge branch 'master' into user-registration Daoud Clarke 2023-03-18 22:37:45 +0000
• 0838157185
  Merge pull request #97 from mwmbl/initialize-with-found-urls Daoud Clarke 2023-02-25 18:20:11 +0000
• 7d0c55c015 Fix broken test #97 Daoud Clarke 2023-02-25 18:18:09 +0000
• e5c08e0d24 Fix big with other URLs Daoud Clarke 2023-02-25 16:48:59 +0000
• a24156ce5c Initialize URLs by processing them like all other URLs to avoid bias Daoud Clarke 2023-02-25 13:45:03 +0000
• 6bb8bdf0c2 Initialize with new URLs Daoud Clarke 2023-02-25 10:48:22 +0000
• a9e2b48840
  Merge pull request #96 from mwmbl/unique-urls-in-queue Daoud Clarke 2023-02-25 10:35:32 +0000
• 5c94dfa669 Shuffle URLs before batching #96 unique-urls-in-queue Daoud Clarke 2023-02-25 10:35:10 +0000
• 6ff62fb119 Ensure URLs in queue are unique Daoud Clarke 2023-02-25 10:34:09 +0000
• c36e1dffcb Remove picolisp as a top domain since there are duplicate URLs Daoud Clarke 2023-02-25 09:56:26 +0000
• 362f9bfa9e Write page to the correct location (metadata size offset bug fix) Daoud Clarke 2023-02-24 21:46:18 +0000
• 5616626fc1
  Merge pull request #89 from mwmbl/update-urls-queue-quickly Daoud Clarke 2023-02-24 21:39:40 +0000
• bc6be8b6d5 Merge branch 'master' into update-urls-queue-quickly #89 Daoud Clarke 2023-02-24 21:37:54 +0000
• a03b76e5cc Fix broken test Daoud Clarke 2023-02-24 21:37:32 +0000
• c97d946fcf Go back to processing 10,000 batches at a time Daoud Clarke 2023-02-24 21:29:42 +0000
• 38a5dbbf3c
  Merge pull request #94 from mwmbl/rishabh-port-configuration Rishabh Singh Ahluwalia 2023-02-23 07:31:07 -0800
• 2aa61a5121
  Merge pull request #95 from mwmbl/rishabh-unit-testing-with-ci Rishabh Singh Ahluwalia 2023-02-23 07:30:48 -0800
• 30aff3b920 Add pytest, unit tests for completer,gh actions ci #95 rishabh-unit-testing-with-ci Rishabh Singh Ahluwalia 2023-02-22 21:37:10 -0800
• 842aec19e2 Add port to args #94 rishabh-port-configuration Rishabh Singh Ahluwalia 2023-02-22 19:59:42 -0800
• 50a059410b
  Merge pull request #93 from mwmbl/add-code-of-conduct-1 Daoud Clarke 2023-02-15 20:36:31 +0000
• 084a870f65
  Merge pull request #92 from mwmbl/rishabh-add-launch-json Rishabh Singh Ahluwalia 2023-02-12 07:17:47 -0800
• 68ecdee145
  Create CONTRIBUTING.md Daoud Clarke 2023-02-11 15:17:35 +0000
• 3a07fb54b5
  Create CODE_OF_CONDUCT.md #93 Daoud Clarke 2023-02-11 15:13:08 +0000
• d8dbe54f9c
  Update README.md Daoud Clarke 2023-02-11 15:10:30 +0000
• 2daf902ca3
  Merge pull request #90 from mwmbl/m1-mmap-issue-fix-2 Daoud Clarke 2023-02-11 08:30:46 +0000
• 7fdc8480bd add launch.json for vscode debugging #92 rishabh-add-launch-json Rishabh Singh Ahluwalia 2023-02-10 20:59:09 -0800
• e890e56661 Offset by metadata size manually to increase compatibility #90 Daoud Clarke 2023-02-05 15:49:09 +0000
• 90ef7ecf46 Rebuild index for bigger metadata section #91 Daoud Clarke 2023-02-05 15:41:40 +0000
• 5783cee6b7 Fix bugs Daoud Clarke 2023-01-24 22:52:58 +0000
• 77e39b4a89 Optimise URL update Daoud Clarke 2023-01-22 20:28:18 +0000
• 66700f8a3e Speed up domain parsing Daoud Clarke 2023-01-20 20:53:50 +0000
• 2b36f2ccc1 Try and balance URLs before adding to queue Daoud Clarke 2023-01-19 21:56:40 +0000
• 603fcd4eb2 Create a custom URL queue Daoud Clarke 2023-01-14 21:59:31 +0000
• 01f08fd88d Return updated URLs Daoud Clarke 2023-01-14 19:15:26 +0000