Commit graph

178 commits

Author SHA1 Message Date
Daoud Clarke
e3baf87918 Remove seemingly extraneous backslashes 2022-06-19 11:27:37 +01:00
Daoud Clarke
c245be775b Use an updated template 2022-06-19 11:25:38 +01:00
Daoud Clarke
01772517da Remove problematic SSL_DIRECTIVES line 2022-06-19 11:23:01 +01:00
Daoud Clarke
a67ca7b298 Enable CORS in nginx 2022-06-19 11:16:03 +01:00
Daoud Clarke
866c17f2dc Use the dokku app storage 2022-06-19 09:53:19 +01:00
Daoud Clarke
16c2692099 Start processing historical data on startup 2022-06-19 08:56:55 +01:00
Daoud Clarke
d400950689 Add script to process historical data 2022-06-18 15:31:35 +01:00
Daoud Clarke
eb1c59990c Expose the port 2022-06-17 23:57:58 +01:00
Daoud Clarke
d7c6dcb5c2 Use the correct port for dokku 2022-06-17 23:54:22 +01:00
Daoud Clarke
77088a8a1b Use a database URL env var 2022-06-17 23:39:24 +01:00
Daoud Clarke
476481c5f8 Put the resources in the package 2022-06-17 23:32:43 +01:00
Daoud Clarke
505e7521d4 Copy the resources 2022-06-17 23:29:04 +01:00
Daoud Clarke
5ea9efcfa2 Fix relative path 2022-06-17 23:19:30 +01:00
Daoud Clarke
1c7420e5fb Don't depend on existing data 2022-06-17 23:12:22 +01:00
Daoud Clarke
a003914e91 Fix boto3 dependency 2022-06-17 22:14:55 +01:00
Daoud Clarke
363103468e Update Dockerfile for changes 2022-06-17 21:26:21 +01:00
Daoud Clarke
e2eb405083 Combine crawler and search servers 2022-06-16 22:49:41 +01:00
Daoud Clarke
7771657684
Merge pull request #53 from mwmbl/record-historical-batches
Record historical batches
2022-06-16 22:09:12 +01:00
Daoud Clarke
14107acc75 Use new server 2022-06-09 22:24:54 +01:00
Daoud Clarke
aaca8b2b6e Record historical batches via the API 2022-06-05 09:15:04 +01:00
Daoud Clarke
617666e3b7
Merge pull request #51 from mwmbl/learning-to-rank
Learning to rank
2022-06-04 12:36:15 +01:00
Daoud Clarke
770b4b945b Refactor feature extraction 2022-05-07 22:52:36 +01:00
Daoud Clarke
87d8b40cad Make order_results public 2022-05-06 23:15:50 +01:00
Daoud Clarke
229819e57e Refactor to allow LTR ranker 2022-03-27 22:32:44 +01:00
Daoud Clarke
94287cec01 Get features for each string separately 2022-03-21 21:49:10 +00:00
Daoud Clarke
4740d89c6a Add domain score feature 2022-03-21 21:13:20 +00:00
Daoud Clarke
af6a28fac3 Implement learning to rank feature extraction and thresholding 2022-03-20 22:01:45 +00:00
Daoud Clarke
2d334074af Make get_results() public for learning to rank 2022-03-20 17:25:54 +00:00
Daoud Clarke
ee5ca6bcf6 Experiment with score variations (best is simple weighted domain score) 2022-02-27 21:24:16 +00:00
Daoud Clarke
6fb310c363 Use addition instead of multiplication 2022-02-25 22:19:26 +00:00
Daoud Clarke
4e6516ccf1 Scale by 0.99 2022-02-25 22:14:49 +00:00
Daoud Clarke
f5afbed2e5 Handle empty list 2022-02-25 22:11:09 +00:00
Daoud Clarke
efafec5214 Rank using item score as well as match score 2022-02-25 22:08:37 +00:00
Daoud Clarke
e1e9e404a3 Dedupe before indexing 2022-02-24 22:01:42 +00:00
Daoud Clarke
f5b20d0128 Index link counts 2022-02-24 20:47:36 +00:00
Daoud Clarke
b5b2005323 Store computed link counts 2022-02-23 22:13:38 +00:00
Daoud Clarke
00d18c3474 Remove unused code 2022-02-23 21:59:24 +00:00
Daoud Clarke
d19e0e51f7
Merge pull request #47 from mwmbl/include-metadata-in-index
Include metadata in index
2022-02-23 21:10:24 +00:00
Daoud Clarke
04a33a134b Fixes to mwmbl API for changes to the index 2022-02-22 22:27:02 +00:00
Daoud Clarke
ae3b334a7f Fixes for API changes 2022-02-22 22:12:39 +00:00
Daoud Clarke
326f7e3d7f Use JSON instead of struct to store metadata 2022-02-18 22:22:47 +00:00
Daoud Clarke
e6273c7f76 WIP: include metadata in index - using struct approach 2022-02-18 22:12:22 +00:00
Daoud Clarke
82c46b50bc
Merge pull request #46 from mwmbl/refactor-for-evaluation
Refactor to enable easier evaluation
2022-02-16 21:28:21 +00:00
Daoud Clarke
e03e379ccf Refactor to enable easier evaluation 2022-02-09 22:43:47 +00:00
Daoud Clarke
4e36ee198c
Merge pull request #42 from mwmbl/update-readme-for-new-crawler
Update readme for recent changes
2022-02-04 23:26:11 +00:00
Daoud Clarke
c4e86ce313 Update readme for recent changes 2022-02-04 22:07:09 +00:00
Daoud Clarke
51f2dd2690 Merge branch 'master' of github.com:mwmbl/mwmbl 2022-02-04 21:49:40 +00:00
Daoud Clarke
9f78d19c8c
Merge pull request #41 from ColinEspinas/add-branding
Add branding to readme
2022-02-04 21:28:41 +00:00
ColinEspinas
b2e01d33e8 docs: better title display on readme 2022-02-04 20:53:55 +01:00
Colin Espinas
95c9bcfe3b Merge branch 'mwmbl:master' into add-branding 2022-02-04 20:51:38 +01:00