* [WIP] remote webgraph client
* [WIP] use remote webgraph for backlinks during indexing. still need to properly batch the requests
* support batch requests in sonic
* [WIP] use remote webgraph in explore and make sure ranking pipeline always sets updated score
* use remote webgraph for inbound similarity
* return correct type from explore api
* Implemented `keybind` module to handle keyboard shortcuts
* Removal of direct DOM querying and the addition of searchbar keybindings
* Remove generics from 'keybind'
It's always used with 'Refs' as context, so there is no need to have it generic
* Revert 'Searchbar' to use simple keydown match instead of 'Keybind'
The functionality didn't work (for instance enter didn't trigger a search). It would require a lot of aditional complexity in 'Keybind' to also support the use case from searchbar. It's okay to have some code duplication if this results in a simpler solution that will therefore be more readable and mantainable long term
* Remove need to know about keyboard event in keybind callbacks
This forces us to not rely on direct manipulation of the event, but instead implement the necesarry functionality in helper methods in the different components
* forgot to remove a console.log...
---------
Co-authored-by: Mikkel Denker <mikkel@stract.com>
we now redirect clients with js disabled using a <noscript><meta ...</noscript> tag. this setup allows us to show an empty serp before the search is finished if one has enabled js. it also makes it posible to add more customization options later like applying optics directly from the search string. otherwise, the server would have no way of knowing what the optic rules for a particular optic name is.
a normal snippet should always be returned from the api. an application can then choose to show the rich snippet instead if one is present. this gives more flexibility when building applications on top of stracts api
* Initial implementation of importing sites from an optic
* Removed unused import
* Updated button text
* Implemented client-side WASM to allow for parsing of imported .optic files
* Removed unneeded deps & updated `CONTRIBUTING.md` to reflect wasm-pack needs
* CI updates
* Added vite-plugin-wasm-pack to ensure wasm modules get copied over
* CI fix >:(
* More CI attempts
* agony - CSP fix & further wasm-pack fixes
* CSP updates
* Package update to prevent an unneccesary build of wasm
* reduce bloat in ci build log from wasm
* fix another non-determinsticly failing test
* only install wasm-pack as part of setup steps in CONTRIBUTING.md
./scripts/ci/check seems to fail if it tries to install wasm-pack while it is already installed (at least on my machine). as it is already added as a step in CONTRIBUTING.md we can assume it has been installed on the system
* add vite plugin to ensure changes to 'crates/client-wasm' gets reflected in the frontend.
adapted from https://github.com/StractOrg/stract/pull/109
* run 'npm run format'
* propagate errors from wasm crate
* update nodejs to v20.10 in github action
* loosen restriction on npm version
frontend should work as long as we have correct nodejs version. don't think the npm version is necessary
* Add basic CI
* Add liburing installation step to CI workflow
* Run `npm install` as part of ci/check
* Add `@types/node` package
* Add `submodules: 'recursive'` to CI
* Skip test if test data is not available
* Install `cargo-about` in CI
When adding an optic, we first make sure that we can actually fetch the optic. This check was performed client-side before, which would cause some CORS errors if it was against the CORS policy of the server hosting the optic. This commit introduces a simple endpoint on our frontend server, so the request to the optic server now doesn't come directly from the client.
This creates a better separation between the frontend and backend, and avoids e.g. 'fetch_discussions' abstraction leak.
It also allows for these endpoints to be used in different contexts separately without having to perform a full search.
the summarization becomes much better if we allow the model to first generate a candidate summarization and then improving on it.
doing the improvement step just once seems to significantly improve the summary.
we also now use an llm (mistral 7b) for the summarisations, as we can then use the same model for multiple tasks and serve it using gpus, thus significantly decreasing the latency.
* [ios] discussion widget had two icons as <summary> list-none was not respected on safari.
* [ios] added a viewport width to remove horizontal scrolling.
* [ios] increased font size of searchbar to prevent ios from zooming when searchbar gets selected.
* 'found x results' was ugly on mobile. now only shown on desktop
* decreased font size of optic and language selector to not remove attention from results. hopefully they are still noticed.
* added og:title and og:image to improve experience when stract is linked from somewhere.
* read raw .zim file format
* iterate articles and images from zim file
* construct entities for entity index using zimba
* parse images from .zim file and store them in the entity index
* add whitespace in entity info to separate <li> elements
* zimba readme
* Remove `.svelte-kit/tsconfig.json`
This is part of .gitignore but somehow got through.
* Migrate to SvelteKit 2
* Update frontend dependencies
* fix lost divider and dynamic title on search page due to missing await
---------
Co-authored-by: Mikkel Denker <mikkel@trystract.com>
