Compare commits

..

No commits in common. "master" and "v2.3.2" have entirely different histories.

25 changed files with 174 additions and 429 deletions

View file

@ -1,60 +0,0 @@
name: buildx
on:
push:
tags:
- 'v*'
jobs:
buildx:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v2
-
name: Prepare
id: prepare
run: |
DOCKER_IMAGE=andreimarcu/linx-server
DOCKER_PLATFORMS=linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/386
VERSION=version-${GITHUB_REF#refs/tags/v}
TAGS="--tag ${DOCKER_IMAGE}:${VERSION} --tag ${DOCKER_IMAGE}:latest"
echo ::set-output name=docker_image::${DOCKER_IMAGE}
echo ::set-output name=version::${VERSION}
echo ::set-output name=buildx_args::--platform ${DOCKER_PLATFORMS} \
--build-arg VERSION=${VERSION} \
--build-arg BUILD_DATE=$(date -u +'%Y-%m-%dT%H:%M:%SZ') \
--build-arg VCS_REF=${GITHUB_SHA::8} \
${TAGS} --file Dockerfile .
-
name: Set up Docker Buildx
uses: crazy-max/ghaction-docker-buildx@v3
-
name: Docker Buildx (build)
run: |
docker buildx build --output "type=image,push=false" ${{ steps.prepare.outputs.buildx_args }}
-
name: Docker Login
if: success()
env:
DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
run: |
echo "${DOCKER_PASSWORD}" | docker login --username "${DOCKER_USERNAME}" --password-stdin
-
name: Docker Buildx (push)
if: success()
run: |
docker buildx build --output "type=image,push=true" ${{ steps.prepare.outputs.buildx_args }}
-
name: Docker Check Manifest
if: always()
run: |
docker run --rm mplatform/mquery ${{ steps.prepare.outputs.docker_image }}:${{ steps.prepare.outputs.version }}
-
name: Clear
if: always()
run: |
rm -f ${HOME}/.docker/config.json

5
.gitignore vendored
View file

@ -29,9 +29,8 @@ _testmain.go
*.prof
linx-server
linx-cleanup/linx-cleanup
linx-genkey/linx-genkey
linx-server.conf
linx-cleanup
linx-genkey
files/
meta/
binaries/

143
README.md
View file

@ -1,34 +1,19 @@
# Development on this repository has been frozen.
Feel free to send a pull request if you are maintaining an active fork of this project to add a link to your repository in this readme.
### Active Forks
- ZizzyDizzyMC: [https://github.com/ZizzyDizzyMC/linx-server/](https://github.com/ZizzyDizzyMC/linx-server/)
- Seb3thehacker: [https://github.com/Seb3thehacker/linx-server](https://github.com/Seb3thehacker/linx-server)
---
linx-server
======
[![Build Status](https://travis-ci.org/andreimarcu/linx-server.svg?branch=master)](https://travis-ci.org/andreimarcu/linx-server)
Self-hosted file/media sharing website.
### Clients
**Official**
- CLI: **linx-client** - [Source](https://github.com/andreimarcu/linx-client)
**Unofficial**
- Android: **LinxShare** - [Source](https://github.com/iksteen/LinxShare/) | [Google Play](https://play.google.com/store/apps/details?id=org.thegraveyard.linxshare)
- CLI: **golinx** - [Source](https://github.com/mutantmonkey/golinx)
### Demo
You can see what it looks like using the demo: [https://demo.linx-server.net/](https://demo.linx-server.net/)
### Features
- Display common filetypes (image, video, audio, markdown, pdf)
- Display syntax-highlighted code with in-place editing
- Documented API with keys for restricting uploads
- Documented API with keys if need to restrict uploads (can use [linx-client](https://github.com/andreimarcu/linx-client) for uploading through command-line)
- Torrent download of files using web seeding
- File expiry, deletion key, file access key, and random filename options
@ -43,14 +28,9 @@ Getting started
-------------------
#### Using Docker
1. Create directories ```files``` and ```meta``` and run ```chown -R 65534:65534 meta && chown -R 65534:65534 files```
2. Create a config file (example provided in repo), we'll refer to it as __linx-server.conf__ in the following examples
Example running
```
docker run -p 8080:8080 -v /path/to/linx-server.conf:/data/linx-server.conf -v /path/to/meta:/data/meta -v /path/to/files:/data/files andreimarcu/linx-server -config /data/linx-server.conf
docker run -p 8080:8080 -v /path/to/meta:/data/meta -v /path/to/files:/data/files andreimarcu/linx-server
```
Example with docker-compose
@ -60,11 +40,11 @@ services:
linx-server:
container_name: linx-server
image: andreimarcu/linx-server
command: -config /data/linx-server.conf
entrypoint: /usr/local/bin/linx-server -bind=0.0.0.0:8080 -filespath=/data/files/ -metapath=/data/meta/
command: -sitename=Linx -siteurl=https://linx.example.com
volumes:
- /path/to/files:/data/files
- /path/to/meta:/data/meta
- /path/to/linx-server.conf:/data/linx-server.conf
network_mode: bridge
ports:
- "8080:8080"
@ -74,63 +54,51 @@ Ideally, you would use a reverse proxy such as nginx or caddy to handle TLS cert
#### Using a binary release
1. Grab the latest binary from the [releases](https://github.com/andreimarcu/linx-server/releases), then run ```go install```
2. Run ```linx-server -config path/to/linx-server.conf```
1. Grab the latest binary from the [releases](https://github.com/andreimarcu/linx-server/releases)
2. Run ```./linx-server```
Usage
-----
#### Configuration
All configuration options are accepted either as arguments or can be placed in a file as such (see example file linx-server.conf.example in repo):
All configuration options are accepted either as arguments or can be placed in an ini-style file as such:
```ini
bind = 127.0.0.1:8080
sitename = myLinx
maxsize = 4294967296
maxexpiry = 86400
# ... etc
```
...and then run ```linx-server -config path/to/linx-server.conf```
allowhotlink = true
# etc
```
...and then invoke ```linx-server -config path/to/config.ini```
#### Options
|Option|Description
|------|-----------
| ```bind = 127.0.0.1:8080``` | what to bind to (default is 127.0.0.1:8080)
| ```sitename = myLinx``` | the site name displayed on top (default is inferred from Host header)
| ```siteurl = https://mylinx.example.org/``` | the site url (default is inferred from execution context)
| ```selifpath = selif``` | path relative to site base url (the "selif" in mylinx.example.org/selif/image.jpg) where files are accessed directly (default: selif)
| ```maxsize = 4294967296``` | maximum upload file size in bytes (default 4GB)
| ```maxexpiry = 86400``` | maximum expiration time in seconds (default is 0, which is no expiry)
| ```allowhotlink = true``` | Allow file hotlinking
| ```contentsecuritypolicy = "..."``` | Content-Security-Policy header for pages (default is "default-src 'self'; img-src 'self' data:; style-src 'self' 'unsafe-inline'; frame-ancestors 'self';")
| ```filecontentsecuritypolicy = "..."``` | Content-Security-Policy header for files (default is "default-src 'none'; img-src 'self'; object-src 'self'; media-src 'self'; style-src 'self' 'unsafe-inline'; frame-ancestors 'self';")
| ```refererpolicy = "..."``` | Referrer-Policy header for pages (default is "same-origin")
| ```filereferrerpolicy = "..."``` | Referrer-Policy header for files (default is "same-origin")
| ```xframeoptions = "..." ``` | X-Frame-Options header (default is "SAMEORIGIN")
| ```remoteuploads = true``` | (optionally) enable remote uploads (/upload?url=https://...)
| ```nologs = true``` | (optionally) disable request logs in stdout
| ```force-random-filename = true``` | (optionally) force the use of random filenames
| ```custompagespath = custom_pages/``` | (optionally) specify path to directory containing markdown pages (must end in .md) that will be added to the site navigation (this can be useful for providing contact/support information and so on). For example, custom_pages/My_Page.md will become My Page in the site navigation
#### Cleaning up expired files
When files expire, access is disabled immediately, but the files and metadata
will persist on disk until someone attempts to access them. You can set the following option to run cleanup every few minutes. This can also be done using a separate utility found the linx-cleanup directory.
|Option|Description
|------|-----------
| ```cleanup-every-minutes = 5``` | How often to clean up expired files in minutes (default is 0, which means files will be cleaned up as they are accessed)
| ```-bind 127.0.0.1:8080``` | what to bind to (default is 127.0.0.1:8080)
| ```-sitename myLinx``` | the site name displayed on top (default is inferred from Host header)
| ```-siteurl "https://mylinx.example.org/"``` | the site url (default is inferred from execution context)
| ```-selifpath "selif"``` | path relative to site base url (the "selif" in mylinx.example.org/selif/image.jpg) where files are accessed directly (default: selif)
| ```-maxsize 4294967296``` | maximum upload file size in bytes (default 4GB)
| ```-maxexpiry 86400``` | maximum expiration time in seconds (default is 0, which is no expiry)
| ```-allowhotlink``` | Allow file hotlinking
| ```-contentsecuritypolicy "..."``` | Content-Security-Policy header for pages (default is "default-src 'self'; img-src 'self' data:; style-src 'self' 'unsafe-inline'; frame-ancestors 'self';")
| ```-filecontentsecuritypolicy "..."``` | Content-Security-Policy header for files (default is "default-src 'none'; img-src 'self'; object-src 'self'; media-src 'self'; style-src 'self' 'unsafe-inline'; frame-ancestors 'self';")
| ```-refererpolicy "..."``` | Referrer-Policy header for pages (default is "same-origin")
| ```-filereferrerpolicy "..."``` | Referrer-Policy header for files (default is "same-origin")
| ```-xframeoptions "..." ``` | X-Frame-Options header (default is "SAMEORIGIN")
| ```-remoteuploads``` | (optionally) enable remote uploads (/upload?url=https://...)
| ```-nologs``` | (optionally) disable request logs in stdout
| ```-force-random-filename``` | (optionally) force the use of random filenames
| ```-custompagespath "custom_pages"``` | (optionally) specify path to directory containing markdown pages (must end in .md) that will be added to the site navigation (this can be useful for providing contact/support information and so on). For example, custom_pages/My_Page.md will become My Page in the site navigation
#### Require API Keys for uploads
|Option|Description
|------|-----------
| ```authfile = path/to/authfile``` | (optionally) require authorization for upload/delete by providing a newline-separated file of scrypted auth keys
| ```remoteauthfile = path/to/remoteauthfile``` | (optionally) require authorization for remote uploads by providing a newline-separated file of scrypted auth keys
| ```basicauth = true``` | (optionally) allow basic authorization to upload or paste files from browser when `-authfile` is enabled. When uploading, you will be prompted to enter a user and password - leave the user blank and use your auth key as the password
| ```-authfile path/to/authfile``` | (optionally) require authorization for upload/delete by providing a newline-separated file of scrypted auth keys
| ```-remoteauthfile path/to/remoteauthfile``` | (optionally) require authorization for remote uploads by providing a newline-separated file of scrypted auth keys
| ```-basicauth``` | (optionally) allow basic authorization to upload or paste files from browser when `-authfile` is enabled. When uploading, you will be prompted to enter a user and password - leave the user blank and use your auth key as the password
A helper utility ```linx-genkey``` is provided which hashes keys to the format required in the auth files.
@ -139,25 +107,45 @@ The following storage backends are available:
|Name|Notes|Options
|----|-----|-------
|LocalFS|Enabled by default, this backend uses the filesystem|```filespath = files/``` -- Path to store uploads (default is files/)<br />```metapath = meta/``` -- Path to store information about uploads (default is meta/)|
|S3|Use with any S3-compatible provider.<br> This implementation will stream files through the linx instance (every download will request and stream the file from the S3 bucket). File metadata will be stored as tags on the object in the bucket.<br><br>For high-traffic environments, one might consider using an external caching layer such as described [in this article](https://blog.sentry.io/2017/03/01/dodging-s3-downtime-with-nginx-and-haproxy.html).|```s3-endpoint = https://...``` -- S3 endpoint<br>```s3-region = us-east-1``` -- S3 region<br>```s3-bucket = mybucket``` -- S3 bucket to use for files and metadata<br>```s3-force-path-style = true``` (optional) -- force path-style addresing (e.g. https://<span></span>s3.amazonaws.com/linx/example.txt)<br><br>Environment variables to provide:<br>```AWS_ACCESS_KEY_ID``` -- the S3 access key<br>```AWS_SECRET_ACCESS_KEY ``` -- the S3 secret key<br>```AWS_SESSION_TOKEN``` (optional) -- the S3 session token|
|LocalFS|Enabled by default, this backend uses the filesystem|```-filespath files/``` -- Path to store uploads (default is files/)<br />```-metapath meta/``` -- Path to store information about uploads (default is meta/)|
|S3|Use with any S3-compatible provider.<br> This implementation will stream files through the linx instance (every download will request and stream the file from the S3 bucket).<br><br>For high-traffic environments, one might consider using an external caching layer such as described [in this article](https://blog.sentry.io/2017/03/01/dodging-s3-downtime-with-nginx-and-haproxy.html).|```-s3-endpoint https://...``` -- S3 endpoint<br>```-s3-region us-east-1``` -- S3 region<br>```-s3-bucket mybucket``` -- S3 bucket to use for files and metadata<br>```-s3-force-path-style``` (optional) -- force path-style addresing (e.g. https://<span></span>s3.amazonaws.com/linx/example.txt)<br><br>Environment variables to provide:<br>```AWS_ACCESS_KEY_ID``` -- the S3 access key<br>```AWS_SECRET_ACCESS_KEY ``` -- the S3 secret key<br>```AWS_SESSION_TOKEN``` (optional) -- the S3 session token|
#### SSL with built-in server
|Option|Description
|------|-----------
| ```certfile = path/to/your.crt``` | Path to the ssl certificate (required if you want to use the https server)
| ```keyfile = path/to/your.key``` | Path to the ssl key (required if you want to use the https server)
| ```-certfile path/to/your.crt``` | Path to the ssl certificate (required if you want to use the https server)
| ```-keyfile path/to/your.key``` | Path to the ssl key (required if you want to use the https server)
#### Use with http proxy
|Option|Description
|------|-----------
| ```realip = true``` | let linx-server know you (nginx, etc) are providing the X-Real-IP and/or X-Forwarded-For headers.
| ```-realip``` | let linx-server know you (nginx, etc) are providing the X-Real-IP and/or X-Forwarded-For headers.
#### Use with fastcgi
|Option|Description
|------|-----------
| ```fastcgi = true``` | serve through fastcgi
| ```-fastcgi``` | serve through fastcgi
Cleaning up expired files
-------------------------
When files expire, access is disabled immediately, but the files and metadata
will persist on disk until someone attempts to access them. If you'd like to
automatically clean up files that have expired, you can use the included
`linx-cleanup` utility. To run it automatically, use a cronjob or similar type
of scheduled task.
You should be careful to ensure that only one instance of `linx-client` runs at
a time to avoid unexpected behavior. It does not implement any type of locking.
|Option|Description
|------|-----------
| ```-filespath files/``` | Path to stored uploads (default is files/)
| ```-nologs``` | (optionally) disable deletion logs in stdout
| ```-metapath meta/``` | Path to stored information about uploads (default is meta/)
Deployment
----------
@ -182,16 +170,19 @@ server {
}
}
```
And run linx-server with the ```fastcgi = true``` option.
And run linx-server with the ```-fastcgi``` option.
#### 2. Using the built-in https server
Run linx-server with the ```certfile = path/to/cert.file``` and ```keyfile = path/to/key.file``` options.
Run linx-server with the ```-certfile path/to/cert.file``` and ```-keyfile path/to/key.file``` options.
#### 3. Using the built-in http server
Run linx-server normally.
Development
-----------
Any help is welcome, PRs will be reviewed and merged accordingly.
The official IRC channel is #linx on irc.oftc.net
1. ```go get -u github.com/andreimarcu/linx-server ```
2. ```cd $GOPATH/src/github.com/andreimarcu/linx-server ```
3. ```go build && ./linx-server```
@ -216,4 +207,4 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
Author
-------
Andrei Marcu, https://andreim.net/
Andrei Marcu, http://andreim.net/

View file

@ -1,4 +1,4 @@
package apikeys
package main
import (
"bufio"
@ -24,18 +24,16 @@ const (
type AuthOptions struct {
AuthFile string
UnauthMethods []string
BasicAuth bool
SiteName string
SitePath string
}
type ApiKeysMiddleware struct {
type auth struct {
successHandler http.Handler
failureHandler http.Handler
authKeys []string
o AuthOptions
}
func ReadAuthKeys(authFile string) []string {
func readAuthKeys(authFile string) []string {
var authKeys []string
f, err := os.Open(authFile)
@ -57,7 +55,7 @@ func ReadAuthKeys(authFile string) []string {
return authKeys
}
func CheckAuth(authKeys []string, key string) (result bool, err error) {
func checkAuth(authKeys []string, key string) (result bool, err error) {
checkKey, err := scrypt.Key([]byte(key), []byte(scryptSalt), scryptN, scryptr, scryptp, scryptKeyLen)
if err != nil {
return
@ -75,74 +73,53 @@ func CheckAuth(authKeys []string, key string) (result bool, err error) {
return
}
func (a ApiKeysMiddleware) getSitePrefix() string {
prefix := a.o.SitePath
if len(prefix) <= 0 || prefix[0] != '/' {
prefix = "/" + prefix
}
return prefix
}
func (a ApiKeysMiddleware) goodAuthorizationHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Location", a.getSitePrefix())
w.WriteHeader(http.StatusFound)
}
func (a ApiKeysMiddleware) badAuthorizationHandler(w http.ResponseWriter, r *http.Request) {
if a.o.BasicAuth {
rs := ""
if a.o.SiteName != "" {
rs = fmt.Sprintf(` realm="%s"`, a.o.SiteName)
}
w.Header().Set("WWW-Authenticate", `Basic`+rs)
}
http.Error(w, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
}
func (a ApiKeysMiddleware) ServeHTTP(w http.ResponseWriter, r *http.Request) {
var successHandler http.Handler
prefix := a.getSitePrefix()
if r.URL.Path == prefix+"auth" {
successHandler = http.HandlerFunc(a.goodAuthorizationHandler)
} else {
successHandler = a.successHandler
}
if sliceContains(a.o.UnauthMethods, r.Method) && r.URL.Path != prefix+"auth" {
func (a auth) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if sliceContains(a.o.UnauthMethods, r.Method) {
// allow unauthenticated methods
successHandler.ServeHTTP(w, r)
a.successHandler.ServeHTTP(w, r)
return
}
key := r.Header.Get("Linx-Api-Key")
if key == "" && a.o.BasicAuth {
if key == "" && Config.basicAuth {
_, password, ok := r.BasicAuth()
if ok {
key = password
}
}
result, err := CheckAuth(a.authKeys, key)
result, err := checkAuth(a.authKeys, key)
if err != nil || !result {
http.HandlerFunc(a.badAuthorizationHandler).ServeHTTP(w, r)
a.failureHandler.ServeHTTP(w, r)
return
}
successHandler.ServeHTTP(w, r)
a.successHandler.ServeHTTP(w, r)
}
func NewApiKeysMiddleware(o AuthOptions) func(*web.C, http.Handler) http.Handler {
func UploadAuth(o AuthOptions) func(*web.C, http.Handler) http.Handler {
fn := func(c *web.C, h http.Handler) http.Handler {
return ApiKeysMiddleware{
return auth{
successHandler: h,
authKeys: ReadAuthKeys(o.AuthFile),
failureHandler: http.HandlerFunc(badAuthorizationHandler),
authKeys: readAuthKeys(o.AuthFile),
o: o,
}
}
return fn
}
func badAuthorizationHandler(w http.ResponseWriter, r *http.Request) {
if Config.basicAuth {
rs := ""
if Config.siteName != "" {
rs = fmt.Sprintf(` realm="%s"`, Config.siteName)
}
w.Header().Set("WWW-Authenticate", `Basic`+rs)
}
http.Error(w, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
}
func sliceContains(slice []string, s string) bool {
for _, v := range slice {
if s == v {

View file

@ -1,4 +1,4 @@
package apikeys
package main
import (
"testing"
@ -10,15 +10,15 @@ func TestCheckAuth(t *testing.T) {
"vFpNprT9wbHgwAubpvRxYCCpA2FQMAK6hFqPvAGrdZo=",
}
if r, err := CheckAuth(authKeys, ""); err != nil && r {
if r, err := checkAuth(authKeys, ""); err != nil && r {
t.Fatal("Authorization passed for empty key")
}
if r, err := CheckAuth(authKeys, "thisisnotvalid"); err != nil && r {
if r, err := checkAuth(authKeys, "thisisnotvalid"); err != nil && r {
t.Fatal("Authorization passed for invalid key")
}
if r, err := CheckAuth(authKeys, "haPVipRnGJ0QovA9nyqK"); err != nil && !r {
if r, err := checkAuth(authKeys, "haPVipRnGJ0QovA9nyqK"); err != nil && !r {
t.Fatal("Authorization failed for valid key")
}
}

View file

@ -4,7 +4,6 @@ import (
"encoding/json"
"io"
"io/ioutil"
"net/http"
"os"
"path"
"time"
@ -83,18 +82,6 @@ func (b LocalfsBackend) Get(key string) (metadata backends.Metadata, f io.ReadCl
return
}
func (b LocalfsBackend) ServeFile(key string, w http.ResponseWriter, r *http.Request) (err error) {
_, err = b.Head(key)
if err != nil {
return
}
filePath := path.Join(b.filesPath, key)
http.ServeFile(w, r, filePath)
return
}
func (b LocalfsBackend) writeMetadata(key string, metadata backends.Metadata) error {
metaPath := path.Join(b.metaPath, key)

View file

@ -3,7 +3,6 @@ package s3
import (
"io"
"io/ioutil"
"net/http"
"os"
"strconv"
"time"
@ -80,43 +79,6 @@ func (b S3Backend) Get(key string) (metadata backends.Metadata, r io.ReadCloser,
return
}
func (b S3Backend) ServeFile(key string, w http.ResponseWriter, r *http.Request) (err error) {
var result *s3.GetObjectOutput
if r.Header.Get("Range") != "" {
result, err = b.svc.GetObject(&s3.GetObjectInput{
Bucket: aws.String(b.bucket),
Key: aws.String(key),
Range: aws.String(r.Header.Get("Range")),
})
w.WriteHeader(206)
w.Header().Set("Content-Range", *result.ContentRange)
w.Header().Set("Content-Length", strconv.FormatInt(*result.ContentLength, 10))
w.Header().Set("Accept-Ranges", "bytes")
} else {
result, err = b.svc.GetObject(&s3.GetObjectInput{
Bucket: aws.String(b.bucket),
Key: aws.String(key),
})
}
if err != nil {
if aerr, ok := err.(awserr.Error); ok {
if aerr.Code() == s3.ErrCodeNoSuchKey || aerr.Code() == "NotFound" {
err = backends.NotFoundErr
}
}
return
}
_, err = io.Copy(w, result.Body)
return
}
func mapMetadata(m backends.Metadata) map[string]*string {
return map[string]*string{
"Expiry": aws.String(strconv.FormatInt(m.Expiry.Unix(), 10)),

View file

@ -3,7 +3,6 @@ package backends
import (
"errors"
"io"
"net/http"
"time"
)
@ -14,7 +13,6 @@ type StorageBackend interface {
Get(key string) (Metadata, io.ReadCloser, error)
Put(key string, r io.Reader, expiry time.Time, deleteKey, accessKey string) (Metadata, error)
PutMetadata(key string, m Metadata) error
ServeFile(key string, w http.ResponseWriter, r *http.Request) error
Size(key string) (int64, error)
}

View file

@ -122,7 +122,6 @@ func fileDisplayHandler(c web.C, w http.ResponseWriter, r *http.Request, fileNam
"forcerandom": Config.forceRandomFilename,
"lines": lines,
"files": metadata.ArchiveFiles,
"siteurl": strings.TrimSuffix(getSiteURL(r), "/"),
}, r, w)
if err != nil {

View file

@ -2,6 +2,7 @@ package main
import (
"fmt"
"io"
"net/http"
"net/url"
"strconv"
@ -60,12 +61,16 @@ func fileServeHandler(c web.C, w http.ResponseWriter, r *http.Request) {
}
if r.Method != "HEAD" {
storageBackend.ServeFile(fileName, w, r)
_, reader, err := storageBackend.Get(fileName)
if err != nil {
oopsHandler(c, w, r, RespAUTO, err.Error())
oopsHandler(c, w, r, RespAUTO, "Unable to open file.")
return
}
defer reader.Close()
if _, err = io.CopyN(w, reader, metadata.Size); err != nil {
oopsHandler(c, w, r, RespAUTO, err.Error())
}
}
}

2
go.mod
View file

@ -8,7 +8,6 @@ require (
github.com/dchest/uniuri v0.0.0-20200228104902-7aecb25e1fe5
github.com/dustin/go-humanize v1.0.0
github.com/flosch/pongo2 v0.0.0-20190707114632-bbf5a6c351f4
github.com/gabriel-vasile/mimetype v1.1.1
github.com/microcosm-cc/bluemonday v1.0.2
github.com/minio/sha256-simd v0.1.1
github.com/russross/blackfriday v1.5.1
@ -16,4 +15,5 @@ require (
github.com/zeebo/bencode v1.0.0
github.com/zenazn/goji v0.9.0
golang.org/x/crypto v0.0.0-20200302210943-78000ba7a073
gopkg.in/h2non/filetype.v1 v1.0.5
)

4
go.sum
View file

@ -15,8 +15,6 @@ github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/flosch/pongo2 v0.0.0-20190707114632-bbf5a6c351f4 h1:GY1+t5Dr9OKADM64SYnQjw/w99HMYvQ0A8/JoUkxVmc=
github.com/flosch/pongo2 v0.0.0-20190707114632-bbf5a6c351f4/go.mod h1:T9YF2M40nIgbVgp3rreNmTged+9HrbNTIQf1PsaIiTA=
github.com/gabriel-vasile/mimetype v1.1.1 h1:qbN9MPuRf3bstHu9zkI9jDWNfH//9+9kHxr9oRBBBOA=
github.com/gabriel-vasile/mimetype v1.1.1/go.mod h1:6CDPel/o/3/s4+bp6kIbsWATq8pmgOisOPG40CJa6To=
github.com/go-check/check v0.0.0-20180628173108-788fd7840127 h1:0gkP6mzaMqkmpcJYCFOLkIBwI7xFExG03bbkOkCvUPI=
github.com/go-check/check v0.0.0-20180628173108-788fd7840127/go.mod h1:9ES+weclKsC9YodN5RgxqK/VD9HM9JsCSh7rNhMZE98=
github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
@ -70,6 +68,8 @@ golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/tools v0.0.0-20181221001348-537d06c36207/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/h2non/filetype.v1 v1.0.5 h1:CC1jjJjoEhNVbMhXYalmGBhOBK2V70Q1N850wt/98/Y=
gopkg.in/h2non/filetype.v1 v1.0.5/go.mod h1:M0yem4rwSX5lLVrkEuRRp2/NinFMD5vgJ4DlAhZcfNo=
gopkg.in/mgo.v2 v2.0.0-20180705113604-9856a29383ce h1:xcEWjVhvbDy+nHP67nPDDpbYrY+ILlfndk4bRioVHaU=
gopkg.in/mgo.v2 v2.0.0-20180705113604-9856a29383ce/go.mod h1:yeKp02qBN3iKW1OzL3MGk2IdtZzaj7SFntXj72NppTA=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=

View file

@ -7,8 +7,8 @@ import (
"unicode"
"github.com/andreimarcu/linx-server/backends"
"github.com/gabriel-vasile/mimetype"
"github.com/minio/sha256-simd"
"gopkg.in/h2non/filetype.v1"
)
func GenerateMetadata(r io.Reader) (m backends.Metadata, err error) {
@ -21,7 +21,7 @@ func GenerateMetadata(r io.Reader) (m backends.Metadata, err error) {
// Get first 512 bytes for mimetype detection
header := make([]byte, 512)
headerlen, err := teeReader.Read(header)
_, err = teeReader.Read(header)
if err != nil {
return
}
@ -47,8 +47,17 @@ func GenerateMetadata(r io.Reader) (m backends.Metadata, err error) {
// Use the bytes we extracted earlier and attempt to determine the file
// type
kind := mimetype.Detect(header[:headerlen])
m.Mimetype = kind.String()
kind, err := filetype.Match(header)
if err != nil {
m.Mimetype = "application/octet-stream"
return m, err
} else if kind.MIME.Value != "" {
m.Mimetype = kind.MIME.Value
} else if printable(header) {
m.Mimetype = "text/plain"
} else {
m.Mimetype = "application/octet-stream"
}
return
}

View file

@ -1,10 +1,8 @@
package helpers
import (
"bytes"
"strings"
"testing"
"unicode/utf16"
)
func TestGenerateMetadata(t *testing.T) {
@ -19,7 +17,7 @@ func TestGenerateMetadata(t *testing.T) {
t.Fatalf("Sha256sum was %q instead of expected value of %q", m.Sha256sum, expectedSha256sum)
}
expectedMimetype := "text/plain; charset=utf-8"
expectedMimetype := "text/plain"
if m.Mimetype != expectedMimetype {
t.Fatalf("Mimetype was %q instead of expected value of %q", m.Mimetype, expectedMimetype)
}
@ -29,45 +27,3 @@ func TestGenerateMetadata(t *testing.T) {
t.Fatalf("Size was %d instead of expected value of %d", m.Size, expectedSize)
}
}
func TestTextCharsets(t *testing.T) {
// verify that different text encodings are detected and passed through
orig := "This is a text string"
utf16 := utf16.Encode([]rune(orig))
utf16LE := make([]byte, len(utf16)*2+2)
utf16BE := make([]byte, len(utf16)*2+2)
utf8 := []byte(orig)
utf16LE[0] = 0xff
utf16LE[1] = 0xfe
utf16BE[0] = 0xfe
utf16BE[1] = 0xff
for i := 0; i < len(utf16); i++ {
lsb := utf16[i] & 0xff
msb := utf16[i] >> 8
utf16LE[i*2+2] = byte(lsb)
utf16LE[i*2+3] = byte(msb)
utf16BE[i*2+2] = byte(msb)
utf16BE[i*2+3] = byte(lsb)
}
testcases := []struct {
data []byte
extension string
mimetype string
}{
{mimetype: "text/plain; charset=utf-8", data: utf8},
{mimetype: "text/plain; charset=utf-16le", data: utf16LE},
{mimetype: "text/plain; charset=utf-16be", data: utf16BE},
}
for i, testcase := range testcases {
r := bytes.NewReader(testcase.data)
m, err := GenerateMetadata(r)
if err != nil {
t.Fatalf("[%d] unexpected error return %v\n", i, err)
}
if m.Mimetype != testcase.mimetype {
t.Errorf("[%d] Expected mimetype '%s', got mimetype '%s'\n", i, testcase.mimetype, m.Mimetype)
}
}
}

View file

@ -1,19 +0,0 @@
linx-cleanup
-------------------------
When files expire, access is disabled immediately, but the files and metadata
will persist on disk until someone attempts to access them.
If you'd like to automatically clean up files that have expired, you can use the included `linx-cleanup` utility. To run it automatically, use a cronjob or similar type
of scheduled task.
You should be careful to ensure that only one instance of `linx-cleanup` runs at
a time to avoid unexpected behavior. It does not implement any type of locking.
|Option|Description
|------|-----------
| ```-filespath files/``` | Path to stored uploads (default is files/)
| ```-nologs``` | (optionally) disable deletion logs in stdout
| ```-metapath meta/``` | Path to stored information about uploads (default is meta/)

View file

@ -1,14 +1,26 @@
package cleanup
package main
import (
"flag"
"log"
"time"
"github.com/andreimarcu/linx-server/backends/localfs"
"github.com/andreimarcu/linx-server/expiry"
)
func Cleanup(filesDir string, metaDir string, noLogs bool) {
func main() {
var filesDir string
var metaDir string
var noLogs bool
flag.StringVar(&filesDir, "filespath", "files/",
"path to files directory")
flag.StringVar(&metaDir, "metapath", "meta/",
"path to metadata directory")
flag.BoolVar(&noLogs, "nologs", false,
"don't log deleted files")
flag.Parse()
fileBackend := localfs.NewLocalfsBackend(metaDir, filesDir)
files, err := fileBackend.List()
@ -32,11 +44,3 @@ func Cleanup(filesDir string, metaDir string, noLogs bool) {
}
}
}
func PeriodicCleanup(minutes time.Duration, filesDir string, metaDir string, noLogs bool) {
c := time.Tick(minutes)
for range c {
Cleanup(filesDir, metaDir, noLogs)
}
}

View file

@ -1,23 +0,0 @@
package main
import (
"flag"
"github.com/andreimarcu/linx-server/cleanup"
)
func main() {
var filesDir string
var metaDir string
var noLogs bool
flag.StringVar(&filesDir, "filespath", "files/",
"path to files directory")
flag.StringVar(&metaDir, "metapath", "meta/",
"path to metadata directory")
flag.BoolVar(&noLogs, "nologs", false,
"don't log deleted files")
flag.Parse()
cleanup.Cleanup(filesDir, metaDir, noLogs)
}

View file

@ -1,12 +0,0 @@
[Unit]
Description=Self-hosted file/code/media sharing (expired files cleanup)
After=network.target
[Service]
User=linx
Group=linx
ExecStart=/usr/bin/linx-cleanup
WorkingDirectory=/srv/linx/
[Install]
WantedBy=multi-user.target

View file

@ -1,8 +0,0 @@
[Unit]
Description=Run linx-cleanup every hour
[Timer]
OnUnitActiveSec=1h
[Install]
WantedBy=timers.target

View file

@ -1,12 +0,0 @@
bind = 127.0.0.1:8080
sitename = myLinx
siteurl = https://mylinx.example.org/
selifpath = s
maxsize = 4294967296
maxexpiry = 86400
allowhotlink = true
remoteuploads = true
nologs = true
force-random-filename = false
cleanup-every-minutes = 5

View file

@ -16,11 +16,9 @@ import (
"time"
rice "github.com/GeertJohan/go.rice"
"github.com/andreimarcu/linx-server/auth/apikeys"
"github.com/andreimarcu/linx-server/backends"
"github.com/andreimarcu/linx-server/backends/localfs"
"github.com/andreimarcu/linx-server/backends/s3"
"github.com/andreimarcu/linx-server/cleanup"
"github.com/flosch/pongo2"
"github.com/vharitonsky/iniflags"
"github.com/zenazn/goji/graceful"
@ -73,7 +71,6 @@ var Config struct {
forceRandomFilename bool
accessKeyCookieExpiry uint64
customPagesDir string
cleanupEveryMinutes uint64
}
var Templates = make(map[string]*pongo2.Template)
@ -111,12 +108,9 @@ func setup() *web.Mux {
mux.Use(AddHeaders(Config.addHeaders))
if Config.authFile != "" {
mux.Use(apikeys.NewApiKeysMiddleware(apikeys.AuthOptions{
mux.Use(UploadAuth(AuthOptions{
AuthFile: Config.authFile,
UnauthMethods: []string{"GET", "HEAD", "OPTIONS", "TRACE"},
BasicAuth: Config.basicAuth,
SiteName: Config.siteName,
SitePath: Config.sitePath,
}))
}
@ -156,10 +150,6 @@ func setup() *web.Mux {
storageBackend = s3.NewS3Backend(Config.s3Bucket, Config.s3Region, Config.s3Endpoint, Config.s3ForcePathStyle)
} else {
storageBackend = localfs.NewLocalfsBackend(Config.metaDir, Config.filesDir)
if Config.cleanupEveryMinutes > 0 {
go cleanup.PeriodicCleanup(time.Duration(Config.cleanupEveryMinutes)*time.Minute, Config.filesDir, Config.metaDir, Config.noLogs)
}
}
// Template setup
@ -200,10 +190,29 @@ func setup() *web.Mux {
mux.Get(Config.sitePath+"upload/", uploadRemote)
if Config.remoteAuthFile != "" {
remoteAuthKeys = apikeys.ReadAuthKeys(Config.remoteAuthFile)
remoteAuthKeys = readAuthKeys(Config.remoteAuthFile)
}
}
if Config.basicAuth {
options := AuthOptions{
AuthFile: Config.authFile,
UnauthMethods: []string{},
}
okFunc := func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Location", Config.sitePath)
w.WriteHeader(http.StatusFound)
}
authHandler := auth{
successHandler: http.HandlerFunc(okFunc),
failureHandler: http.HandlerFunc(badAuthorizationHandler),
authKeys: readAuthKeys(Config.authFile),
o: options,
}
mux.Head(Config.sitePath+"auth", authHandler)
mux.Get(Config.sitePath+"auth", authHandler)
}
mux.Post(Config.sitePath+"upload", uploadPostHandler)
mux.Post(Config.sitePath+"upload/", uploadPostHandler)
mux.Put(Config.sitePath+"upload", uploadPutHandler)
@ -302,8 +311,6 @@ func main() {
flag.Uint64Var(&Config.accessKeyCookieExpiry, "access-cookie-expiry", 0, "Expiration time for access key cookies in seconds (set 0 to use session cookies)")
flag.StringVar(&Config.customPagesDir, "custompagespath", "",
"path to directory containing .md files to render as custom pages")
flag.Uint64Var(&Config.cleanupEveryMinutes, "cleanup-every-minutes", 0,
"How often to clean up expired files in minutes (default is 0, which means files will be cleaned up as they are accessed)")
iniflags.Parse()

View file

@ -1,12 +1,9 @@
{% extends "base.html" %}
{% block head %}
<meta property="og:audio" content="{{ siteurl }}{{ sitepath }}{{ selifpath }}{{ filename }}" />
{% endblock %}
{% block main %}
<audio class="display-audio" controls preload='auto'>
<source src='{{ sitepath }}{{ selifpath }}{{ filename }}'>
<a href='{{ sitepath }}{{ selifpath }}{{ filename }}'>Download it instead</a>
</audio>
{% endblock %}
{% endblock %}

View file

@ -1,11 +1,7 @@
{% extends "base.html" %}
{% block head %}
<meta property="og:image" content="{{ siteurl }}{{ sitepath }}{{ selifpath }}{{ filename }}" />
{% endblock %}
{% block main %}
<a href="{{ sitepath }}{{ selifpath }}{{ filename }}">
<img class="display-image" src="{{ sitepath }}{{ selifpath }}{{ filename }}" />
</a>
{% endblock %}
{% endblock %}

View file

@ -1,12 +1,8 @@
{% extends "base.html" %}
{% block head %}
<meta property="og:video" content="{{ siteurl }}{{ sitepath }}{{ selifpath }}{{ filename }}" />
{% endblock %}
{% block main %}
<video class="display-video" controls autoplay>
<source src="{{ sitepath }}{{ selifpath }}{{ filename }}" />
<source src="{{ sitepath }}{{ selifpath }}{{ filename }}"/>
<a href='{{ sitepath }}{{ selifpath }}{{ filename }}'>Download it instead</a>
</video>
{% endblock %}
{% endblock %}

View file

@ -15,12 +15,11 @@ import (
"strings"
"time"
"github.com/andreimarcu/linx-server/auth/apikeys"
"github.com/andreimarcu/linx-server/backends"
"github.com/andreimarcu/linx-server/expiry"
"github.com/dchest/uniuri"
"github.com/gabriel-vasile/mimetype"
"github.com/zenazn/goji/web"
"gopkg.in/h2non/filetype.v1"
)
var FileTooLargeError = errors.New("File too large.")
@ -167,16 +166,13 @@ func uploadRemote(c web.C, w http.ResponseWriter, r *http.Request) {
key = password
}
}
result, err := apikeys.CheckAuth(remoteAuthKeys, key)
result, err := checkAuth(remoteAuthKeys, key)
if err != nil || !result {
if Config.basicAuth {
rs := ""
if Config.siteName != "" {
rs = fmt.Sprintf(` realm="%s"`, Config.siteName)
}
w.Header().Set("WWW-Authenticate", `Basic`+rs)
badAuthorizationHandler(w, r)
} else {
unauthorizedHandler(c, w, r)
}
unauthorizedHandler(c, w, r)
return
}
}
@ -267,11 +263,11 @@ func processUpload(upReq UploadRequest) (upload Upload, err error) {
header = header[:n]
// Determine the type of file from header
kind := mimetype.Detect(header)
if len(kind.Extension()) < 2 {
kind, err := filetype.Match(header)
if err != nil || kind.Extension == "unknown" {
extension = "file"
} else {
extension = kind.Extension()[1:] // remove leading "."
extension = kind.Extension
}
}