mirror of
https://github.com/JamesTurland/JimsGarage.git
synced 2024-11-25 01:10:22 +00:00
Merge branch 'main' of https://github.com/JamesTurland/JimsGarage
This commit is contained in:
James Turland
commit
b49f5dc82d
4 changed files with 104 additions and 53 deletions
|
|
@ -4,39 +4,51 @@ Intel VT-d & VT-x – Intel Compatible list
|
|||
All AMD CPUs from Bulldozer onwards should be compatible.
|
||||
|
||||
2) Get device IDs:
|
||||
lspci -nn
|
||||
`lspci -nn`
|
||||
|
||||
3) Enable IOMMU in GRUB (check Intel or AMD commands below - choose the right one)
|
||||
nano /etc/default/grub
|
||||
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
|
||||
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
|
||||
save and exit
|
||||
|
||||
5) run the command "update-grub"
|
||||
now reboot
|
||||
`GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"`
|
||||
|
||||
6) Enable VFIO Modules
|
||||
nano /etc/modules
|
||||
`GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"`
|
||||
|
||||
Verify that IOMMU is enabled by running `dmesg | grep -e DMAR -e IOMMU` and looking for a line indicating it is enabled
|
||||
|
||||
Save and exit
|
||||
|
||||
4) run the command `"update-grub"` now reboot
|
||||
|
||||
5) Enable VFIO Modules
|
||||
`nano /etc/modules`
|
||||
Add the following modules:
|
||||
```
|
||||
vfio
|
||||
vfio_iommu_type1
|
||||
vfio_pci
|
||||
vfio_virqfd
|
||||
```
|
||||
Then, save and exit
|
||||
|
||||
Next run:
|
||||
update-initramfs -u -k all
|
||||
and reboot
|
||||
6) `update-initramfs -u -k all` and reboot
|
||||
|
||||
7) GPU Isolation From the Host (amend the below to include the IDs of the device you want to isolate)
|
||||
echo "options vfio-pci ids=10de:1381,10de:0fbc disable_vga=1" > /etc/modprobe.d/vfio.conf
|
||||
Verify the modules are enabled with `dmesg | grep -i vfio` and checking the driver version line is present
|
||||
|
||||
8) GPU Isolation From the Host (amend the below to include the IDs of the device you want to isolate)
|
||||
|
||||
`echo "options vfio-pci ids=10de:1381,10de:0fbc disable_vga=1" > /etc/modprobe.d/vfio.conf`
|
||||
|
||||
8) Blacklist GPU drivers (here are all that you would ever need)
|
||||
```
|
||||
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
|
||||
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
|
||||
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
|
||||
echo "blacklist nvidiafb" >> /etc/modprobe.d/blacklist.conf
|
||||
echo "blacklist nvidia_drm" >> /etc/modprobe.d/blacklist.conf
|
||||
```
|
||||
|
||||
Reboot your machine, be sure to use the web gui because with the gpu passed through you will no longer get video out.
|
||||
The start up will begin but then appear to hang even though proxmox has started fine.
|
||||
|
||||
9) Create a new VM and add the GPU via hardware menu
|
||||
You may need to set it as primary GPU
|
||||
|
|
|
|||
|
|
@ -9,10 +9,11 @@ services:
|
|||
- /home/ubuntu/docker/immich/upload:/usr/src/app/upload
|
||||
env_file:
|
||||
- .env
|
||||
ports:
|
||||
- 2283:3001
|
||||
depends_on:
|
||||
- redis
|
||||
- database
|
||||
- typesense
|
||||
restart: always
|
||||
|
||||
immich-microservices:
|
||||
|
|
@ -29,7 +30,6 @@ services:
|
|||
depends_on:
|
||||
- redis
|
||||
- database
|
||||
- typesense
|
||||
restart: always
|
||||
|
||||
immich-machine-learning:
|
||||
|
|
@ -41,25 +41,6 @@ services:
|
|||
- .env
|
||||
restart: always
|
||||
|
||||
immich-web:
|
||||
container_name: immich_web
|
||||
image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release}
|
||||
env_file:
|
||||
- .env
|
||||
restart: always
|
||||
|
||||
typesense:
|
||||
container_name: immich_typesense
|
||||
image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
|
||||
environment:
|
||||
- TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
|
||||
- TYPESENSE_DATA_DIR=/data
|
||||
# remove this to get debug messages
|
||||
- GLOG_minloglevel=1
|
||||
volumes:
|
||||
- /home/ubuntu/docker/immich/tsdata:/data
|
||||
restart: always
|
||||
|
||||
redis:
|
||||
container_name: immich_redis
|
||||
image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
|
||||
|
|
@ -67,7 +48,7 @@ services:
|
|||
|
||||
database:
|
||||
container_name: immich_postgres
|
||||
image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441
|
||||
image: tensorchord/pgvecto-rs:pg14-v0.1.11
|
||||
env_file:
|
||||
- .env
|
||||
environment:
|
||||
|
|
@ -78,18 +59,4 @@ services:
|
|||
- /home/ubuntu/docker/immich/pgdata:/var/lib/postgresql/data
|
||||
restart: always
|
||||
|
||||
immich-proxy:
|
||||
container_name: immich_proxy
|
||||
image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release}
|
||||
environment:
|
||||
# Make sure these values get passed through from the env file
|
||||
- IMMICH_SERVER_URL
|
||||
- IMMICH_WEB_URL
|
||||
ports:
|
||||
- 2283:8080
|
||||
depends_on:
|
||||
- immich-server
|
||||
- immich-web
|
||||
restart: always
|
||||
|
||||
## there is a known issue with Traefik: see here https://github.com/immich-app/immich/discussions/437#discussioncomment-3609797
|
||||
|
|
|
|||
|
|
@ -42,3 +42,73 @@ kubectl get svc -n cattle-system
|
|||
# Go to Rancher GUI
|
||||
Hit the url… and create your account
|
||||
Be patient as it downloads and configures a number of pods in the background to support the UI (can be 5-10mins)
|
||||
|
||||
# Bonus: Accessing Rancher through Ingress (Traefik)
|
||||
Do you want that precious green lock in your URL bar? If you have Traefik (or another Kubernetes Ingress controller)
|
||||
deployed and a Let's Encrypt issuer with Cert-Manager, the Rancher Helm chart offers support to automatically configure
|
||||
an ingress route with TLS certificate injection to access the Rancher UI via the `rancher` Kubernetes Service created
|
||||
by the Helm install (`kubectl -n cattle-system get service`). This can be configured retroactively after your initial
|
||||
`helm install ...`, but is a bit simpler to set up as part of your initial Rancher installation if you have the
|
||||
prerequisites in place.
|
||||
|
||||
You first need to save the TLS certificate and key that you want Traefik to use for Rancher as a Kubernetes Secret
|
||||
called `tls-rancher-ingress`. You can do this manually, or let Cert-Manager generate a certificate for you and
|
||||
store it in a Secret, using `kubectl` to create a Kubernetes Certificate resource to generate the certificate and
|
||||
populate the Secret for you:
|
||||
```yaml
|
||||
---
|
||||
apiVersion: cert-manager.io/v1
|
||||
kind: Certificate
|
||||
metadata:
|
||||
name: rancher-my-org
|
||||
namespace: cattle-system
|
||||
spec:
|
||||
commonName: rancher.my.org
|
||||
dnsNames:
|
||||
- rancher.my.org
|
||||
issuerRef:
|
||||
name: letsencrypt-production
|
||||
kind: ClusterIssuer
|
||||
secretName: tls-rancher-ingress
|
||||
```
|
||||
This can take anywhere from a few minutes to 15-20 minutes to generate, so sit tight. You'll know it is ready when
|
||||
the certificate's `Ready` status shows `True` in the output of:
|
||||
```bash
|
||||
kubectl -n cattle-system get certificate rancher-my-org
|
||||
```
|
||||
While you're waiting, make sure that your DNS record for `rancher.my.org` points to your Traefik deployment,
|
||||
instead of a LoadBalancer IP from kubeVIP.
|
||||
This can be an CNAME record using your Traefik FQDN, or an A record using the same IP address as Traefik.
|
||||
|
||||
Once the certificate and DNS record are ready, you can run your Rancher installation with one extra value set to
|
||||
configure your Ingress provider to use your custom certificate:
|
||||
```bash
|
||||
helm install rancher rancher-latest/rancher \
|
||||
--namespace cattle-system \
|
||||
--set hostname=rancher.my.org \
|
||||
--set bootstrapPassword=admin \
|
||||
--set ingress.tls.source=secret
|
||||
```
|
||||
If you are doing this after already installing Rancher (with the default setting of `ingress.tls.source=rancher`),
|
||||
you can overwrite the self-generated `tls-rancher-ingress` secret with your own certificate, then update your
|
||||
deployment. You may want to get your current Rancher version using `helm ls -n cattle-system` and provide it
|
||||
in your `helm upgrade` command so you don't unexpectedly upgrade your Rancher version.
|
||||
```bash
|
||||
helm upgrade rancher rancher-stable/rancher \
|
||||
--namespace cattle-system \
|
||||
--set hostname=rancher.my.org \
|
||||
--set ingress.tls.source=secret \
|
||||
--version <DEPLOYED_RANCHER_VERSION>
|
||||
```
|
||||
## Ingress TLS Troubleshooting
|
||||
You can validate the contents of your `tls-rancher-ingress` Secret using commands like this:
|
||||
```bash
|
||||
kubectl -n cattle-system get secret tls-rancher-ingress -o jsonpath='{.data}' | jq '."tls.crt"' | tr -d '"' | base64 --decode | openssl x509 -text
|
||||
```
|
||||
If you previously had your `rancher.my.org` DNS record associated with your LoadBalancer IP, your browser may be caching
|
||||
that old record. You may need to clear your browser's DNS cache, use an Incognito/Private window, etc.
|
||||
|
||||
There are also helpful instructions covering a handful of situations in Rancher's documentation:
|
||||
* [Adding TLS Secrets](https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/resources/add-tls-secrets)
|
||||
* [Update Rancher Certificate](https://ranchermanager.docs.rancher.com/getting-started/installation-and-upgrade/resources/update-rancher-certificate)
|
||||
|
||||
|
|
|
|||
|
|
@ -29,18 +29,20 @@ services:
|
|||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.traefik.entrypoints=http" # restricts dashboard to internal entrypoint
|
||||
- "traefik.http.routers.traefik.rule=Host(`traefik-docker.yourdomain.com`)"
|
||||
- "traefik.http.routers.traefik.rule=Host(`traefik-dashboard.yourdomain.co.uk`)" # if you want a internal domain, get the wildcard cert for it and then choos traefik-dashboard.home.yourdomain.co.uk or what you want
|
||||
- "traefik.http.middlewares.traefik-auth.basicauth.users=admin:password-hash"
|
||||
- "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
|
||||
- "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
|
||||
- "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
|
||||
- "traefik.http.routers.traefik-secure.entrypoints=https"
|
||||
- "traefik.http.routers.traefik-secure.rule=Host(`traefik-docker.yourdomain.com`)"
|
||||
- "traefik.http.routers.traefik-secure.rule=Host(`traefik-dashboard.yourdomain.co.uk`)" # if you want a internal domain, get the wildcard cert for it and then choos traefik-dashboard.home.yourdomain.co.uk or what you want
|
||||
- "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
|
||||
- "traefik.http.routers.traefik-secure.tls=true"
|
||||
- "traefik.http.routers.traefik-secure.tls.certresolver=cloudflare"
|
||||
- "traefik.http.routers.traefik-secure.tls.domains[0].main=yourdomain.co.uk"
|
||||
- "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.yourdomain.co.uk"
|
||||
#- "traefik.http.routers.traefik-secure.tls.domains[0].main=home.yourdomain.co.uk" # If you want *.home.yourdomain.co.uk subdomain or something else, you have to get the certifcates at first
|
||||
#- "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.home.yourdomain.co.uk" # get a wildcard certificat for your .home.yourdomain.co.uk
|
||||
- "traefik.http.routers.traefik-secure.tls.domains[0].main=yourdomain.co.uk" #if you use the .home.yourdomain.co.uk entry you have to change the [0] into [1]
|
||||
- "traefik.http.routers.traefik-secure.tls.domains[0].sans=*.yourdomain.co.uk" # same here, change 0 to 1
|
||||
- "traefik.http.routers.traefik-secure.service=api@internal"
|
||||
|
||||
networks:
|
||||
|
|
|
|||
Loading…
Reference in a new issue