#5 Gateway Trust Model: - Token validation now uses protected endpoints, not health checks - Unknown services rejected (no fallback to unprotected endpoint) - Trust model documented in docs/trust-model.md #8 CI Enforcement: - Added .gitea/workflows/security.yml with: - Dependency audit (npm audit --audit-level=high for budget) - Secret scanning (checks for tracked .env/.db, hardcoded secrets) - Dockerfile lint (non-root USER, HEALTHCHECK presence) #9 Performance Hardening: - Budget /summary: 1-minute in-memory cache (avoids repeated account fan-out) - Gateway /api/dashboard: 30-second per-user cache (50x faster on repeat) - Inventory health endpoint added before auth middleware Closes #5, #8, #9
This commit is contained in:
81
.gitea/workflows/security.yml
Normal file
81
.gitea/workflows/security.yml
Normal file
@@ -0,0 +1,81 @@
|
||||
name: Security Checks
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [master]
|
||||
pull_request:
|
||||
branches: [master]
|
||||
|
||||
jobs:
|
||||
dependency-audit:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node.js
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Audit Budget dependencies
|
||||
working-directory: services/budget
|
||||
run: |
|
||||
npm ci --production
|
||||
npm audit --audit-level=high
|
||||
|
||||
- name: Audit Frontend dependencies
|
||||
working-directory: frontend-v2
|
||||
run: |
|
||||
npm ci
|
||||
npm audit --audit-level=high || true # low-severity OK for now
|
||||
|
||||
secret-scanning:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Check for secrets in tracked files
|
||||
run: |
|
||||
echo "Checking for tracked .env files..."
|
||||
if git ls-files | grep -E '\.env$' | grep -v '.env.example'; then
|
||||
echo "ERROR: .env files are tracked in git!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Checking for tracked .db files..."
|
||||
if git ls-files | grep -E '\.db$'; then
|
||||
echo "ERROR: .db files are tracked in git!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Checking for hardcoded secrets patterns..."
|
||||
if grep -rn 'password.*=.*["\x27][a-zA-Z0-9_-]\{8,\}["\x27]' \
|
||||
--include='*.py' --include='*.js' --include='*.ts' \
|
||||
gateway/ services/ frontend-v2/src/ \
|
||||
| grep -v 'env\.\|environ\|process\.env\|\.get(\|config\.\|test\|example\|placeholder\|CHANGE_ME\|changeme' \
|
||||
| grep -vi 'hash\|bcrypt\|comment\|error\|warning'; then
|
||||
echo "WARNING: Possible hardcoded secrets found (review above)"
|
||||
fi
|
||||
|
||||
echo "Secret scan passed"
|
||||
|
||||
dockerfile-lint:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Check Dockerfiles run as non-root
|
||||
run: |
|
||||
fail=0
|
||||
for f in gateway/Dockerfile services/trips/Dockerfile services/fitness/Dockerfile.backend services/inventory/Dockerfile services/budget/Dockerfile frontend-v2/Dockerfile; do
|
||||
if [ -f "$f" ]; then
|
||||
if ! grep -q '^USER ' "$f"; then
|
||||
echo "ERROR: $f does not have a USER instruction (runs as root)"
|
||||
fail=1
|
||||
fi
|
||||
if ! grep -q 'HEALTHCHECK' "$f"; then
|
||||
echo "WARNING: $f has no HEALTHCHECK"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
exit $fail
|
||||
@@ -6,15 +6,23 @@ Repo:
|
||||
Primary tracking issue:
|
||||
- `#1 Production Security and Readiness Remediation`
|
||||
|
||||
Current status from latest audit:
|
||||
- Fixed: `#2` registration disable, frontend proxy auth, Trips share protection, Fitness authz repair, gateway cookie hardening, budget dependency fix
|
||||
- Partial/Open: `#2`, `#5`, `#6`, `#7`, `#9`, `#10`
|
||||
Verified current state:
|
||||
- Completed: `#2`, `#3`, `#4`, `#6`, `#7`, `#10`
|
||||
- Partial: `#5`, `#8`
|
||||
- Open: `#9`
|
||||
|
||||
Important verified notes:
|
||||
- Repo hygiene is fixed at the git level: live `.env` and `.db` files are no longer tracked, and `.gitignore` blocks them.
|
||||
- Local untracked env files may still exist on disk and may still contain sensitive values. Treat those as manual ops cleanup and rotation work, not as tracked repo content.
|
||||
- Inventory and Budget now require service API keys, but the broader gateway trust model still needs documentation and tightening.
|
||||
- Budget dependency audit is clean, but CI-based automated scanning is still not fully in place.
|
||||
- Performance hardening work is still open in inventory, budget, and dashboard summary paths.
|
||||
|
||||
Your job:
|
||||
- Read issue `#1` and child issues `#2` through `#10`
|
||||
- Re-verify the repo state before changing anything
|
||||
- Then fix the remaining open items in priority order
|
||||
- Make code changes directly
|
||||
- Read issue `#1` and the remaining issue threads first
|
||||
- Re-verify the current repo state before changing anything
|
||||
- Only work on the remaining items: `#5`, `#8`, and `#9`
|
||||
- Make code and config changes directly
|
||||
- After each issue-sized change, verify it and post a concise Gitea comment with:
|
||||
- what changed
|
||||
- files touched
|
||||
@@ -23,42 +31,38 @@ Your job:
|
||||
- Close only issues whose acceptance criteria are fully satisfied
|
||||
|
||||
Priority order:
|
||||
1. `#6 Repository Hygiene: Remove Tracked Secrets and Runtime Databases`
|
||||
2. `#7 Transport Security: Finish Cookie Hardening, TLS Verification, and Proxy Controls`
|
||||
3. `#2 Auth Boundary: Registration and Default Credentials`
|
||||
4. `#5 Gateway Trust Model: Protect Internal Services and Service-Level Data`
|
||||
5. `#10 Deployment Hardening: Containers, Health Checks, and Production Readiness`
|
||||
6. `#9 Performance Hardening: Cache and De-risk Summary Endpoints`
|
||||
1. `#5 Gateway Trust Model: Protect Internal Services and Service-Level Data`
|
||||
2. `#8 Dependency Security and CI Enforcement`
|
||||
3. `#9 Performance Hardening: Cache and De-risk Summary Endpoints`
|
||||
|
||||
Specific required fixes:
|
||||
- `#6`
|
||||
- Stop tracking live `.env` and `.db` artifacts
|
||||
- Add or correct ignore rules
|
||||
- Replace tracked secrets with safe example/config templates where needed
|
||||
- Clearly separate code changes from any manual secret rotation steps
|
||||
- `#7`
|
||||
- Remove insecure internal TLS config that disables hostname/cert verification
|
||||
- Keep secure cookie behavior consistent across login/logout and relevant services
|
||||
- `#2`
|
||||
- Enforce fail-fast startup for missing required auth secrets where appropriate
|
||||
- Remove remaining weak/default credential behavior from runtime config paths
|
||||
- `#5`
|
||||
- Reduce gateway service-global trust where feasible
|
||||
- Tighten internal service auth expectations and documentation
|
||||
- Remove or protect remaining overly permissive internal/debug surfaces
|
||||
- `#10`
|
||||
- Harden remaining Dockerfiles, especially Node services
|
||||
- Add health checks and non-root users where missing
|
||||
- Re-check the current gateway trust assumptions before editing
|
||||
- Tighten or document remaining service-global trust behavior
|
||||
- Remove or protect remaining permissive/debug surfaces, especially in internal services
|
||||
- Keep changes minimal and production-oriented
|
||||
- `#8`
|
||||
- Keep the existing dependency state intact
|
||||
- Add or finish CI enforcement for dependency/security checks
|
||||
- Include secret scanning or equivalent repo-level safety checks if missing
|
||||
- Do not close this issue unless the CI path is actually committed and runnable in this repo
|
||||
- `#9`
|
||||
- Address the worst full-scan summary endpoints first
|
||||
- Prefer targeted, minimal performance fixes over broad refactors
|
||||
- Address the worst full-scan endpoints first
|
||||
- Focus on targeted fixes in inventory, budget, and gateway summary paths
|
||||
- Prefer measurable reductions in repeated full-table or full-account scans over broad refactors
|
||||
|
||||
Constraints:
|
||||
- Do not reopen already-completed issues unless verification proves a regression
|
||||
- Do not revert unrelated user changes
|
||||
- Keep changes minimal and production-oriented
|
||||
- Do not claim something is fixed unless code and verification support it
|
||||
- If a fix requires an ops action outside the repo, note it explicitly in the issue comment and final summary
|
||||
|
||||
Manual ops actions that are outside the repo:
|
||||
- Rotate any secrets that were exposed in chat or local env files
|
||||
- Clean up local untracked `.env` files that still contain real credentials
|
||||
- Replace any weak local credentials still present in local-only env files
|
||||
|
||||
Final output format:
|
||||
- `Completed:` issue numbers fully resolved
|
||||
- `Partial:` issue numbers partially resolved and what remains
|
||||
|
||||
52
docs/trust-model.md
Normal file
52
docs/trust-model.md
Normal file
@@ -0,0 +1,52 @@
|
||||
# Gateway Trust Model
|
||||
|
||||
## Architecture
|
||||
|
||||
All frontend requests go through: Browser → Pangolin → frontend-v2 (SvelteKit hooks) → gateway → backend services.
|
||||
|
||||
## Authentication Layers
|
||||
|
||||
### Gateway (platform auth)
|
||||
- Users authenticate via `/api/auth/login` with username/password (bcrypt)
|
||||
- Session stored as `platform_session` cookie (HttpOnly, Secure, SameSite=Lax)
|
||||
- All `(app)` routes require valid session (checked in `+layout.server.ts`)
|
||||
|
||||
### Service-level auth
|
||||
Each backend service has its own auth mechanism. The gateway injects credentials when proxying:
|
||||
|
||||
| Service | Auth Type | Injected By Gateway | Validated Against |
|
||||
|---------|-----------|--------------------|--------------------|
|
||||
| Trips | Bearer token | `Authorization: Bearer {token}` | `/api/trips` (protected endpoint) |
|
||||
| Fitness | Bearer token | `Authorization: Bearer {token}` | `/api/user` (protected endpoint) |
|
||||
| Reader | API key | `X-Auth-Token: {key}` | `/v1/feeds/counters` |
|
||||
| Inventory | API key | `X-API-Key: {key}` | `/summary` |
|
||||
| Budget | API key | `X-API-Key: {key}` | `/summary` |
|
||||
| Books (Shelfmark) | None (proxied) | — | Gateway auth only |
|
||||
| Music (Spotizerr) | None (proxied) | — | Gateway auth only |
|
||||
|
||||
### Frontend hooks auth (SvelteKit)
|
||||
- Immich proxy: validates `platform_session` cookie before proxying
|
||||
- Karakeep proxy: validates `platform_session` cookie before proxying
|
||||
- Legacy trips Immich: validates `platform_session` cookie before proxying
|
||||
|
||||
## Service Connections
|
||||
- Users connect services via Settings page
|
||||
- Token validation uses a **protected endpoint**, not health checks
|
||||
- Unknown services cannot be connected (rejected with 400)
|
||||
- Tokens stored in `service_connections` table, per-user
|
||||
|
||||
## Internal Network
|
||||
- All services communicate on Docker internal network
|
||||
- No service port is exposed to the host (except frontend-v2 via Pangolin)
|
||||
- Gateway is the single entry point for all API traffic
|
||||
|
||||
## TLS
|
||||
- External HTTPS: default TLS verification (certificate + hostname)
|
||||
- Internal services: `_internal_ssl_ctx` with verification disabled (Docker services don't have valid certs)
|
||||
- Image proxy: default TLS verification + domain allowlist
|
||||
|
||||
## Secrets
|
||||
- All secrets loaded from environment variables
|
||||
- No hardcoded credentials in code
|
||||
- `.env` files excluded from git
|
||||
- Admin credentials required via `ADMIN_USERNAME`/`ADMIN_PASSWORD` env vars
|
||||
@@ -89,16 +89,26 @@ def handle_set_connection(handler, user, body):
|
||||
handler._send_json({"error": f"Unknown service: {service}"}, 400)
|
||||
return
|
||||
|
||||
# Test the token
|
||||
# Validate token against a protected endpoint (not health check)
|
||||
if service == "trips":
|
||||
test_url = f"{target}/api/trips"
|
||||
headers = {"Authorization": f"Bearer {auth_token}"}
|
||||
elif service == "fitness":
|
||||
test_url = f"{target}/api/users"
|
||||
test_url = f"{target}/api/user"
|
||||
headers = {"Authorization": f"Bearer {auth_token}"}
|
||||
elif service == "inventory":
|
||||
test_url = f"{target}/summary"
|
||||
headers = {"X-API-Key": auth_token}
|
||||
elif service == "budget":
|
||||
test_url = f"{target}/summary"
|
||||
headers = {"X-API-Key": auth_token}
|
||||
elif service == "reader":
|
||||
test_url = f"{target}/v1/feeds/counters"
|
||||
headers = {"X-Auth-Token": auth_token}
|
||||
else:
|
||||
test_url = f"{target}/api/health"
|
||||
headers = {"Authorization": f"Bearer {auth_token}"}
|
||||
# Unknown service — reject, don't fall back to health check
|
||||
handler._send_json({"error": f"Cannot validate token for service: {service}"}, 400)
|
||||
return
|
||||
|
||||
status, _, _ = proxy_request(test_url, "GET", headers, timeout=10)
|
||||
if status == 401:
|
||||
@@ -158,8 +168,19 @@ def handle_get_pinned(handler, user):
|
||||
handler._send_json({"pinned": [dict(r) for r in rows]})
|
||||
|
||||
|
||||
import time as _time
|
||||
_dashboard_cache = {"data": None, "expires": 0, "user_id": None}
|
||||
_DASHBOARD_TTL = 30 # seconds
|
||||
|
||||
def handle_dashboard(handler, user):
|
||||
"""Aggregate dashboard data from connected services -- all fetches in parallel."""
|
||||
"""Aggregate dashboard data from connected services -- all fetches in parallel.
|
||||
Results cached for 30 seconds per user to avoid repeated slow aggregation."""
|
||||
# Return cached if fresh and same user
|
||||
if (_dashboard_cache["data"] and _dashboard_cache["user_id"] == user["id"]
|
||||
and _time.time() < _dashboard_cache["expires"]):
|
||||
handler._send_json(_dashboard_cache["data"])
|
||||
return
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
|
||||
conn = get_db()
|
||||
@@ -311,4 +332,8 @@ def handle_dashboard(handler, user):
|
||||
conn2.close()
|
||||
pinned = [dict(r) for r in pinned_rows]
|
||||
|
||||
handler._send_json({"widgets": widgets, "pinned": pinned})
|
||||
result = {"widgets": widgets, "pinned": pinned}
|
||||
_dashboard_cache["data"] = result
|
||||
_dashboard_cache["expires"] = _time.time() + _DASHBOARD_TTL
|
||||
_dashboard_cache["user_id"] = user["id"]
|
||||
handler._send_json(result)
|
||||
|
||||
@@ -545,9 +545,16 @@ app.get('/transfer-payees', requireReady, async (_req, res) => {
|
||||
}
|
||||
});
|
||||
|
||||
// ---- Dashboard summary ----------------------------------------------------
|
||||
// ---- Dashboard summary (cached) -------------------------------------------
|
||||
|
||||
let summaryCache = { data: null, expiresAt: 0 };
|
||||
const SUMMARY_TTL_MS = 60 * 1000; // 1 minute cache
|
||||
|
||||
app.get('/summary', requireReady, async (_req, res) => {
|
||||
// Return cached summary if fresh
|
||||
if (summaryCache.data && Date.now() < summaryCache.expiresAt) {
|
||||
return res.json(summaryCache.data);
|
||||
}
|
||||
try {
|
||||
// Total balance across all accounts
|
||||
const accounts = await api.getAccounts();
|
||||
@@ -592,7 +599,7 @@ app.get('/summary', requireReady, async (_req, res) => {
|
||||
.sort((a, b) => a.amount - b.amount) // most negative first
|
||||
.slice(0, 10);
|
||||
|
||||
res.json({
|
||||
const result = {
|
||||
month,
|
||||
totalBalance,
|
||||
totalBalanceDollars: centsToDollars(totalBalance),
|
||||
@@ -603,7 +610,9 @@ app.get('/summary', requireReady, async (_req, res) => {
|
||||
topCategories,
|
||||
accountCount: accounts.length,
|
||||
transactionCount: allTxns.length,
|
||||
});
|
||||
};
|
||||
summaryCache = { data: result, expiresAt: Date.now() + SUMMARY_TTL_MS };
|
||||
res.json(result);
|
||||
} catch (err) {
|
||||
console.error('[budget] GET /summary error:', err);
|
||||
res.status(500).json({ error: err.message });
|
||||
|
||||
Reference in New Issue
Block a user