feat: scenarios for curator/memo-inbox/pi-grok, deploy and backup tooling
Scenarios - memo-inbox: mirrored by copying; the live directory was not moved or modified and the service was not restarted. All four tracked files match byte for byte (pi-diff.sh reports SAME). Marked deploy = "mirror" so deploy-scenario.sh refuses --apply: applying a mirror would invert the direction of truth and could change a service in daily use. - curator: target configuration, not yet deployed. .pi/SYSTEM.md replaces pi's coding-assistant prompt; durable role text is in .pi/APPEND_SYSTEM.md; profile.toml is the single source of truth for the launch contract. - pi-grok: registered only. It is genuinely a coding agent, so the isolation baseline does not apply in full. Corrections to the documentation, found by testing rather than by reading - AGENTS.override.md does NOT block parent-directory context files; it only shadows its own directory. Verified: with an override file in the workspace, a marker in /tmp/AGENTS.md still reached the system prompt. The only effective switch is --no-context-files, so durable role text must live in .pi/APPEND_SYSTEM.md, which is a system-prompt file and unaffected by -nc. Verified end state: no coding-assistant framing, no pi-docs block, own identity and role text present, no parent pollution, only own skills/tools. - PI_CODING_AGENT_DIR isolates settings/models/auth/trust/extensions/skills/ prompts/themes under the agent directory -- stronger than the --no-* flags because it also repoints credentials -- but does NOT cover ~/.agents/skills. Measured: find-skills, modsearch and summarize still leak. So it complements --no-skills rather than replacing it. - --append-system-prompt accepts a file path, which pi-grok relies on. - cwd is what anchors .pi discovery: a probe that forgot cwd silently lost .pi/SYSTEM.md and kept the coding-assistant persona. Tooling (all dry-run by default; none of them restarts a service) - pi-diff.sh: compares tracked config against the live install in both directions, with a key-redacted comparison for models.json - deploy-scenario.sh: installs a workspace and renders profile.toml into .pi/launch.json, then checks that every referenced path exists - deploy-runtime.sh: renders models.json from its template, refusing placeholder or missing keys. Verified byte-identical to the live file - pi-backup.sh / pi-restore.sh: archives outside the repo, sha256 manifest verified before any restore, live paths preserved rather than overwritten Fixed while testing: pi-backup.sh compared the destination against the repo root literally, so a relative --dest ./backups wrote credential archives into the work tree. Now canonicalised with realpath; ./backups, an absolute in-repo path and ./docs/../backups are all refused.
This commit is contained in:
Kai
Executable
+110
@@ -0,0 +1,110 @@
|
||||
#!/usr/bin/env bash
|
||||
# ---------------------------------------------------------------------------
|
||||
# Install the user-level Pi configuration from runtime/ into ~/.pi/agent,
|
||||
# rendering models.json from its template.
|
||||
#
|
||||
# Dry run by default.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/deploy-runtime.sh
|
||||
# scripts/deploy-runtime.sh --apply
|
||||
# ---------------------------------------------------------------------------
|
||||
# shellcheck source=lib/common.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/lib/common.sh"
|
||||
|
||||
APPLY=0
|
||||
[ "${1:-}" = "--apply" ] && APPLY=1
|
||||
|
||||
SRC="$REPO_ROOT/runtime/agent"
|
||||
DST="$HOME/.pi/agent"
|
||||
SECRETS="$REPO_ROOT/secrets/zenmux.env"
|
||||
|
||||
head1 "deploy runtime -> $DST"
|
||||
[ "$APPLY" -eq 1 ] || info "${C_DIM}(dry run; pass --apply to write)${C_OFF}"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# models.json: render the template
|
||||
# ---------------------------------------------------------------------------
|
||||
head1 "models.json"
|
||||
if [ ! -f "$SECRETS" ]; then
|
||||
die "missing $SECRETS
|
||||
cp secrets/zenmux.env.example secrets/zenmux.env && chmod 600 secrets/zenmux.env
|
||||
then fill in ZENMUX_API_KEY"
|
||||
fi
|
||||
PERM="$(stat -c '%a' "$SECRETS")"
|
||||
[ "$PERM" = "600" ] || warn "$SECRETS has mode $PERM; expected 600"
|
||||
|
||||
# shellcheck disable=SC1090
|
||||
set -a; . "$SECRETS"; set +a
|
||||
[ -n "${ZENMUX_API_KEY:-}" ] || die "ZENMUX_API_KEY is empty in $SECRETS"
|
||||
case "$ZENMUX_API_KEY" in
|
||||
'<REDACTED>'|'<'*'>'|'${'*) die "ZENMUX_API_KEY in $SECRETS is still a placeholder" ;;
|
||||
esac
|
||||
|
||||
RENDERED="$(python3 - "$SRC/models.json.template" <<'PY'
|
||||
import json, os, pathlib, re, sys
|
||||
text = pathlib.Path(sys.argv[1]).read_text()
|
||||
|
||||
def sub(match):
|
||||
name = match.group(1)
|
||||
value = os.environ.get(name)
|
||||
if not value:
|
||||
sys.exit(f"unresolved placeholder ${{{name}}}")
|
||||
return json.dumps(value)[1:-1] # escape for a JSON string context
|
||||
|
||||
text = re.sub(r'\$\{([A-Z_][A-Z0-9_]*)\}', sub, text)
|
||||
json.loads(text) # fail early on malformed output
|
||||
sys.stdout.write(text)
|
||||
PY
|
||||
)" || die "cannot render models.json.template"
|
||||
|
||||
if [ -f "$DST/models.json" ] && [ "$RENDERED" = "$(cat "$DST/models.json")" ]; then
|
||||
ok "models.json already current"
|
||||
else
|
||||
info " write models.json ${C_DIM}(rendered, mode 600)${C_OFF}"
|
||||
if [ "$APPLY" -eq 1 ]; then
|
||||
install -d -m 700 "$DST"
|
||||
printf '%s' "$RENDERED" > "$DST/models.json"
|
||||
chmod 600 "$DST/models.json"
|
||||
fi
|
||||
fi
|
||||
unset ZENMUX_API_KEY
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Everything else is copied verbatim
|
||||
# ---------------------------------------------------------------------------
|
||||
head1 "settings, extensions, prompts"
|
||||
CHANGES=0
|
||||
while IFS= read -r rel; do
|
||||
src="$SRC/$rel"
|
||||
dst="$DST/$rel"
|
||||
if [ -f "$dst" ] && cmp -s "$src" "$dst"; then continue; fi
|
||||
CHANGES=$((CHANGES + 1))
|
||||
[ -f "$dst" ] && info " update $rel" || info " create $rel"
|
||||
if [ "$APPLY" -eq 1 ]; then
|
||||
install -d -m 700 "$(dirname "$dst")"
|
||||
install -m 600 "$src" "$dst"
|
||||
fi
|
||||
done < <(cd "$SRC" && find settings.json extensions prompts -type f 2>/dev/null | sort)
|
||||
[ "$CHANGES" -eq 0 ] && ok "already current"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Report host-side files this repository deliberately does not own
|
||||
# ---------------------------------------------------------------------------
|
||||
head1 "not managed here"
|
||||
for f in auth.json trust.json models-store.json; do
|
||||
[ -e "$DST/$f" ] && info " ${C_DIM}$f (host state)${C_OFF}"
|
||||
done
|
||||
if [ -e "$DST/extensions/herdr-agent-state.ts" ]; then
|
||||
info " ${C_DIM}extensions/herdr-agent-state.ts (installed by herdr)${C_OFF}"
|
||||
fi
|
||||
|
||||
printf '\n'
|
||||
if [ "$APPLY" -eq 1 ]; then
|
||||
ok "applied"
|
||||
info ""
|
||||
info "Restart any Pi gateway that should pick up new provider settings:"
|
||||
info " systemctl --user restart curator.service pi-memo-telegram.service"
|
||||
else
|
||||
info "${C_DIM}dry run complete; re-run with --apply${C_OFF}"
|
||||
fi
|
||||
Executable
+187
@@ -0,0 +1,187 @@
|
||||
#!/usr/bin/env bash
|
||||
# ---------------------------------------------------------------------------
|
||||
# Install a scenario's tracked workspace into its live location, and render the
|
||||
# launch contract from profile.toml.
|
||||
#
|
||||
# Dry run by default. Never restarts a service: restarting is a decision about
|
||||
# when to interrupt a live conversation, and belongs to the operator.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/deploy-scenario.sh <scenario> # show what would change
|
||||
# scripts/deploy-scenario.sh <scenario> --apply
|
||||
# scripts/deploy-scenario.sh --list
|
||||
# ---------------------------------------------------------------------------
|
||||
# shellcheck source=lib/common.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/lib/common.sh"
|
||||
|
||||
if [ "${1:-}" = "--list" ]; then
|
||||
list_scenarios
|
||||
exit 0
|
||||
fi
|
||||
|
||||
NAME="${1:-}"
|
||||
APPLY=0
|
||||
[ "${2:-}" = "--apply" ] && APPLY=1
|
||||
|
||||
DIR="$(require_scenario "$NAME")"
|
||||
PROFILE="$DIR/profile.toml"
|
||||
[ -f "$PROFILE" ] || die "scenario '$NAME' has no profile.toml"
|
||||
|
||||
TRACKED="$(toml_get "$PROFILE" scenario tracked)"
|
||||
if [ "$TRACKED" = "false" ]; then
|
||||
die "scenario '$NAME' is registered but not tracked; nothing to deploy"
|
||||
fi
|
||||
|
||||
# A scenario is either an authoritative source ("managed") or a record of what
|
||||
# the live host already does ("mirror"). Applying a mirror would invert the
|
||||
# direction of truth and, for a service in daily use, risks a behaviour change
|
||||
# that nobody asked for. Mirrors are diffable but not deployable.
|
||||
MODE="$(toml_get "$PROFILE" scenario deploy)"
|
||||
MODE="${MODE:-managed}"
|
||||
if [ "$MODE" = "mirror" ] && [ "$APPLY" -eq 1 ]; then
|
||||
die "scenario '$NAME' is a mirror of the live configuration, not an authoritative source.
|
||||
Refusing --apply. Use scripts/pi-diff.sh $NAME to compare, and promote the
|
||||
scenario to deploy = \"managed\" in profile.toml once it is the source of truth."
|
||||
fi
|
||||
|
||||
WORKSPACE="$(toml_get "$PROFILE" scenario workspace)"
|
||||
[ -n "$WORKSPACE" ] || die "scenario '$NAME': [scenario].workspace is not set"
|
||||
[ -d "$DIR/workspace" ] || die "scenario '$NAME' has no tracked workspace/"
|
||||
|
||||
head1 "deploy $NAME -> $WORKSPACE"
|
||||
[ "$APPLY" -eq 1 ] || info "${C_DIM}(dry run; pass --apply to write)${C_OFF}"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. Workspace files
|
||||
# ---------------------------------------------------------------------------
|
||||
CHANGES=0
|
||||
while IFS= read -r rel; do
|
||||
src="$DIR/workspace/$rel"
|
||||
dst="$WORKSPACE/$rel"
|
||||
if [ -f "$dst" ] && cmp -s "$src" "$dst"; then
|
||||
continue
|
||||
fi
|
||||
CHANGES=$((CHANGES + 1))
|
||||
if [ -f "$dst" ]; then
|
||||
info " update $rel"
|
||||
else
|
||||
info " create $rel"
|
||||
fi
|
||||
if [ "$APPLY" -eq 1 ]; then
|
||||
install -d -m 700 "$(dirname "$dst")"
|
||||
install -m 600 "$src" "$dst"
|
||||
fi
|
||||
done < <(cd "$DIR/workspace" && find . -type f -printf '%P\n' | sort)
|
||||
|
||||
# Executables under bin/ need the execute bit back.
|
||||
if [ "$APPLY" -eq 1 ] && [ -d "$DIR/workspace/bin" ]; then
|
||||
while IFS= read -r rel; do
|
||||
[ -x "$DIR/workspace/$rel" ] && chmod 700 "$WORKSPACE/$rel"
|
||||
done < <(cd "$DIR/workspace" && find bin -type f -printf '%P\n' 2>/dev/null | sed 's|^|bin/|')
|
||||
fi
|
||||
|
||||
[ "$CHANGES" -eq 0 ] && ok "workspace already matches"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Launch contract
|
||||
#
|
||||
# profile.toml is the single source of truth for launch flags. Rendering it to
|
||||
# JSON lets the gateway read the contract instead of hard-coding flags, and lets
|
||||
# the gateway fail closed when the file is missing -- silently running without
|
||||
# --no-extensions would widen the agent's reach.
|
||||
# ---------------------------------------------------------------------------
|
||||
if [ "$MODE" = "mirror" ]; then
|
||||
head1 "launch contract"
|
||||
info " ${C_DIM}skipped: '$NAME' is a mirror; its launch flags live in its own"
|
||||
info " gateway, not in profile.toml${C_OFF}"
|
||||
printf '\n'
|
||||
ok "dry run complete (mirror scenario; nothing to apply)"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
LAUNCH="$WORKSPACE/.pi/launch.json"
|
||||
head1 "launch contract -> $LAUNCH"
|
||||
|
||||
RENDERED="$(python3 - "$PROFILE" "$WORKSPACE" <<'PY'
|
||||
import json, sys, tomllib, pathlib
|
||||
|
||||
profile = tomllib.loads(pathlib.Path(sys.argv[1]).read_text())
|
||||
workspace = pathlib.Path(sys.argv[2])
|
||||
|
||||
def rel(paths):
|
||||
return [str(workspace / p) for p in paths or []]
|
||||
|
||||
iso = profile.get("isolation", {})
|
||||
res = profile.get("resources", {})
|
||||
pers = profile.get("personality", {})
|
||||
out = {
|
||||
"_generated_from": "profile.toml",
|
||||
"_note": "Do not edit. Regenerate with scripts/deploy-scenario.sh.",
|
||||
"scenario": profile.get("scenario", {}).get("name"),
|
||||
"workspace": str(workspace),
|
||||
"session_dir": profile.get("scenario", {}).get("session_dir"),
|
||||
"model": profile.get("model", {}),
|
||||
"session": profile.get("session", {}),
|
||||
"isolation": iso,
|
||||
"personality": {
|
||||
"system_prompt": str(workspace / pers["system_prompt"]) if pers.get("system_prompt") else "",
|
||||
"append_system_prompt": str(workspace / pers["append_system_prompt"]) if pers.get("append_system_prompt") else "",
|
||||
"context_files": pers.get("context_files", []),
|
||||
},
|
||||
"resources": {
|
||||
"extensions": rel(res.get("extensions")),
|
||||
"skills": rel(res.get("skills")),
|
||||
},
|
||||
"tools": profile.get("tools", {}),
|
||||
"budget": profile.get("budget", {}),
|
||||
"env": profile.get("env", {}),
|
||||
"bridge": profile.get("bridge", {}),
|
||||
}
|
||||
print(json.dumps(out, ensure_ascii=False, indent=2))
|
||||
PY
|
||||
)" || die "cannot render profile.toml"
|
||||
|
||||
if [ -f "$LAUNCH" ] && [ "$RENDERED" = "$(cat "$LAUNCH")" ]; then
|
||||
ok "launch.json already current"
|
||||
else
|
||||
info " write .pi/launch.json"
|
||||
if [ "$APPLY" -eq 1 ]; then
|
||||
install -d -m 700 "$WORKSPACE/.pi"
|
||||
printf '%s\n' "$RENDERED" > "$LAUNCH"
|
||||
chmod 600 "$LAUNCH"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. Referenced paths must exist
|
||||
# ---------------------------------------------------------------------------
|
||||
head1 "referenced resources"
|
||||
MISSING=0
|
||||
while IFS= read -r p; do
|
||||
[ -n "$p" ] || continue
|
||||
if [ -e "$p" ]; then ok "$p"; else warn "missing: $p"; MISSING=$((MISSING + 1)); fi
|
||||
done < <(printf '%s\n' "$RENDERED" | python3 -c '
|
||||
import json, sys
|
||||
d = json.load(sys.stdin)
|
||||
for p in d["resources"]["extensions"] + d["resources"]["skills"]:
|
||||
print(p)
|
||||
for key in ("system_prompt", "append_system_prompt"):
|
||||
if d["personality"].get(key):
|
||||
print(d["personality"][key])
|
||||
')
|
||||
|
||||
printf '\n'
|
||||
if [ "$APPLY" -eq 1 ]; then
|
||||
ok "applied"
|
||||
info ""
|
||||
SERVICE="$(toml_get "$PROFILE" scenario service)"
|
||||
if [ -n "$SERVICE" ]; then
|
||||
info "The service was NOT restarted. When you are ready:"
|
||||
info " systemctl --user restart $SERVICE"
|
||||
info " systemctl --user status $SERVICE"
|
||||
fi
|
||||
else
|
||||
info "${C_DIM}dry run complete; re-run with --apply${C_OFF}"
|
||||
fi
|
||||
[ "$MISSING" -eq 0 ] || warn "$MISSING referenced path(s) missing; the agent will not start correctly"
|
||||
exit 0
|
||||
Executable
+81
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env bash
|
||||
# Shared helpers for the pi-agent-config scripts.
|
||||
# shellcheck shell=bash
|
||||
|
||||
set -uo pipefail
|
||||
|
||||
REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null)" || {
|
||||
echo "error: not inside the pi-agent-config git work tree" >&2
|
||||
exit 1
|
||||
}
|
||||
export REPO_ROOT
|
||||
|
||||
if [ -t 1 ]; then
|
||||
C_RED=$'\033[31m'; C_GREEN=$'\033[32m'; C_YELLOW=$'\033[33m'
|
||||
C_BOLD=$'\033[1m'; C_DIM=$'\033[2m'; C_OFF=$'\033[0m'
|
||||
else
|
||||
C_RED=''; C_GREEN=''; C_YELLOW=''; C_BOLD=''; C_DIM=''; C_OFF=''
|
||||
fi
|
||||
|
||||
info() { printf '%s\n' "$*"; }
|
||||
ok() { printf '%s %s%s\n' "${C_GREEN}OK${C_OFF}" "$*" ''; }
|
||||
warn() { printf '%s %s\n' "${C_YELLOW}WARN${C_OFF}" "$*" >&2; }
|
||||
die() { printf '%s %s\n' "${C_RED}ERROR${C_OFF}" "$*" >&2; exit 1; }
|
||||
head1() { printf '\n%s%s%s\n' "$C_BOLD" "$*" "$C_OFF"; }
|
||||
|
||||
# List the scenarios that exist, excluding the template.
|
||||
list_scenarios() {
|
||||
find "$REPO_ROOT/scenarios" -mindepth 1 -maxdepth 1 -type d \
|
||||
-not -name '_*' -printf '%f\n' | sort
|
||||
}
|
||||
|
||||
require_scenario() {
|
||||
local name="${1:-}"
|
||||
[ -n "$name" ] || die "a scenario name is required. Available: $(list_scenarios | tr '\n' ' ')"
|
||||
[ -d "$REPO_ROOT/scenarios/$name" ] \
|
||||
|| die "unknown scenario '$name'. Available: $(list_scenarios | tr '\n' ' ')"
|
||||
printf '%s' "$REPO_ROOT/scenarios/$name"
|
||||
}
|
||||
|
||||
# Read a flat "key = value" entry from a TOML section.
|
||||
# Intentionally minimal: these profiles are hand-written and flat. Anything more
|
||||
# structured should be read by the Python consumer, not by shell.
|
||||
toml_get() {
|
||||
local file="$1" section="$2" key="$3"
|
||||
awk -v want_section="$section" -v want_key="$key" '
|
||||
/^[[:space:]]*\[/ {
|
||||
s = $0; sub(/^[[:space:]]*\[/, "", s); sub(/\][[:space:]]*$/, "", s)
|
||||
current = s; next
|
||||
}
|
||||
{
|
||||
line = $0
|
||||
sub(/[[:space:]]*#.*$/, "", line)
|
||||
if (current != want_section) next
|
||||
if (line !~ /=/) next
|
||||
k = line; sub(/=.*$/, "", k); gsub(/[[:space:]]/, "", k)
|
||||
if (k != want_key) next
|
||||
v = line; sub(/^[^=]*=[[:space:]]*/, "", v)
|
||||
gsub(/^"|"$/, "", v)
|
||||
gsub(/[[:space:]]+$/, "", v)
|
||||
print v; exit
|
||||
}
|
||||
' "$file"
|
||||
}
|
||||
|
||||
# Print a unified diff between a tracked file and its live counterpart.
|
||||
# Returns 0 when identical, 1 when different or missing.
|
||||
diff_file() {
|
||||
local tracked="$1" live="$2" label="$3"
|
||||
if [ ! -f "$live" ]; then
|
||||
printf '%s %s %s(live file absent)%s\n' "${C_YELLOW}DIFF${C_OFF}" "$label" "$C_DIM" "$C_OFF"
|
||||
return 1
|
||||
fi
|
||||
if cmp -s "$tracked" "$live"; then
|
||||
printf '%s %s\n' "${C_GREEN}SAME${C_OFF}" "$label"
|
||||
return 0
|
||||
fi
|
||||
printf '%s %s\n' "${C_YELLOW}DIFF${C_OFF}" "$label"
|
||||
diff -u --label "repo/$label" "$tracked" --label "live/$label" "$live" \
|
||||
| sed 's/^/ /' || true
|
||||
return 1
|
||||
}
|
||||
Executable
+140
@@ -0,0 +1,140 @@
|
||||
#!/usr/bin/env bash
|
||||
# ---------------------------------------------------------------------------
|
||||
# Back up the live Pi installation and every scenario workspace.
|
||||
#
|
||||
# Archives are written OUTSIDE the repository by design. The precedent to avoid
|
||||
# is hermes-agent-config, which committed hermes-secrets-*.tar.gz into its own
|
||||
# working tree; had that tree ever been pushed, the credentials would have gone
|
||||
# with it.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/pi-backup.sh # default destination
|
||||
# scripts/pi-backup.sh --dest /path/to/dir
|
||||
# scripts/pi-backup.sh --no-secrets # skip credential archives
|
||||
# ---------------------------------------------------------------------------
|
||||
# shellcheck source=lib/common.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/lib/common.sh"
|
||||
|
||||
DEST_ROOT="/mnt/truenas/multimedia/curator/backup/pi-agent-config"
|
||||
WITH_SECRETS=1
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--dest) DEST_ROOT="${2:?--dest needs a path}"; shift 2 ;;
|
||||
--no-secrets) WITH_SECRETS=0; shift ;;
|
||||
-h|--help) sed -n '2,16p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) die "unknown argument: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Canonicalise before comparing: a relative path such as ./backups would
|
||||
# otherwise slip past a literal prefix test and write credential archives into
|
||||
# the working tree. (.gitignore would still stop them being committed, but the
|
||||
# archives should not be there at all.)
|
||||
DEST_ROOT="$(realpath -m -- "$DEST_ROOT")"
|
||||
REPO_REAL="$(realpath -- "$REPO_ROOT")"
|
||||
case "$DEST_ROOT" in
|
||||
"$REPO_REAL"|"$REPO_REAL"/*)
|
||||
die "refusing to write backups inside the repository
|
||||
requested: $DEST_ROOT
|
||||
repository: $REPO_REAL
|
||||
Backups contain plaintext credentials and must live outside the work tree." ;;
|
||||
esac
|
||||
|
||||
STAMP="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
DEST="$DEST_ROOT/$STAMP"
|
||||
install -d -m 700 "$DEST" || die "cannot create $DEST"
|
||||
|
||||
head1 "backup -> $DEST"
|
||||
|
||||
# --- 1. Pi agent directory, excluding sessions -----------------------------
|
||||
if [ -d "$HOME/.pi/agent" ]; then
|
||||
tar --exclude='agent/sessions' -czf "$DEST/pi-agent-runtime.tar.gz" -C "$HOME/.pi" agent
|
||||
ok "pi-agent-runtime.tar.gz ${C_DIM}(contains plaintext apiKey)${C_OFF}"
|
||||
chmod 600 "$DEST/pi-agent-runtime.tar.gz"
|
||||
fi
|
||||
|
||||
# --- 2. Sessions, separately: bulky and lower value -----------------------
|
||||
if [ -d "$HOME/.pi/agent/sessions" ]; then
|
||||
tar -czf "$DEST/pi-global-sessions.tar.gz" -C "$HOME/.pi/agent" sessions
|
||||
ok "pi-global-sessions.tar.gz"
|
||||
fi
|
||||
|
||||
# --- 3. Each scenario workspace and session directory ---------------------
|
||||
while IFS= read -r name; do
|
||||
profile="$REPO_ROOT/scenarios/$name/profile.toml"
|
||||
[ -f "$profile" ] || continue
|
||||
workspace="$(toml_get "$profile" scenario workspace)"
|
||||
session_dir="$(toml_get "$profile" scenario session_dir)"
|
||||
|
||||
if [ -n "$workspace" ] && [ -d "$workspace" ]; then
|
||||
tar --exclude='.venv' --exclude='node_modules' --exclude='__pycache__' \
|
||||
--exclude='.ccgram-uploads' \
|
||||
-czf "$DEST/workspace-$name.tar.gz" \
|
||||
-C "$(dirname "$workspace")" "$(basename "$workspace")"
|
||||
ok "workspace-$name.tar.gz"
|
||||
fi
|
||||
|
||||
if [ -n "$session_dir" ] && [ -d "$session_dir" ]; then
|
||||
tar -czf "$DEST/sessions-$name.tar.gz" \
|
||||
-C "$(dirname "$session_dir")" "$(basename "$session_dir")"
|
||||
ok "sessions-$name.tar.gz"
|
||||
fi
|
||||
done < <(list_scenarios)
|
||||
|
||||
# --- 4. Credentials -------------------------------------------------------
|
||||
if [ "$WITH_SECRETS" -eq 1 ]; then
|
||||
STAGE="$(mktemp -d)"
|
||||
found=0
|
||||
while IFS= read -r f; do
|
||||
[ -f "$f" ] || continue
|
||||
cp -p "$f" "$STAGE/$(printf '%s' "$f" | tr '/' '_')"
|
||||
found=1
|
||||
done <<EOF
|
||||
$HOME/.config/curator/curator.env
|
||||
$HOME/.secrets/pi-memo-telegram.env
|
||||
$HOME/.ccgram/.env
|
||||
$REPO_ROOT/secrets/zenmux.env
|
||||
EOF
|
||||
if [ "$found" -eq 1 ]; then
|
||||
tar -czf "$DEST/secrets.tar.gz" -C "$STAGE" .
|
||||
chmod 600 "$DEST/secrets.tar.gz"
|
||||
ok "secrets.tar.gz ${C_DIM}(mode 600 -- never commit)${C_OFF}"
|
||||
fi
|
||||
rm -rf "$STAGE"
|
||||
else
|
||||
info " ${C_DIM}secrets skipped (--no-secrets)${C_OFF}"
|
||||
fi
|
||||
|
||||
# --- 5. systemd units and an environment freeze ---------------------------
|
||||
STAGE="$(mktemp -d)"
|
||||
cp -p "$HOME"/.config/systemd/user/*.service "$HOME"/.config/systemd/user/*.timer "$STAGE/" 2>/dev/null || true
|
||||
systemctl --user list-units --all --no-pager --no-legend > "$STAGE/list-units.txt" 2>&1 || true
|
||||
systemctl --user list-unit-files --no-pager --no-legend > "$STAGE/list-unit-files.txt" 2>&1 || true
|
||||
tar -czf "$DEST/systemd-units.tar.gz" -C "$STAGE" .
|
||||
rm -rf "$STAGE"
|
||||
ok "systemd-units.tar.gz"
|
||||
|
||||
{
|
||||
echo "# Environment freeze -- $(date -u +%FT%TZ)"
|
||||
echo; echo "## pi"; pi --version 2>&1
|
||||
echo; echo "## npm -g"; npm ls -g --depth=0 2>/dev/null
|
||||
echo; echo "## node"; node --version
|
||||
echo; echo "## python3"; python3 --version
|
||||
echo; echo "## pi-agent-config HEAD"
|
||||
git -C "$REPO_ROOT" log --oneline -1
|
||||
git -C "$REPO_ROOT" rev-parse HEAD
|
||||
} > "$DEST/environment-freeze.txt"
|
||||
ok "environment-freeze.txt"
|
||||
|
||||
# --- 6. Manifest ----------------------------------------------------------
|
||||
( cd "$DEST" && sha256sum ./*.tar.gz environment-freeze.txt > MANIFEST.sha256 )
|
||||
ok "MANIFEST.sha256"
|
||||
|
||||
printf '\n'
|
||||
( cd "$DEST" && sha256sum -c MANIFEST.sha256 >/dev/null 2>&1 ) \
|
||||
&& ok "checksums verified" \
|
||||
|| die "checksum verification FAILED -- do not rely on this backup"
|
||||
|
||||
info ""
|
||||
info "Restore with: scripts/pi-restore.sh --from $DEST"
|
||||
Executable
+120
@@ -0,0 +1,120 @@
|
||||
#!/usr/bin/env bash
|
||||
# ---------------------------------------------------------------------------
|
||||
# Compare tracked configuration against the live installation.
|
||||
#
|
||||
# Run this before every deploy, and after every migration, to prove that the
|
||||
# repository and the host agree.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/pi-diff.sh # runtime + every scenario
|
||||
# scripts/pi-diff.sh runtime
|
||||
# scripts/pi-diff.sh <scenario>
|
||||
#
|
||||
# Exit status: 0 when everything matches, 1 when anything differs.
|
||||
# ---------------------------------------------------------------------------
|
||||
# shellcheck source=lib/common.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/lib/common.sh"
|
||||
|
||||
DIFFS=0
|
||||
|
||||
diff_runtime() {
|
||||
head1 "runtime (~/.pi/agent)"
|
||||
local live="$HOME/.pi/agent"
|
||||
local tracked="$REPO_ROOT/runtime/agent"
|
||||
|
||||
diff_file "$tracked/settings.json" "$live/settings.json" "settings.json" || DIFFS=$((DIFFS + 1))
|
||||
|
||||
# models.json is rendered from a template, so compare the redacted forms.
|
||||
if [ -f "$live/models.json" ]; then
|
||||
local a b
|
||||
a="$(mktemp)"; b="$(mktemp)"
|
||||
python3 - "$tracked/models.json.template" >"$a" <<'PY'
|
||||
import json, re, sys
|
||||
print(re.sub(r'"apiKey":\s*".*?"', '"apiKey": "<X>"',
|
||||
json.dumps(json.load(open(sys.argv[1])), ensure_ascii=False, indent=2, sort_keys=True)))
|
||||
PY
|
||||
python3 - "$live/models.json" >"$b" <<'PY'
|
||||
import json, re, sys
|
||||
print(re.sub(r'"apiKey":\s*".*?"', '"apiKey": "<X>"',
|
||||
json.dumps(json.load(open(sys.argv[1])), ensure_ascii=False, indent=2, sort_keys=True)))
|
||||
PY
|
||||
if cmp -s "$a" "$b"; then
|
||||
ok "models.json (key-redacted comparison)"
|
||||
else
|
||||
printf '%s models.json (key-redacted comparison)\n' "${C_YELLOW}DIFF${C_OFF}"
|
||||
diff -u --label repo/models.json.template "$a" --label live/models.json "$b" | sed 's/^/ /'
|
||||
DIFFS=$((DIFFS + 1))
|
||||
fi
|
||||
rm -f "$a" "$b"
|
||||
else
|
||||
warn "live ~/.pi/agent/models.json is absent"
|
||||
DIFFS=$((DIFFS + 1))
|
||||
fi
|
||||
|
||||
local rel
|
||||
while IFS= read -r rel; do
|
||||
diff_file "$tracked/$rel" "$live/$rel" "$rel" || DIFFS=$((DIFFS + 1))
|
||||
done < <(cd "$tracked" && find extensions prompts -type f 2>/dev/null | sort)
|
||||
}
|
||||
|
||||
diff_scenario() {
|
||||
local name="$1" dir
|
||||
dir="$(require_scenario "$name")"
|
||||
local profile="$dir/profile.toml"
|
||||
[ -f "$profile" ] || die "scenario '$name' has no profile.toml"
|
||||
|
||||
local workspace
|
||||
workspace="$(toml_get "$profile" scenario workspace)"
|
||||
[ -n "$workspace" ] || die "scenario '$name': [scenario].workspace is not set"
|
||||
|
||||
head1 "$name ($workspace)"
|
||||
|
||||
if [ ! -d "$dir/workspace" ]; then
|
||||
warn "no tracked workspace/ for '$name'; nothing to compare"
|
||||
return
|
||||
fi
|
||||
if [ ! -d "$workspace" ]; then
|
||||
warn "live workspace $workspace does not exist yet"
|
||||
DIFFS=$((DIFFS + 1))
|
||||
return
|
||||
fi
|
||||
|
||||
local rel
|
||||
while IFS= read -r rel; do
|
||||
diff_file "$dir/workspace/$rel" "$workspace/$rel" "$rel" || DIFFS=$((DIFFS + 1))
|
||||
done < <(cd "$dir/workspace" && find . -type f -printf '%P\n' | sort)
|
||||
|
||||
# Report live files that the repository does not track, so drift is visible in
|
||||
# both directions. Runtime state and toolchains are expected and are skipped.
|
||||
local untracked=()
|
||||
while IFS= read -r rel; do
|
||||
case "$rel" in
|
||||
.ccgram-uploads/*|*/__pycache__/*|*/.venv/*|.venv/*|*.pyc|*.bak-*|\
|
||||
telegram-gateway/*|download.html|.pi/launch.json|*.log) continue ;;
|
||||
esac
|
||||
[ -f "$dir/workspace/$rel" ] || untracked+=("$rel")
|
||||
done < <(cd "$workspace" && find . -type f -printf '%P\n' | sort)
|
||||
|
||||
if [ "${#untracked[@]}" -gt 0 ]; then
|
||||
printf '%sUNTRACKED in live workspace:%s\n' "$C_DIM" "$C_OFF"
|
||||
printf ' %s\n' "${untracked[@]}"
|
||||
fi
|
||||
}
|
||||
|
||||
TARGET="${1:-}"
|
||||
if [ -z "$TARGET" ]; then
|
||||
diff_runtime
|
||||
while IFS= read -r s; do diff_scenario "$s"; done < <(list_scenarios)
|
||||
elif [ "$TARGET" = "runtime" ]; then
|
||||
diff_runtime
|
||||
else
|
||||
diff_scenario "$TARGET"
|
||||
fi
|
||||
|
||||
printf '\n'
|
||||
if [ "$DIFFS" -eq 0 ]; then
|
||||
ok "repository and live installation agree"
|
||||
else
|
||||
warn "$DIFFS difference(s) found"
|
||||
fi
|
||||
exit $((DIFFS > 0))
|
||||
Executable
+126
@@ -0,0 +1,126 @@
|
||||
#!/usr/bin/env bash
|
||||
# ---------------------------------------------------------------------------
|
||||
# Restore a backup produced by scripts/pi-backup.sh.
|
||||
#
|
||||
# Deliberately conservative:
|
||||
# - refuses to run unless the manifest verifies
|
||||
# - moves the current directory aside instead of overwriting it
|
||||
# - never restarts a service
|
||||
# - dry run by default
|
||||
#
|
||||
# Usage:
|
||||
# scripts/pi-restore.sh --from <dir> [--only <component>] [--apply]
|
||||
#
|
||||
# Components: runtime, sessions, workspace-<scenario>, sessions-<scenario>,
|
||||
# systemd. Secrets are never restored automatically -- unpack
|
||||
# secrets.tar.gz by hand so that each file lands where you intend.
|
||||
# ---------------------------------------------------------------------------
|
||||
# shellcheck source=lib/common.sh
|
||||
. "$(dirname "${BASH_SOURCE[0]}")/lib/common.sh"
|
||||
|
||||
FROM=""
|
||||
ONLY=""
|
||||
APPLY=0
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--from) FROM="${2:?--from needs a path}"; shift 2 ;;
|
||||
--only) ONLY="${2:?--only needs a component}"; shift 2 ;;
|
||||
--apply) APPLY=1; shift ;;
|
||||
-h|--help) sed -n '2,18p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) die "unknown argument: $1" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[ -n "$FROM" ] || die "--from <backup dir> is required"
|
||||
[ -d "$FROM" ] || die "no such directory: $FROM"
|
||||
[ -f "$FROM/MANIFEST.sha256" ] || die "no MANIFEST.sha256 in $FROM"
|
||||
|
||||
head1 "verify $FROM"
|
||||
( cd "$FROM" && sha256sum -c MANIFEST.sha256 ) || die "manifest verification failed; refusing to restore"
|
||||
ok "manifest verified"
|
||||
|
||||
STAMP="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
[ "$APPLY" -eq 1 ] || info "\n${C_DIM}(dry run; pass --apply to write)${C_OFF}"
|
||||
|
||||
# Move a live path aside rather than overwriting, so a bad restore is reversible.
|
||||
preserve() {
|
||||
local path="$1"
|
||||
[ -e "$path" ] || return 0
|
||||
local aside="$path.pre-restore-$STAMP"
|
||||
info " preserve $path -> $aside"
|
||||
[ "$APPLY" -eq 1 ] && mv "$path" "$aside"
|
||||
return 0
|
||||
}
|
||||
|
||||
extract() {
|
||||
local archive="$1" into="$2"
|
||||
info " extract $(basename "$archive") -> $into"
|
||||
if [ "$APPLY" -eq 1 ]; then
|
||||
install -d -m 700 "$into"
|
||||
tar -xzf "$archive" -C "$into"
|
||||
fi
|
||||
}
|
||||
|
||||
want() { [ -z "$ONLY" ] || [ "$ONLY" = "$1" ]; }
|
||||
|
||||
if want runtime && [ -f "$FROM/pi-agent-runtime.tar.gz" ]; then
|
||||
head1 "runtime"
|
||||
preserve "$HOME/.pi/agent"
|
||||
extract "$FROM/pi-agent-runtime.tar.gz" "$HOME/.pi"
|
||||
fi
|
||||
|
||||
if want sessions && [ -f "$FROM/pi-global-sessions.tar.gz" ]; then
|
||||
head1 "global sessions"
|
||||
extract "$FROM/pi-global-sessions.tar.gz" "$HOME/.pi/agent"
|
||||
fi
|
||||
|
||||
while IFS= read -r name; do
|
||||
profile="$REPO_ROOT/scenarios/$name/profile.toml"
|
||||
[ -f "$profile" ] || continue
|
||||
|
||||
if want "workspace-$name" && [ -f "$FROM/workspace-$name.tar.gz" ]; then
|
||||
workspace="$(toml_get "$profile" scenario workspace)"
|
||||
if [ -n "$workspace" ]; then
|
||||
head1 "workspace $name"
|
||||
preserve "$workspace"
|
||||
extract "$FROM/workspace-$name.tar.gz" "$(dirname "$workspace")"
|
||||
fi
|
||||
fi
|
||||
|
||||
if want "sessions-$name" && [ -f "$FROM/sessions-$name.tar.gz" ]; then
|
||||
session_dir="$(toml_get "$profile" scenario session_dir)"
|
||||
if [ -n "$session_dir" ]; then
|
||||
head1 "sessions $name"
|
||||
preserve "$session_dir"
|
||||
extract "$FROM/sessions-$name.tar.gz" "$(dirname "$session_dir")"
|
||||
fi
|
||||
fi
|
||||
done < <(list_scenarios)
|
||||
|
||||
if want systemd && [ -f "$FROM/systemd-units.tar.gz" ]; then
|
||||
head1 "systemd units"
|
||||
info " ${C_DIM}not extracted automatically: the archive holds every user unit,"
|
||||
info " and most are unrelated to Pi. Unpack and install selectively:${C_OFF}"
|
||||
info " mkdir /tmp/units && tar -xzf $FROM/systemd-units.tar.gz -C /tmp/units"
|
||||
info " install -m 600 /tmp/units/<unit> ~/.config/systemd/user/"
|
||||
info " systemctl --user daemon-reload"
|
||||
fi
|
||||
|
||||
if [ -f "$FROM/secrets.tar.gz" ]; then
|
||||
head1 "secrets"
|
||||
info " ${C_DIM}not restored automatically. Filenames are path-encoded; unpack and"
|
||||
info " place each file deliberately, then chmod 600:${C_OFF}"
|
||||
info " mkdir -m 700 /tmp/sec && tar -xzf $FROM/secrets.tar.gz -C /tmp/sec && ls /tmp/sec"
|
||||
fi
|
||||
|
||||
printf '\n'
|
||||
if [ "$APPLY" -eq 1 ]; then
|
||||
ok "restore applied; preserved copies carry the suffix .pre-restore-$STAMP"
|
||||
info ""
|
||||
info "Restart the affected gateways yourself, then check health:"
|
||||
info " systemctl --user restart curator.service pi-memo-telegram.service"
|
||||
info " curl -fsS http://127.0.0.1:8766/api/health"
|
||||
else
|
||||
info "${C_DIM}dry run complete; re-run with --apply${C_OFF}"
|
||||
fi
|
||||
Reference in New Issue
Block a user