wangjia
4d67a90659
Deploy Server / deploy-server (push) Failing after 22s
隧道 /healthz 依赖 cloudflared/CF 边缘,与本次二进制是否健康是两回事; CF 边缘抖动或隧道尚未 provision 不应判整次部署失败。改为失败仅告警。 (最终 whole-branch review 的 Minor A 决议) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013nMthbVEmQquxBRKb9Fj8u
70 lines
3.5 KiB
Bash
70 lines
3.5 KiB
Bash
#!/usr/bin/env bash
|
|
# deploy-server.sh <tag> — deploy the pangolin control-plane binaries
|
|
# (pangolin-server / pangolin-agent / pangolin-migrate) to pangolin1, in the
|
|
# exact manual sequence used for F3/F4: stop → wal checkpoint → backup db →
|
|
# migrate (rollback db + restart old binary on failure) → swap binaries →
|
|
# start → healthcheck. Assumes compile-backend.sh has already produced
|
|
# server/out/{pangolin-server,pangolin-agent,pangolin-migrate}.
|
|
#
|
|
# Usage: scripts/ci/deploy-server.sh <tag> (e.g. server-v1.2.3)
|
|
# Requires env: DEPLOY_SSH_KEY (see lib-ssh.sh).
|
|
set -euo pipefail
|
|
|
|
# shellcheck source=scripts/ci/lib-ssh.sh
|
|
. scripts/ci/lib-ssh.sh
|
|
|
|
DB=/var/lib/pangolin/pangolin.db
|
|
BIN=/usr/local/bin
|
|
TAG="${1:?usage: deploy-server.sh <tag>}"
|
|
|
|
# Refuse anything that isn't a strict server-vX.Y.Z[-suffix] tag before it can
|
|
# reach the remote heredoc / backup paths below (command-injection guard).
|
|
# An anchored regex is used instead of a `case` glob: a trailing `*` in a
|
|
# case pattern matches ANY trailing characters (including shell metachars
|
|
# like `; rm -rf /`), which would defeat the point of this check.
|
|
if ! [[ "$TAG" =~ ^server-v[0-9]+\.[0-9]+\.[0-9]+(-[A-Za-z0-9.]+)?$ ]]; then
|
|
echo "deploy-server: refusing unexpected tag '$TAG'" >&2
|
|
exit 1
|
|
fi
|
|
|
|
# setup_ssh registers the EXIT cleanup trap itself (before writing the key),
|
|
# so a mid-setup failure still cleans up — see lib-ssh.sh. It exports
|
|
# SSH_KEY_FILE / DEPLOY_PORT / SSH_KNOWN_HOSTS_FILE / DEPLOY_HOST used below
|
|
# to build SCP (mirroring the SSH/RSYNC_SSH command-string convention).
|
|
setup_ssh
|
|
SCP="scp -i ${SSH_KEY_FILE} -P ${DEPLOY_PORT} -o StrictHostKeyChecking=accept-new -o UserKnownHostsFile=${SSH_KNOWN_HOSTS_FILE}"
|
|
|
|
echo "==> deploy-server: tag=${TAG} host=${DEPLOY_HOST}"
|
|
echo "==> deploy-server: uploading binaries to ${DEPLOY_HOST}:/tmp/"
|
|
$SCP server/out/pangolin-server server/out/pangolin-agent server/out/pangolin-migrate "root@${DEPLOY_HOST}:/tmp/"
|
|
|
|
$SSH "root@${DEPLOY_HOST}" "bash -s" <<REMOTE
|
|
set -euo pipefail
|
|
systemctl stop pangolin-server
|
|
runuser -u pangolin -- sqlite3 "$DB" 'PRAGMA wal_checkpoint(TRUNCATE);'
|
|
cp -p "$DB" "$DB.bak-pre-$TAG"
|
|
if ! runuser -u pangolin -- env DB_DRIVER=sqlite DB_DSN=$DB /tmp/pangolin-migrate up; then
|
|
echo "!! migrate 失败,回滚"; cp -p "$DB.bak-pre-$TAG" "$DB"; systemctl start pangolin-server; exit 1
|
|
fi
|
|
cp -p "$BIN/pangolin-server" "$BIN/pangolin-server.bak-$TAG" || true
|
|
install -m755 /tmp/pangolin-server "$BIN/pangolin-server"
|
|
install -m755 /tmp/pangolin-agent "$BIN/pangolin-agent"
|
|
install -m755 /tmp/pangolin-migrate "$BIN/pangolin-migrate"
|
|
systemctl start pangolin-server
|
|
systemctl is-active pangolin-server
|
|
REMOTE
|
|
|
|
# 8080 现仅 loopback(经 cloudflared 隧道对外)。
|
|
# 本地 /healthz 是本次二进制部署成败的**权威闸**:新 server 起来即通过。
|
|
$SSH "root@${DEPLOY_HOST}" 'curl -fsS -m 10 --retry 5 --retry-connrefused http://127.0.0.1:8080/healthz >/dev/null && echo "healthz(local) OK"'
|
|
# 隧道 /healthz 是端到端冒烟(最贴近真实客户端路径),但**非致命**:它依赖 cloudflared/CF 边缘,
|
|
# 与「本次二进制是否健康」是两回事——CF 边缘抖动或隧道尚未 provision 不应判整次部署失败
|
|
# (本地闸已证明 server 健康)。失败只告警,不 exit。
|
|
if curl -fsS -m 10 --retry 3 "https://api.yanmeiai.com/healthz" >/dev/null; then
|
|
echo "healthz(tunnel) OK"
|
|
else
|
|
echo "==> deploy-server: 警告 —— 隧道 https://api.yanmeiai.com/healthz 不通(CF 边缘抖动/隧道未就绪?);本地 healthz 已通过,不阻断部署。" >&2
|
|
fi
|
|
|
|
echo "==> deploy-server: done"
|