feat(provision): 弹性节点基建 Terraform + 一键更换 (tsk_6u0FxmbC7Yeq)
IaC 面 (infra/) 与控制面 (server/internal/provision) 双产出，落地 doc/04 §4 「节点是牲口」弹性拓扑与 make-before-break 一键更换。 server/internal/provision: - CloudAdapter 适配层 + Registry（首发 vultr 消耗品池 / hetzner 精品池各一）； 厂商凭证仅从 PROVISION_<VENDOR>_* env 注入，不入库不入 git。 - ProvisionService：CreateNode(幂等键重放不重复开机)、DestroyNode(幂等)、 RotateIP(换 IP 不换机 + version bump)、ListProviders。 - Replace 一键更换：先建后拆，新机 up 先于旧机 draining（容量不下降）， replacement_uuid 幂等键 + replacements 表分步记录，崩溃可续跑不重复。 - RotatePool：池内滚动轮换，并发度 1–2。 - cmd/nodectl CLI：create/destroy/rotate-ip/replace/rotate-pool/providers。 - 单测（mock 厂商 API + 内存 Store）：幂等重放、make-before-break 时序断言、 开机失败/探活超时→destroyed+失败计数+告警钩子、崩溃续跑、RotatePool。 infra/: - terraform/：探针机 + 控制面基线模块化（probe / control-plane）+ README， 低频基线进 state，节点不进 Terraform。 - cloud-init/node.yaml.tmpl：节点引导模板（注入一次性 bootstrap token，task #5）。 - identity-isolation.md：身份隔离登记表（doc/06 §2 红线，无任何凭证）。 migrations/000008：nodes 增 provider_instance_id/elastic_ip_id、node_events 增 ip_rotated、provision_idempotency / replacements 表（附加式，不动现网）。 红线：仅面向新厂商池，绝不纳管现网生产 EC2(deploy/ marzban)。 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
wangjia
@@ -0,0 +1,30 @@
|
||||
# Module: `control-plane`
|
||||
|
||||
Provisions the **control-plane baseline host** (API + MySQL + Redis), doc/04 §5.2.
|
||||
|
||||
Key properties:
|
||||
|
||||
- Deployed on a **stable cloud unrelated to the data-plane node vendors**. Nodes
|
||||
can all be destroyed without affecting the control plane, and vice-versa. The
|
||||
control plane carries **no proxy traffic**.
|
||||
- `prevent_destroy = true` — it holds the user DB; it is never recreated
|
||||
implicitly. Application bring-up (containers, migrations) is done by the server
|
||||
deploy pipeline, not Terraform.
|
||||
- Admin/SSH surface is **never on the open internet** (doc/06 §2): inbound is
|
||||
locked to `allowlist_cidrs`; the public API sits behind a CDN/proxy terminated
|
||||
elsewhere.
|
||||
|
||||
Disabled by default in the root module (`control_plane_enabled = false`) because
|
||||
it changes rarely.
|
||||
|
||||
## Inputs
|
||||
|
||||
| name | description |
|
||||
|------|-------------|
|
||||
| `server_type` / `location` / `image` | host sizing |
|
||||
| `ssh_key_ids` | authorised SSH keys |
|
||||
| `allowlist_cidrs` | admin/SSH inbound allowlist (tighten in prod) |
|
||||
|
||||
## Outputs
|
||||
|
||||
`ipv4_address`, `server_id`.
|
||||
@@ -0,0 +1,53 @@
|
||||
terraform {
|
||||
required_providers {
|
||||
hcloud = {
|
||||
source = "hetznercloud/hcloud"
|
||||
version = "~> 1.45"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Control-plane baseline host (API + MySQL + Redis). Deployed on a stable cloud
|
||||
# COMPLETELY UNRELATED to the data-plane node vendors (doc/04 §5.2): nodes can
|
||||
# all die without touching the control plane, and vice-versa. Carries NO proxy
|
||||
# traffic.
|
||||
#
|
||||
# Provisioned once and rarely changed — hence Terraform-managed. Application
|
||||
# bring-up (containers, migrations) is handled separately by the server deploy
|
||||
# pipeline, not here.
|
||||
resource "hcloud_server" "control_plane" {
|
||||
name = "pangolin-control-plane"
|
||||
server_type = var.server_type
|
||||
image = var.image
|
||||
location = var.location
|
||||
ssh_keys = var.ssh_key_ids
|
||||
|
||||
labels = {
|
||||
role = "control-plane"
|
||||
project = "pangolin"
|
||||
}
|
||||
|
||||
lifecycle {
|
||||
# Never recreate the control plane implicitly — it holds the user DB.
|
||||
prevent_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# Admin/SSH surface is never on the open internet (doc/06 §2). Inbound is locked
|
||||
# to the management allowlist; the public API is expected to sit behind a CDN /
|
||||
# reverse proxy terminated elsewhere.
|
||||
resource "hcloud_firewall" "control_plane" {
|
||||
name = "pangolin-control-plane"
|
||||
|
||||
rule {
|
||||
direction = "in"
|
||||
protocol = "tcp"
|
||||
port = "22"
|
||||
source_ips = length(var.allowlist_cidrs) > 0 ? var.allowlist_cidrs : ["0.0.0.0/0", "::/0"]
|
||||
}
|
||||
}
|
||||
|
||||
resource "hcloud_firewall_attachment" "control_plane" {
|
||||
firewall_id = hcloud_firewall.control_plane.id
|
||||
server_ids = [hcloud_server.control_plane.id]
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
output "ipv4_address" {
|
||||
description = "Public IPv4 of the control-plane host."
|
||||
value = hcloud_server.control_plane.ipv4_address
|
||||
}
|
||||
|
||||
output "server_id" {
|
||||
description = "Vendor server ID."
|
||||
value = hcloud_server.control_plane.id
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
variable "server_type" {
|
||||
description = "Vendor server type/size for the control-plane host."
|
||||
type = string
|
||||
default = "cpx21"
|
||||
}
|
||||
|
||||
variable "location" {
|
||||
description = "Vendor location/datacenter."
|
||||
type = string
|
||||
default = "hel1"
|
||||
}
|
||||
|
||||
variable "image" {
|
||||
description = "OS image."
|
||||
type = string
|
||||
default = "debian-12"
|
||||
}
|
||||
|
||||
variable "ssh_key_ids" {
|
||||
description = "Vendor SSH key IDs authorised on the host."
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "allowlist_cidrs" {
|
||||
description = "CIDRs allowed to reach SSH/admin. Empty = open (MUST be tightened in prod; doc/06 §2)."
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
# Module: `probe`
|
||||
|
||||
Provisions **one overseas reference probe point** (doc/04 §1, §4.2).
|
||||
|
||||
Probes are low-frequency baseline resources, so they live in Terraform state —
|
||||
unlike data-plane nodes, which are disposable cattle managed by the provision
|
||||
service via vendor APIs (never in Terraform).
|
||||
|
||||
## What it creates
|
||||
|
||||
- `hcloud_server` running the probe agent (installed out-of-band; the probe
|
||||
holds **no** data-plane secrets and **no** user data).
|
||||
- A firewall locking inbound to SSH (optionally CIDR-restricted).
|
||||
|
||||
## Role in the system
|
||||
|
||||
Overseas probes cross-check the in-China probe fleet so the scheduler can tell
|
||||
**blocked** (in-China fails, overseas OK) from **machine down** (both fail),
|
||||
avoiding false positives before triggering an automatic replacement (doc/04 §4.2).
|
||||
|
||||
> In-China probe points use serverless / third-party dialing services under a
|
||||
> separate isolated identity and are **out of Terraform scope** — registered in
|
||||
> [`../../identity-isolation.md`](../../identity-isolation.md).
|
||||
|
||||
## Inputs
|
||||
|
||||
| name | description |
|
||||
|------|-------------|
|
||||
| `name` | short id (e.g. `eu`, `us`) |
|
||||
| `location` | vendor location |
|
||||
| `server_type` | vendor size |
|
||||
| `image` | OS image |
|
||||
| `ssh_key_ids` | authorised SSH key IDs |
|
||||
| `ssh_allowlist_cidrs` | inbound SSH allowlist |
|
||||
|
||||
## Outputs
|
||||
|
||||
`ipv4_address`, `server_id`.
|
||||
@@ -0,0 +1,57 @@
|
||||
terraform {
|
||||
required_providers {
|
||||
hcloud = {
|
||||
source = "hetznercloud/hcloud"
|
||||
version = "~> 1.45"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# One overseas reference probe point (doc/04 §4.2). It runs the probe script and
|
||||
# reports TCP/TLS/real-handshake reachability of data-plane nodes back to the
|
||||
# control plane, used to distinguish "blocked" from "machine down".
|
||||
resource "hcloud_server" "probe" {
|
||||
name = "pangolin-probe-${var.name}"
|
||||
server_type = var.server_type
|
||||
image = var.image
|
||||
location = var.location
|
||||
ssh_keys = var.ssh_key_ids
|
||||
|
||||
labels = {
|
||||
role = "probe"
|
||||
project = "pangolin"
|
||||
scope = "overseas-reference"
|
||||
}
|
||||
|
||||
# Probe bootstrap: pin + install the probe agent out-of-band. Kept minimal;
|
||||
# probes hold NO data-plane secrets and NO user data.
|
||||
user_data = <<-EOT
|
||||
#cloud-config
|
||||
package_update: true
|
||||
runcmd:
|
||||
- sed -i 's/^#\?PermitRootLogin.*/PermitRootLogin prohibit-password/' /etc/ssh/sshd_config
|
||||
- sed -i 's/^#\?PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
|
||||
- systemctl restart sshd || true
|
||||
EOT
|
||||
|
||||
lifecycle {
|
||||
create_before_destroy = true
|
||||
}
|
||||
}
|
||||
|
||||
# Firewall: probes only need outbound; lock inbound to SSH from management.
|
||||
resource "hcloud_firewall" "probe" {
|
||||
name = "pangolin-probe-${var.name}"
|
||||
|
||||
rule {
|
||||
direction = "in"
|
||||
protocol = "tcp"
|
||||
port = "22"
|
||||
source_ips = length(var.ssh_allowlist_cidrs) > 0 ? var.ssh_allowlist_cidrs : ["0.0.0.0/0", "::/0"]
|
||||
}
|
||||
}
|
||||
|
||||
resource "hcloud_firewall_attachment" "probe" {
|
||||
firewall_id = hcloud_firewall.probe.id
|
||||
server_ids = [hcloud_server.probe.id]
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
output "ipv4_address" {
|
||||
description = "Public IPv4 of the probe machine."
|
||||
value = hcloud_server.probe.ipv4_address
|
||||
}
|
||||
|
||||
output "server_id" {
|
||||
description = "Vendor server ID."
|
||||
value = hcloud_server.probe.id
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
variable "name" {
|
||||
description = "Short probe identifier (e.g. eu, us)."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "location" {
|
||||
description = "Vendor location/datacenter for the probe."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "server_type" {
|
||||
description = "Vendor server type/size."
|
||||
type = string
|
||||
default = "cpx11"
|
||||
}
|
||||
|
||||
variable "image" {
|
||||
description = "OS image."
|
||||
type = string
|
||||
default = "debian-12"
|
||||
}
|
||||
|
||||
variable "ssh_key_ids" {
|
||||
description = "Vendor SSH key IDs authorised on the probe."
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "ssh_allowlist_cidrs" {
|
||||
description = "CIDRs allowed to SSH the probe. Empty = open (tighten in prod)."
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
Reference in New Issue
Block a user