feat(anc): add check-hotfix subcommand to read hotfix pointer from LPS#8696
Draft
Devinwong wants to merge 1 commit into
Draft
feat(anc): add check-hotfix subcommand to read hotfix pointer from LPS#8696Devinwong wants to merge 1 commit into
Devinwong wants to merge 1 commit into
Conversation
5fff98d to
061ba60
Compare
64e782d to
ede050a
Compare
0c90761 to
b33ec66
Compare
b33ec66 to
07b497b
Compare
Add a fail-open 'check-hotfix' CLI subcommand that reads the base->hotfix
pointer map from the live-patching-service (LPS) over the IMDS-attested SNI
path that is reachable pre-kubelet, and stages the resolved {hotfixes:{...}}
pointer to the path download-hotfix already reads. download-hotfix keeps its
unchanged patch-only, strictly-higher gating; check-hotfix only fetches and
writes the pointer.
- Raw net/http HTTPS GET (no client-go). TLS ServerName pinned to the LPS
SNI host while the TCP dial is forced to the apiserver FQDN (curl --resolve
trick); Authorization is the IMDS attested-data signature; the server cert
is verified against the cluster CA from the provision-config.
- FQDN + cluster CA come from the AKSNodeConfig ANC already parses (the only
credential source present pre-provisioning); caSource is logged.
- Shares the hotfixConfig parser/data contract with download-hotfix.
- Always exits 0; emits CheckHotfix telemetry (lpsRead, noHotfixForBase,
notEnrolled, customDataFallback, failed).
- A reachable LPS with nothing for this node (HTTP 401 pool-not-enrolled,
403, 404) is a benign no-op (notEnrolled): no overlay is staged and it is
never classified as a failure. Only transport/5xx failures fall back.
- PoC cold-start fallback reads a lenient top-level hotfixes object from the
node config when the LPS read fails (TODO: typed contract field).
- Injectable App fields (checkHotfixFetcher, fetchAttestedToken,
nodeConfigPath) for network-free unit tests.
- The LPS route + response schema are a planned-maintenance deliverable that
is not finalized; lpsHotfixPath is a clearly-marked placeholder with a TODO.
The IMDS/LPS client helpers mirror the connectivity prototype and should be
de-duplicated into a shared LPS client when that lands.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
07b497b to
0d6f945
Compare
Contributor
Changes cached containers or packages on windows VHDsPlease get a Windows SIG member to approve. The following dif file shows any additions or deletions from what will be cached on windows VHDs organised by VHD type.
diff --git a/vhd_files/2022-containerd-gen2.txt b/vhd_files/2022-containerd-gen2.txt
index 7039bac..c51a47f 100644
--- a/vhd_files/2022-containerd-gen2.txt
+++ b/vhd_files/2022-containerd-gen2.txt
@@ -122,0 +123 @@ mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.34.6-windows-hp
+mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.2-windows-hp
@@ -124 +124,0 @@ mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.3-windows-hp
-mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.4-windows-hp
diff --git a/vhd_files/2022-containerd.txt b/vhd_files/2022-containerd.txt
index 5915cf1..7312c49 100644
--- a/vhd_files/2022-containerd.txt
+++ b/vhd_files/2022-containerd.txt
@@ -122,0 +123 @@ mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.34.6-windows-hp
+mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.2-windows-hp
@@ -124 +124,0 @@ mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.3-windows-hp
-mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.4-windows-hp
diff --git a/vhd_files/2025-gen2.txt b/vhd_files/2025-gen2.txt
index 37d9326..36e3641 100644
--- a/vhd_files/2025-gen2.txt
+++ b/vhd_files/2025-gen2.txt
@@ -52,0 +53 @@ mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.34.6-windows-hp
+mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.2-windows-hp
@@ -54 +54,0 @@ mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.3-windows-hp
-mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.4-windows-hp
diff --git a/vhd_files/2025.txt b/vhd_files/2025.txt
index 5b08280..b8873d5 100644
--- a/vhd_files/2025.txt
+++ b/vhd_files/2025.txt
@@ -52,0 +53 @@ mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.34.6-windows-hp
+mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.2-windows-hp
@@ -54 +54,0 @@ mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.3-windows-hp
-mcr.microsoft.com/oss/v2/kubernetes-csi/azurefile-csi:v1.35.4-windows-hp |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a
check-hotfixsubcommand that reads the hotfix pointer from the live-patching-serviceThis adds a new fail-open
check-hotfixsubcommand toaks-node-controller. It reads a base-to-hotfix version pointer map from the live-patching-service (LPS) over an IMDS-attested HTTPS path that is reachable before kubelet is up, and writes that pointer to the filedownload-hotfixalready consumes.download-hotfixthen re-resolves the pointer and keeps its existing patch-only, strictly-higher gating.check-hotfixonly fetches and stages the pointer - it never installs anything and never blocks provisioning.Stacking
This branch is stacked on the base-to-version hotfix map change (PR #8694). The PR base is set to that branch so the diff shows only this change (
app.gowiring +checkhotfix.go+checkhotfix_test.go). It must merge after #8694; if #8694 merges first, retarget this PR tomain.This PR's
check-hotfixcommand is always-on by itself - it has no feature gate. The gating arrives later in 2.1d (#8717), which adds anenable_provisioning_hotfixcontract field socheck-hotfixself-gates on it.What it does
net/httpHTTPS GET (no client-go dependency). The transport mirrors the proven pre-kubelet connectivity path:ServerNameis pinned to the LPS SNI host, while a custom dialer forces the TCP connection to the apiserver FQDN on port 443 (thecurl --resolvetrick). The apiserver front routes the pinned SNI to the LPS backend.Authorizationis the IMDS attested-data document signature (http://169.254.169.254/metadata/attested/document). IMDS is reachable pre-kubelet, so this works before any kube credential exists. The token fetch is injectable for tests.check-hotfixruns before the provisioning scripts, so the decoded on-node CA file (/etc/kubernetes/certs/ca.crt) does not exist yet - the provision config is the only credential source guaranteed to be present at this point.{"hotfixes":{...}}JSON object, which unmarshals directly into the same config typedownload-hotfixuses, so both commands share one parser and data contract./opt/azure/containers/aks-node-controller-hotfix.jsonin the same{"hotfixes":{...}}shape (atomic temp-file + rename), sodownload-hotfixre-resolves it and applies its unchanged gating.notEnrolled.hotfixesobject embedded in the provision config and uses that. (Marked with a TODO to switch to a typed config field once that contract exists.)CheckHotfixwith outcomeslpsRead,noHotfixForBase,notEnrolled,customDataFallback,failed.Open dependency (placeholder route)
The LPS route and response schema for the base-to-hotfix pointer are a planned-maintenance deliverable that is not finalized yet; the connectivity prototype only proved reachability of the LPS read path. The route is a clearly-named placeholder constant (
lpsHotfixPath = "/v1/anc-hotfix") with a TODO, and the expected response body is documented as{"hotfixes":{"<YYYYMM.DD base>":"<YYYYMM.DD.PATCH>"}}. The IMDS/LPS client helpers mirror the connectivity prototype and are flagged in-code to be de-duplicated into a shared LPS client when that lands.Net effect (examples)
LPS serves the pointer body:
{"hotfixes":{"202604.01":"202604.01.1","202605.01":"202605.01.2"}}check-hotfixstages/opt/azure/containers/aks-node-controller-hotfix.json:{"hotfixes":{"202604.01":"202604.01.1","202605.01":"202605.01.2"}}hotfixesTests
New network-free unit tests (LPS fetcher and attested-token injected, no real networking): success read+write, benign not-enrolled 401/403/404 no-op (no overlay), timeout/connection/5xx fail-open, invalid pointer JSON fail-open,
noHotfixForBase, cold-start fallback (and no-pointer failure), telemetry outcomes and always-exit-0 wiring, shared-parser equivalence withdownload-hotfix, attested-token injection, provision-config endpoint/CA resolution, and the SNI-pinned TLS client (provision-config trust vs insecure fallback).All new tests pass. The full
go test ./...run shows no new failures versus the base branch. The only failures are pre-existing Windows-only environmental ones (they need/etc/os-release, bash, and unix file perms) that pass in Linux CI.Note: wiring this command into the provisioning wrapper script is intentionally out of scope for this PR and will land separately behind a feature flag.