Skip to content

kubeadm upgrade from v1.28.0 to v1.28.3 fails #2957

Description

@alexarefev

What happened?

The following command

kubeadm upgrade apply v1.28.3 -f --certificate-renewal=true --ignore-preflight-errors='CoreDNSUnsupportedPlugins,Port-6443' --patches=/etc/kubernetes/patches

fails with the error:

[upgrade/apply] FATAL: fatal error when trying to upgrade the etcd cluster, rolled the state back to pre-upgrade state: couldn't upgrade control plane. kubeadm has tried to recover everything into the earlier state. Errors faced: static Pod hash for component etcd on Node qa-fullha-master1 did not change after 5m0s: timed out waiting for the condition

if the kubeadm v1.28.3

The kubeadm v1.28.0 upgrades cluster successfully

What did you expect to happen?

The kubeadm v1.28.3 upgrades cluster successfully

How can we reproduce it (as minimally and precisely as possible)?

Download kubeadm v1.28.3 and run upgrade of Kubernetes v1.28.0

Anything else we need to know?

The issue might be fixed by the --etcd-upgrade flag.

Kubernetes version

Details
$ kubectl version
Client Version: v1.28.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.28.3

Cloud provider

Details Not applicable

OS version

Details
# On Linux:
$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
$ uname -a
Linux ubuntu 5.15.0-43-generic kubernetes/kubernetes#46-Ubuntu SMP Tue Jul 12 10:30:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Install tools

Details

Container runtime (CRI) and version (if applicable)

Details

Related plugins (CNI, CSI, ...) and versions (if applicable)

Details

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/upgradeskind/bugCategorizes issue or PR as related to a bug.lifecycle/frozenIndicates that an issue or PR should not be auto-closed due to staleness.priority/backlogHigher priority than priority/awaiting-more-evidence.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions