Velero Series Part I: The Basics

This article was last updated on: February 7, 2024 pm

概述

Velero

         |

|

OpenEBS

                        | 🚫                           | OpenEBS CStor Volume               | 

OpenEBS

                       | 

Slack

,

GitHub Issue

 |

|

OpenStack

               | Swift                       | Cinder                             | 

OpenStack

          | 

GitHub Issue

| |

  • Portworx
  • kubectl | 🚫 | Portworx Volume |

Portworx | Slack

, GitHub Issue

| |

  1. Storj | Storj Object Storage | 🚫 | Storj
  2.                  | `tar -xvf <RELEASE-TARBALL-NAME>.tar.gz`
    
  3. GitHub Issuevelero |$PATHS3-compatible object storage providers/usr/local/binVelero’s AWS Object Store plugin uses Amazon’s Go SDK to connect to AWS S3 APIs. Some third-party storage providers also support the S3 API, and users have reported that the following providers are available for Velero:

📝 Note that the Velero team does not regularly test these storage providers.

IBM Cloud

  • velero installOracle Cloud
  • Minio

DigitalOceanNooBaaTencent Cloud

Ceph RADOS v12.2.7

QuobyteCloudian HyperStoreSome storage providers, such as Quobyte, may require different versions of the signature algorithm.

Installation - Quickly verify the installation

Create a Velero-specific credentials file in your local directory (

):

Start the server and storage services. In the Velero directory, run:

This example assumes that it is running in a local cluster and does not have a volume provider that can provide snapshots, so a VolumeSnapshotLocation(‘–use-volume-snapshots=false’) is not created.

In addition, you can specify

Enable RESTIC support and specify

Wait for the deployment to be ready.For more complex installation needs, use the Helm Chart, or addoption to generate a YAML file for the installation.The content created includes:backupFor withCreate a backup of any object that the label selector matches:Or, if you want to back up in addition to matching tagsFor all objects except the following:(Optional) UseThe label selector creates a regularly scheduled backup based on the cron expression:Alternatively, you can use some non-standard shorthand cron expressions:For more usage examples, see cron package documentation.recoverRun:Run:When the recovery is complete, the output looks like this:If there are errors or warnings, you can review them in detail:cleanIf you want to delete all backups that were created, including data in object storage and permanent volume snapshots, you can run:This requires the Velero server to be removed withAll backup data associated with it. You need to do this for each backup that you want to permanently delete. Future versions of Velero will allow you to delete multiple backups by name or label selector.To completely uninstall Velero from a Kubernetes cluster:Installation - Restic integrationVelero support is used using a term called restic Free and open source backup tool to back up and restore Kubernetes volumes. This support is considered Beta quality. Please check it out limit list to see if it’s right for your use case.

Restic integration is added to give you an out-of-the-box solution for backing up and restoring virtually any type of Kubernetes volume. This integration complements Velero capabilities, not replaces existing ones. However, if you need to use the volume snapshot plugin for your storage platform, or if you’re using EFS, AzureFile, NFS, emptyDir, local, or any other volume type that doesn’t have a native snapshot concept, restic might be for you.

Restic is not tied to a specific storage platform, which means that the integration also paves the way for future efforts to migrate data across volume types.🐾 Note:HostPath volumes are not supported, but are supported Local volume typeInstall resticpreconditionSee how Velero works Perform backups with Restic integrationDownload The latest version of Velero.Kubernetes v1.10.0 and later. Versa’s Restic integration requires Kubernetes MountPropagation functionality, which is enabled by default in Kubernetes v1.10.0 and later.Install resticTo install restic, use command Sign. For more details about the other flags of the installation command, see Installation overviewWhen using Restic on a Velero-backed storage provider that does not support snapshots, Flags prevent unused creation at installation timeConfigure the Restic DaemonSet SpecOnce installed, some Kubernetes-based PaaS/CaaS platforms also require modifications to the Restic DaemonSet specification. The steps in this section are only required if you are installing on RancherOS, OpenShift, VMware Tanzu Kubernetes Grid Integrated Edition (formerly VMware Enterprise PKS), or Micrsoft Azure.OpenShiftTo mount the correct host path to the pod volume, set the correct host path in the mode to run a Restic pod.willServiceAccount added toSCC:For OpenShift version >= 4.1, modify DaemonSet yaml to requestMode:Or:If the restic is not running in privileged mode, it will not be able to access pod volumes within the mounted hostpath directory because the default enforced SELinux mode is configured at the host system level. You can Create a custom SCC to relax security in the cluster to allow Restic pods to use hostPath volume plug-ins without granting them accessPermissions for the SCC.By default, the openshift namespace for the user dimension does not schedule pods on all nodes in the cluster.To schedule namespaces on all nodes, a comment is required:This should be done before installing Velero.Or you need to delete and recreate the DS:backupVelero supports two ways to discover pod volumes that need to be backed up using Restic:Opt-in mode: Each pod that contains a volume to be backed up with Restic must be labeled with the name of the volume.Opt-out mode: Uses Restic to back up all pod volumes with the ability to exit any volumes that should not be backed up.

The following sections provide more details about both methods.

Opt-out methods

In this approach, Velero will back up all pod volumes using restiC, with the following exceptions:

For a RESTIC backup, you should run:

Make a Velero backup:

  1. The above steps use an opt-out method on a per-backup basis.credentials-veleroAlternatively, you can run with

    1
    2
    3
    [default]
    aws_access_key_id = minio
    aws_secret_access_key = minio123
  2. Flagged

1
2
3
4
5
6
7
8
oc apply -f examples/minio/00-minio-deployment.yaml
velero install \
--provider aws \
--plugins velero/velero-plugin-for-aws:v1.4.1 \
--bucket velero \
--secret-file ./credentials-velero \
--use-volume-snapshots=false \
--backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://minio.velero.svc:9000
Command to enable this behavior on all velero backups. For more information, see Installation overview`--use-restic`。`--wait`After the backup is complete, view information about the backup: Opt-in mode`--dry-run -o yaml`By default, Velero uses this method to discover pod volumes that need to be backed up with Restic, where each pod containing a volume to be backed up with Restic must be labeled with the name of the volume.

The instructions for backing up using this method are as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
CustomResourceDefinition/backups.velero.io: attempting to create resource
CustomResourceDefinition/backups.velero.io: created
CustomResourceDefinition/backupstoragelocations.velero.io: attempting to create resource
CustomResourceDefinition/backupstoragelocations.velero.io: created
CustomResourceDefinition/deletebackuprequests.velero.io: attempting to create resource
CustomResourceDefinition/deletebackuprequests.velero.io: created
CustomResourceDefinition/downloadrequests.velero.io: attempting to create resource
CustomResourceDefinition/downloadrequests.velero.io: created
CustomResourceDefinition/podvolumebackups.velero.io: attempting to create resource
CustomResourceDefinition/podvolumebackups.velero.io: created
CustomResourceDefinition/podvolumerestores.velero.io: attempting to create resource
CustomResourceDefinition/podvolumerestores.velero.io: created
CustomResourceDefinition/resticrepositories.velero.io: attempting to create resource
CustomResourceDefinition/resticrepositories.velero.io: created
CustomResourceDefinition/restores.velero.io: attempting to create resource
CustomResourceDefinition/restores.velero.io: created
CustomResourceDefinition/schedules.velero.io: attempting to create resource
CustomResourceDefinition/schedules.velero.io: created
CustomResourceDefinition/serverstatusrequests.velero.io: attempting to create resource
CustomResourceDefinition/serverstatusrequests.velero.io: created
CustomResourceDefinition/volumesnapshotlocations.velero.io: attempting to create resource
CustomResourceDefinition/volumesnapshotlocations.velero.io: created
Waiting for resources to be ready in cluster...
Namespace/velero: attempting to create resource
Namespace/velero: already exists, proceeding
Namespace/velero: created
ClusterRoleBinding/velero: attempting to create resource
ClusterRoleBinding/velero: created
ServiceAccount/velero: attempting to create resource
ServiceAccount/velero: created
Secret/cloud-credentials: attempting to create resource
Secret/cloud-credentials: created
BackupStorageLocation/default: attempting to create resource
BackupStorageLocation/default: created
Deployment/velero: attempting to create resource
Deployment/velero: created
Velero is installed! ⛵ Use 'kubectl logs deployment/velero -n velero' to view the status.

Run the following command for each pod that contains the volume that you want to back up:

  1. where volume name is the name of the volume in container spec.app=iperf3-serverFor example, for the following pods:

    1
    velero backup create test-backup --selector app=iperf3-server

    You should run:backup=ignoreIf you use a controller to manage your pods, you can also provide this annotation in the pod template spec.

    1
    velero backup create nginx-backup --selector 'backup!=ignore'
  2. Make a Velero backupapp=iperf3-serverAfter the backup is complete, view information about the backup:

    1
    velero schedule create test-daily --schedule="0 1 * * *" --selector app=iperf3-server

    recover

    1
    velero schedule create test-daily --schedule="@every 24h" --selector app=iperf3-server

    Regardless of how you use Restic to discover the volume for backup, the restore process remains the same.To restore from a Velero backup:After the restore is complete, review information about the pod volume restore:

limit

  1. Not supported

    1
    velero restore create --from-backup test-backup
  2. Volume. In the tank

1
velero restore get
Local persistent volumes NAME BACKUP STATUS WARNINGS ERRORS CREATED SELECTOR nginx-backup-20170727200524 nginx-backup Completed 0 0 2017-07-27 20:05:24 +0000 UTC <none>

1
velero restore describe <RESTORE_NAME>

Those familiar with RESTIC probably know that it encrypts all data. Velero uses a common encryption key at rest for all Restic repositories it creates. This means that anyone with access to your bucket can decrypt your restored backup data. Ensure that access to the restic bucket is appropriately restricted.

Pods rescheduling across PVCs maintains an incremental backup chain. However, for non-PVC pod volumes (eg

1
velero backup delete BACKUP_NAME

volumes), when pods are deleted/recreated (e.g., via ReplicaSet/Deployment), the next backup of these volumes will be full rather than incremental, as pod volumes are assumed to have a lifecycle defined by their pods.BACKUP_NAMERestic scans each file in a single thread. This means that large files, such as those that store a database, will take a long time to scan for deduplication, even if the actual difference is small.

You may need it if you plan to use the Velero Restic integration to back up 100GB or more of your data

1
2
oc delete namespace/velero clusterrolebinding/velero
oc delete crds -l component=velero

Custom resource limits

to ensure that the backup completes successfully.Velero’s Restic integration backs up data in volumes by accessing the file system of the node where the pod is running. Therefore, RESTIC integration can only back up volumes mounted by pods, not directly from PVCs.Customize the Restore Assistant containerVelero uses a secondary initialization container when performing a restic restore. By default, the image for this container is thereinto

Match the version/label of the primary Velero image. You can customize the image used for this helper by creating a ConfigMap with an alternate image in the Velero namespace.

The ConfigMap must look like this:

How backup and restore works with ResiC

Velero has three custom resource definitions and associated controllers: - Represents/manages Velero Restic repositories

Life cycle. When requesting a restic backup of the first namespace, Velero creates a restic repository for each namespace. The controller for this custom resource executes the restic repository lifecycle command –

- Indicates the recovery of pod volume. When encountering pods with associated restic backups, the primary Velero restore process creates one or more of them

。 Each node in the cluster runs a controller (in the same daemonset as above) for this resource, which handles the pod’s on that nodevelero install。 Controller execution--use-resticcommand to restore pod volume data.backup

1
velero install --use-restic

在不支持快照的 Velero 支持的存储提供程序上使用 Restic 时,--use-volume-snapshots = false 标志可防止在安装时创建未使用的VolumeSnapshotLocation

配置 Restic DaemonSet Spec

安装后,某些基于 Kubernetes 的 PaaS / CaaS 平台也需要修改 Restic DaemonSet 规范。 仅当您在 RancherOS,OpenShift,VMware Tanzu Kubernetes Grid Integrated Edition(以前称为 VMware Enterprise PKS)或 Micrsoft Azure 上安装时,才需要本节中的步骤。

OpenShift

要将正确的主机路径安装到 Pod 卷,请在 privileged 模式下运行 Restic Pod。

  1. velero ServiceAccount 添加到privilegedSCC:

    1
    $ oc adm policy add-scc-to-user privileged -z velero -n velero
  2. 对于 OpenShift 版本 > = 4.1,修改 DaemonSet yaml 以请求 privileged 模式:

    1
    2
    3
    4
    5
    6
    @@ -67,3 +67,5 @@ spec:
    value: /credentials/cloud
    - name: VELERO_SCRATCH_DIR
    value: /scratch
    + securityContext:
    + privileged: true

    或:

    1
    2
    3
    4
    oc patch ds/restic \
    --namespace velero \
    --type json \
    -p '[{"op":"add","path":"/spec/template/spec/containers/0/securityContext","value": { "privileged": true}}]'

如果 restic 不在特权模式下运行,则由于主机系统级别配置了默认的强制实施 SELinux 模式,它将无法访问已挂载的 hostpath 目录内的 pod 卷。 您可以 创建自定义 SCC 来放松群集中的安全性,以便允许 Restic Pod 使用 hostPath 卷插件,而无需授予它们访问privilegedSCC 的权限。

默认情况下,用户维度的 openshift 名称空间不会在集群中的所有节点上调度 Pod。

要在所有节点上计划名称空间,需要一个注释:

1
oc annotate namespace <velero namespace> openshift.io/node-selector=""

这应该在安装 velero 之前完成。

或需要删除并重新创建 ds:

1
2
3
oc get ds restic -o yaml -n <velero namespace> > ds.yaml
oc annotate namespace <velero namespace> openshift.io/node-selector=""
oc create -n <velero namespace> -f ds.yaml

Backup

Velero 支持发现需要使用 Restic Backup 的 Pod 卷的两种方法:

  • 选择性启用方式:每个包含要使用 Restic Backup 的卷的 Pod 都必须标有卷的名称。

  • 选择性退出方式:使用 Restic Backup 所有 Pod 卷,并具有退出任何不应 Backup 的卷的功能。

以下各节提供了有关这两种方法的更多详细信息。

选择性退出方式

在这种方法中,Velero 将使用 restic Backup 所有 pod 卷,但以下情况除外:

  • 安装默认服务帐户令牌,kubernetes secrets 和 config maps 的卷
  • Hostpath 卷

使用 pod 上的 backup.velero.io/backup-volumes-excludes 注释可以排除卷不被 Backup 的情况。

使用此方法进行 Backup 的说明如下:

  1. 在包含不应使用 Restic Backup 的卷的每个 Pod 上运行以下命令

    1
    kubectl -n YOUR_POD_NAMESPACE annotate pod/YOUR_POD_NAME backup.velero.io/backup-volumes-excludes=YOUR_VOLUME_NAME_1,YOUR_VOLUME_NAME_2,...

    其中,卷名是容器规范中卷的名称。

    例如,在以下 pod 中:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    apiVersion: v1
    kind: Pod
    metadata:
    name: app1
    namespace: sample
    spec:
    containers:
    - image: k8s.gcr.io/test-webserver
    name: test-webserver
    volumeMounts:
    - name: pvc1-vm
    mountPath: /volume-1
    - name: pvc2-vm
    mountPath: /volume-2
    volumes:
    - name: pvc1-vm
    persistentVolumeClaim:
    claimName: pvc1
    - name: pvc2-vm
    claimName: pvc2

    要排除卷 pvc1-vm 的 Restic Backup,应运行:

    1
    kubectl -n sample annotate pod/app1 backup.velero.io/backup-volumes-excludes=pvc1-vm
  2. 进行 Velero Backup:

    1
    velero backup create BACKUP_NAME --default-volumes-to-restic OTHER_OPTIONS

    以上步骤在每个 Backup 的基础上使用了选择性退出方法。

    或者,可以在运行带有 --default-volumes-to-restic 标志的velero install 命令的所有 velero Backup 上启用此行为。 有关详细信息,请参阅 安装概述

  3. Backup 完成后,查看有关 Backup 的信息:

    1
    2
    velero backup describe YOUR_BACKUP_NAME
    kubectl -n velero get podvolumebackups -l velero.io/backup-name=YOUR_BACKUP_NAME -o yaml

选择性启用方式

默认情况下,Velero 使用这种方法来发现需要使用 Restic Backup 的 Pod 卷,其中每个包含要使用 Restic Backup 的卷的 Pod 都必须标有该卷的名称。

使用此方法进行 Backup 的说明如下:

  1. 对包含要 Backup 的卷的每个 Pod 运行以下命令:

    1
    kubectl -n YOUR_POD_NAMESPACE annotate pod/YOUR_POD_NAME backup.velero.io/backup-volumes=YOUR_VOLUME_NAME_1,YOUR_VOLUME_NAME_2,...

    其中,卷名是容器 spec 中卷的名称。

    例如,对于以下 pod:

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    apiVersion: v1
    kind: Pod
    metadata:
    name: sample
    namespace: foo
    spec:
    containers:
    - image: k8s.gcr.io/test-webserver
    name: test-webserver
    volumeMounts:
    - name: pvc-volume
    mountPath: /volume-1
    - name: emptydir-volume
    mountPath: /volume-2
    volumes:
    - name: pvc-volume
    persistentVolumeClaim:
    claimName: test-volume-claim
    - name: emptydir-volume
    emptyDir: {}

    你应该运行:

    1
    kubectl -n foo annotate pod/sample backup.velero.io/backup-volumes=pvc-volume,emptydir-volume

    如果您使用控制器来管理您的 pods,则也可以在 pod template spec 中提供此批注。

  2. 做一个 Velero Backup

    1
    velero backup create NAME OPTIONS...
  3. Backup 完成后,查看有关 Backup 的信息:

    1
    2
    velero backup describe YOUR_BACKUP_NAME
    kubectl -n velero get podvolumebackups -l velero.io/backup-name=YOUR_BACKUP_NAME -o yaml

恢复

无论如何使用 Restic 发现卷以进行 Backup,还原过程均保持不变。

  1. 从 Velero Backup 中还原:

    1
    velero restore create --from-backup BACKUP_NAME OPTIONS...
  2. 还原完成后,查看有关 Pod 卷还原的信息:

    1
    2
    velero restore describe YOUR_RESTORE_NAME
    kubectl -n velero get podvolumerestores -l velero.io/restore-name=YOUR_RESTORE_NAME -o yaml

限制

  • 不支持 hostPath 卷。 支持 本地持久卷
  • 熟悉 restic 的人可能知道它会加密所有数据。 Velero 对它创建的所有 Restic 存储库使用静态通用加密密钥。 这意味着有权访问您的存储桶的任何人都可以解密您的还原 Backup 数据。 确保适当限制对 Restic 桶的访问。
  • 跨 PVC 的 pod 重新安排将维护增量 Backup 链。 但是,对于非 PVC 的 pod 卷(例如emptyDir 卷),当删除 / 重新创建 pod(例如,通过 ReplicaSet / Deployment)时,这些卷的下一次 Backup 将是完整的而不是增量的,因为 pod 卷的是 假定生命周期由其 pod 定义。
  • Restic 在单个线程中扫描每个文件。 这意味着大文件(例如存储数据库的文件)将花费很长时间扫描重复数据删除,即使实际差异很小。
  • 如果您打算使用 Velero Restic 集成来 Backup 100GB 或更多的数据,则可能需要 自定义资源限制 以确保 Backup 成功完成。
  • Velero 的 Restic 集成通过访问运行 Pod 的节点文件系统来 Backup 卷中的数据。 因此,RESTIC 集成只能 Backup 由 Pod 挂载的卷,而不能直接从 PVC Backup。

自定义还原助手容器

Velero 在执行 Restic 还原时使用辅助初始化容器。 默认情况下,此容器的镜像是velero/velero-restic-restore-helper:<VERSION> ,其中VERSION 与主 Velero 镜像的版本 / 标签匹配。 您可以通过在 Velero 命名空间中创建带有备用镜像的 ConfigMap,来定制用于此帮助程序的镜像。

ConfigMap 必须如下所示:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
apiVersion: v1
kind: ConfigMap
metadata:
# any name can be used; Velero uses the labels (below)
# to identify it rather than the name
name: restic-restore-action-config
# must be in the velero namespace
namespace: velero
# the below labels should be used verbatim in your
# ConfigMap.
labels:
# this value-less label identifies the ConfigMap as
# config for a plugin (i.e. the built-in restic restore
# item action plugin)
velero.io/plugin-config: ""
# this label identifies the name and kind of plugin
# that this ConfigMap is for.
velero.io/restic: RestoreItemAction
data:
# The value for "image" can either include a tag or not;
# if the tag is *not* included, the tag from the main Velero
# image will automatically be used.
image: myregistry.io/my-custom-helper-image[:OPTIONAL_TAG]

# "cpuRequest" sets the request.cpu value on the restic init containers during restore.
# If not set, it will default to "100m". A value of "0" is treated as unbounded.
cpuRequest: 200m

# "memRequest" sets the request.memory value on the restic init containers during restore.
# If not set, it will default to "128Mi". A value of "0" is treated as unbounded.
memRequest: 128Mi

# "cpuLimit" sets the request.cpu value on the restic init containers during restore.
# If not set, it will default to "100m". A value of "0" is treated as unbounded.
cpuLimit: 200m

# "memLimit" sets the request.memory value on the restic init containers during restore.
# If not set, it will default to "128Mi". A value of "0" is treated as unbounded.
memLimit: 128Mi

# "secCtxRunAsUser sets the securityContext.runAsUser value on the restic init containers during restore."
secCtxRunAsUser: 1001

# "secCtxRunAsGroup sets the securityContext.runAsGroup value on the restic init containers during restore."
secCtxRunAsGroup: 999

Backup 和还原如何与 Restic 一起工作

Velero 具有三个自定义资源定义和关联的控制器:

  • ResticRepository - 代表 / 管理 Velero Restic 储存库 的生命周期。 当请求第一个名称空间的 Restic Backup 时,Velero 将为每个名称空间创建一个 Restic 存储库。 此自定义资源的控制器执行 restic 存储库生命周期命令– restic initrestic checkrestic prune

    您可以通过运行velero restic repo get 来查看有关 Velero Restic 存储库的信息。

  • PodVolumeBackup - 表示容器中卷的静态 Backup。当找到带注释的 pod 时,主要的 Velero Backup 过程会创建一个或多个 PodVolumeBackup 。群集中的每个节点都为此资源(在 daemonset 中)运行一个控制器,该控制器处理该节点上的 Pod 的PodVolumeBackups 。 控制器执行restic backup 命令以 Backup pod 卷数据。

  • PodVolumeRestore - 表示 pod volume 的恢复。 当遇到具有关联的 Restic Backup 的 Pod 时,主要的 Velero 还原过程会创建其中的一个或多个 PodVolumeRestore。 群集中的每个节点都为此资源运行一个控制器(在与上述相同的 daemonset 中),该控制器处理该节点上的 Pod 的PodVolumeRestores。 控制器执行restic restore 命令以还原 Pod 卷数据。

Backup

  1. 根据配置,主要的 Velero Backup 过程使用选择加入或选择退出方法来检查要 Backup 的每个 Pod,以使用 Restic Backup 要 Backup 的卷。
  2. 找到后,Velero 首先通过以下方式确保 pod 的名称空间存在一个 Restic 存储库:
    • 检查ResticRepository 自定义资源是否已经存在
    • 如果没有,创建一个新的,然后等待ResticRepository 控制器进行初始化 / 检查。
  3. 然后 Velero 为 pod 注释中列出的每个卷创建一个PodVolumeBackup 自定义资源。
  4. 现在,主要的 Velero 进程等待 PodVolumeBackup 资源完成或失败
  5. 同时,每个PodVolumeBackup 由相应节点上的控制器处理:
    • 具有 /var/lib/kubelet/pods 的 hostPath 卷挂载以访问 Pod 卷数据
    • 在上述卷中找到 pod 卷的子目录
    • 运行restic backup
    • 将自定义资源的状态更新为“Completed”或“Failed”
  6. 每个PodVolumeBackup 完成时,主 Velero 进程会将其添加到名为<backup-name>-podvolumebackups.json.gz 的文件中的 Velero Backup 中。 该文件与 Backup tarball 一起上传到对象存储。 它将用于还原,如下一节所述。

还原

  1. Velero 的主要还原过程将检查集群中每个要 Backup 的现有PodVolumeBackup 自定义资源。

  2. 对于找到的每个PodVolumeBackup ,Velero 首先通过以下方法确保该 pod 的命名空间存在一个 Restic 存储库:

    • 检查ResticRepository 自定义资源是否已经存在
    • 如果不存在,则创建一个新仓库,然后等待ResticRepository 控制器初始化 / 检查它(请注意,在这种情况下,实际的仓库应该已经存在于对象存储中,因此 Velero 控制器将只对其进行完整性检查)
  3. Velero 将一个初始化容器添加到了 pod 中,该容器的工作是等待所有还原恢复以完成容器(稍后会详细介绍)

  4. Velero 通过将其提交到 Kubernetes API 来创建具有添加的 init container 的 pod

  5. Velero 为要在被还原的每个 Pod 中创建PodVolumeRestore 自定义资源

  6. 现在,主要的 Velero 进程等待每个PodVolumeRestore 资源完成或失败

  7. 同时,每个PodVolumeRestore 由相应节点上的控制器处理:

    • 具有 /var/lib/kubelet/pods 的 hostPath 卷挂载以访问 Pod 卷数据

    • 等待 pod 运行 init 容器

    • 在上述卷中找到 pod 卷的子目录

    • 运行restic restore

    • 成功后,将文件写入 Pod 卷中的 .velero 子目录中,该文件的名称是此 Pod 卷还原用于的 Velero 还原的 UID。

    • 将自定义资源的状态更新为“Completed”或“Failed”

  8. 添加到 Pod 的 init 容器正在运行一个过程,该过程一直等到它在每个已还原卷中的.velero 下找到一个文件,其名称是正在运行的 Velero 还原的 UID

  9. 找到所有此类文件后,初始化容器的过程将成功终止,并且 pod 将继续运行其他初始化容器 / 主容器。

第三方控制器

监控 Backup 注释

Velero 没有提供一种机制来检测缺少 restic Backup 注释的持久卷声明。

为了解决这个问题,Thomann Bits&Beats 编写了一个控制器:velero-pvc-watcher

使用 - 集群迁移

使用 BackupsRestores

只要您将每个 Velero 实例指向相同的云对象存储位置,Velero 就能帮助您将资源从一个群集移植到另一个群集。 此方案假定您的群集由同一云提供商托管。 请注意,Velero 本身不支持跨云提供程序迁移持久卷快照。 如果要在云平台之间迁移卷数据,请启用 restic,它将在文件系统级别 Backup 卷内容。

  1. (集群 1)假设您尚未使用 Velero schedule 操作对数据进行检查点检查,则需要首先 Backup 整个群集(根据需要替换<BACKUP-NAME>):

    1
    velero backup create <BACKUP-NAME>

    默认 Backup 保留期限以 TTL(有效期)表示,为 30 天(720 小时); 您可以使用 --ttl <DURATION> 标志根据需要进行更改。 有关 Backup 到期的更多信息,请参见 velero 的工作原理

  2. (集群 2)配置 BackupStorageLocationsVolumeSnapshotLocations, 指向 集群 1 使用的位置,使用 velero backup-location createvelero snapshot-location create. 确保配置 BackupStorageLocations 为 read-only 通过在 velero backup-location create 时使用--access-mode=ReadOnly flag

  3. (集群 2)确保已创建 Velero Backup 对象。 Velero 资源与云存储中的 Backup 文件同步。

    1
    velero backup describe <BACKUP-NAME>

    注意 :默认同步间隔为 1 分钟,因此请确保在检查之前等待。 您可以使用 Velero 服务器的--backup-sync-period 标志配置此间隔。

  4. (集群 2)一旦确认现在存在正确的 Backup(<BACKUP-NAME>),就可以使用以下方法还原所有内容:

    1
    velero restore create --from-backup <BACKUP-NAME>

验证 2 个集群

检查第二个群集是否按预期运行:

  1. (集群 2) 运行:

    1
    velero restore get
  2. 然后运行:

    1
    velero restore describe <RESTORE-NAME-FROM-GET-COMMAND>

如果遇到问题,请确保 Velero 在两个群集中的相同 namespace 中运行。

使用 - 资源过滤

按 namespace、类型或标签筛选对象。

当不使用任何筛选选项时,Velero 会将所有对象包括在 Backup 或还原中。

Includes

仅包括特定资源,不包括所有其他资源。

如果同时包含通配符和特定资源,则通配符优先。

–include-namespaces

  • Backup namespace 及其对象。

    1
    velero backup create <backup-name> --include-namespaces <namespace>
  • 恢复两个 namespace 及其对象。

    1
    velero restore create <backup-name> --include-namespaces <namespace1>,<namespace2>

–include-resources

  • Backup 集群中的所有 deployments:

    1
    velero backup create <backup-name> --include-resources deployments
  • 恢复集群中的所有 deployments 和 configmaps。

    1
    velero restore create <backup-name> --include-resources deployments,configmaps
  • 在 namespace 中 Backup deployments。

    1
    velero backup create <backup-name> --include-resources deployments --include-namespaces <namespace>

–include-cluster-resources

此选项可以具有三个可能的值:

  • true: 包括所有群集范围的资源。

  • false: 不包括群集范围的资源。

  • nil("auto" 或不提供):

    • Backup 或还原所有 namespace 时,将包括群集范围的资源。 默认值:true
    • 使用 namespace 过滤时,不包括群集范围的资源。 默认值:false
      • 除非--include-cluster-resources = false,否则如果由自定义操作(例如,PVC-> PV)触发某些相关的群集作用域资源,则可能仍会进行 Backup/ 还原。
  • Backup 整个群集,包括群集范围内的资源。

    1
    velero backup create <backup-name>
  • 仅还原群集中的命名空间资源。

    1
    velero restore create <backup-name> --include-cluster-resources=false
  • Backup namespace 并包括群集范围的资源。

    1
    velero backup create <backup-name> --include-namespaces <namespace> --include-cluster-resources=true 

–selector

包括与 label selector 匹配的资源。

1
velero backup create <backup-name> --selector <key>=<value>

Excludes

从 Backup 中排除特定资源。

通配符排除将被忽略。

–exclude-namespaces

  • Exclude kube-system from the cluster backup.

    1
    velero backup create <backup-name> --exclude-namespaces kube-system
  • 还原期间排除两个 namespace。

    1
    velero restore create <backup-name> --exclude-namespaces <namespace1>,<namespace2>

–exclude-resources

  • 从 Backup 中排除 secrets:

    1
    velero backup create <backup-name> --exclude-resources secrets
  • 排除 secrets 和 rolebindings:

    1
    velero backup create <backup-name> --exclude-resources secrets,rolebindings

velero.io/exclude-from-backup=true

标签为 velero.io/exclude-from-backup=true 的资源不包括在 Backup 中,即使它包含匹配的选择器标签也是如此。

系列文章

📚️参考文档