Cloudera Docs

Configuring external storage in ECS for DRS automatic backups

Before you initiate Data Recovery Service (DRS) automatic backups to the external storage in Longhorn, you must complete the prerequisites.

  1. Complete the following prerequisites:
    1. Ensure that the following requirements are met depending on the storage you choose for DRS automatic backups:
      • An S3 compatible storage, such as Ozone, must be available in the base cluster. You must have the required access key and secret to the storage, and the provisioned bucket must have a minimum of 5 TB storage space.
      • An NFS v4 storage must have a minimum of 5 TB of free space.
    2. You must have SSH access to the base cluster node.
    3. You must have SSH access to the ECS master node.
  2. Perform the following steps to change the default volume snapshot class value from snap (this value saves snapshots in the in-cluster storage in Longhorn) to bak (this value saves snapshots in the external storage in Longhorn):
    1. Run the kubectl edit vsclass longhorn kubectl command.
    2. Change the type parameter to bak as shown in the following sample snippet: 
      apiVersion: snapshot.storage.k8s.io/v1
deletionPolicy: Delete
driver: driver.longhorn.io
kind: VolumeSnapshotClass
metadata:
name: longhorn
parameters:
type: bak
  3. Complete the following steps if you are using Ozone S3 storage in Longhorn:
    1. Run the scp root@[***BASE_CLUSTER_HOST***]:/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_cacerts.pem command to obtain the TLS certificate for Ozone.
      DRS uses this certificate to communicate with the S3 gateway service using HTTPS.
    2. Create a secret that Longhorn can use for S3 access. To accomplish this task, you must have the S3 access key, S3 secret, S3 endpoint, and S3 certificate for Ozone storage. You must also enable a virtual host to use the S3 compatible endpoint (Ozone).
      The following sample snippet shows the kubectl command to create a secret:
      kubectl create secret generic ozone-secret 
--from-literal=AWS_ACCESS_KEY_ID=s3g/[email protected]
--from-literal=AWS_SECRET_ACCESS_KEY=9d9e46cc77bb510821f0dbc42c584a8b7482b51dec9d3eb63c
--from-literal=AWS_ENDPOINTS=https://drs1.root.hwx.site:9879/longhorn
--from-literal=VIRTUAL_HOSTED_STYLE=true --from-file=AWS_CERT=cm-auto-global_cacerts.pem
-n longhorn-system

      For more information, see Longhorn documentation.

  4. Run the kubectl edit deploy cdp-release-thunderhead-drsprovider -n cdp-drs command, and set the TAKE_PVC_CLONE environment value to false.
    This step ensures that the backups do not create a persistent volume claim (PVC) clone for external snapshot.

    By default, Longhorn configuration is set to in-cluster storage and this storage requires a PVC copy to perform the DRS restore operation (DRS uses CSI snapshot technology). Therefore, to use the external storage, you must configure the volume snapshot class to bak and then configure the TAKE_PVC_CLONE environment value to false.

  5. Save the backups by configuring the volume for NFS storage or the bucket for Ozone S3.
    • Perform the following steps if you are using Cloudera Data Services on premises 1.5.5 SP1 and earlier and Longhorn 1.8.1 and earlier:
      1. Go to the Longhorn UI > Setting > General page.
      2. For NFS storage, enter the nfs://… URL in the Backup Target field.
      3. For Ozone S3 storage:
        • Enter s3://[***BUCKET***]@[***DUMMY REGION***]/ URL in the Backup Target field. For example, s3://drs1-1@cdp/.
        • Enter [***SECRET THAT YOU GENERATED IN STEP 3B***] in the Backup Target Credential Secret field. For example, ozone-secret.

        The s3://[***BUCKET***]@[***DUMMY REGION***]/ URL is a virtual S3 URL that you can create using the original Ozone S3 URL. The [*** BUCKET ***] value is the hostname. Longhorn appends the AWS_ENDPOINTS to the bucket value. For example, the sample snippet in Step 3 shows the hostname value as drs1-1.drs1.root.hwx.site where, drs1-1 is the bucket name, and drs1.root.hwx.site is the AWS_ENDPOINT. The [*** DUMMY REGION ***] can be dummy value that is not used by Ozone.

    • Perform the following steps if you are using Cloudera Data Services on premises 1.5.5 SP2 and higher and Longhorn 1.8.2 and higher:
      1. Go to the Longhorn UI > Setting > Backup Target page.
      2. Verify whether a configured default backup target is available. If the default backup target is not configured, configure a default target.
      3. Edit the Default target.
      4. Perform the following steps on the Edit Backup Target default modal window:
        • For NFS storage, enter the nfs://… URL in the URL field.
        • For Ozone S3 storage:
          • Enter s3://[***BUCKET***]@[***DUMMY REGION***]/ URL in the URL field. For example, s3://drs1-1@cdp/.
          • Enter [***SECRET THAT YOU GENERATED IN STEP 3B***] in the Credential Secret field. For example, ozone-secret.
  6. Verify whether Longhorn successfully registered the Ozone S3 credential secret by clicking the Backup page. If no errors appear on the page, then the Ozone S3 credential secret has been registered successfully.
    If any error message appears about the secret and the certificate having newlines or space, perform the following steps depending on the versions you are using.
    • For Cloudera Data Services on premises 1.5.5 SP1 and earlier and Longhorn 1.8.1 and earlier, run the kubectl edit lhs backup-target-credential-secret -n longhorn-system command, and set the value of the spec.credentialSecret parameter to the secret you created in Step 3b.
    • For Cloudera Data Services on premises 1.5.5 SP2 and higher and Longhorn 1.8.2 and higher, run the kubectl edit backuptarget default -n longhorn-system command, and set the value of the spec.credentialSecret parameter to the secret you created in Step 3b.
Initiate the DRS automatic backups using the updateAutoBackupPolicy CDP CLI command. Alternatively, you can edit the “automatic-backup” (a Kubernetes cron job) to initiate the DRS automatic backups.