Pre-upgrade - Upgrading Cloudera Data Engineering service

You can seamlessly upgrade a previous Cloudera Data Engineering service version to a new version.

important

Upgrading Cloudera Data Engineering service supports endpoint stability only when you are upgrading from 1.5.4 SP2 or earlier versions to 1.5.5 or 1.5.5 SP1. Endpoint stability enables you to access the Cloudera Data Engineering service of the new version with the original endpoint. Thus, you can use the existing endpoints without changing configurations at the application level. The Cloudera Data Engineering service endpoint migration process lets you migrate your resources, jobs, job run history, Spark jobs logs, and event logs from your old cluster to the new cluster.
If you are upgrading Cloudera Data Engineering to 1.5.5 SP2 or higher versions, endpoint stability is not supported. This means that the links to your Cloudera Data Engineering Service and Virtual Cluster will change after the upgrade.
After upgrading Cloudera Data Engineering from 1.5.4 SP2 or earlier versions to 1.5.5 or higher versions, the Cloudera Data Engineering Services and Virtual Clusters that were created in the earlier versions does not work in Cloudera Data Engineering 1.5.5 or higher versions. Cloudera does not recommend using the old Cloudera Data Engineering Services and Virtual Clusters that were created before the upgrade. Instead, create new Services and Virtual Clusters in Cloudera Data Engineering 1.5.5 or higher version that you upgraded to and use them.
After upgrading Cloudera Data Engineering, the upgraded Virtual Cluster retains the same base OS images used in the source Virtual Cluster. This ensures maximum compatibility, particularly for jobs that depend on specific Python and Scala versions, such as Spark jobs. For example, if the source Virtual Cluster uses a standard Red Hat image, the upgraded Virtual Cluster retains that image type. The same applies to security-hardened images. No automation path is supported from a standard Red Hat image to a security-hardened image or the other way around.

Once you upgrade to Cloudera Data Engineering version 1.5.5 SP2 or higher, the endpoints that you were using in the previous version are not supported.