Pre-upgrade - Upgrading Cloudera Data Engineering service

You can seamlessly upgrade a previous Cloudera Data Engineering service version to a new version.

important

Upgrading Cloudera Data Engineering service from version 1.5.4 SP2 or earlier to 1.5.5 or higher does not support endpoint stability. This means the links to your Cloudera Data Engineering Service and Virtual Cluster will change after the upgrade.
After upgrading Cloudera Data Engineering from version 1.5.4 SP2 or earlier to 1.5.5 or higher, the Cloudera Data Engineering Services and Virtual Clusters that were created in the earlier version does not work in the Cloudera Data Engineering 1.5.5 or higher. Cloudera recommends you not to use the old Cloudera Data Engineering Services and Virtual Clusters that were created before upgrade. Instead, create new Services and Virtual Clusters in the Cloudera Data Engineering 1.5.5 or higher version that you upgraded to and use them.
After upgrading Cloudera Data Engineering, the upgraded Virtual Cluster retains the same base OS images that were used in the source Virtual Cluster before upgrade to ensure maximum compatibility particularly for jobs that depend on particular python and scala versions such as Spark jobs. For example, if an old Virtual Cluster is Redhat insecure based, then the new restored Virtual Cluster will also be Redhat insecure based only and if the old Virtual Cluster is security hardened based, then the new restored Virtual Cluster will also be security hardened based only. No automation path is supported from Redhat insecure to security hardened or vice versa.

Once you upgrade to Cloudera Data Engineering 1.5.5 or higher, the endpoints that you were using in the previous version are not supported. The Cloudera Data Engineering service endpoint migration process lets you migrate your resources, jobs, job run history, Spark jobs’ logs, and event logs from your old cluster to the new cluster.