What's new in 1.5.5 SP2
Cloudera Data Engineering on premises 1.5.5 SP2 delivers a set of new features for Cloudera Data Engineering.
- Support monitoring Cloudera Data Engineering with Cloudera Observability SaaS
- In Cloudera Data Services on premises 1.5.5 SP2 and higher releases, you can monitor Cloudera Data Engineering metrics with Cloudera Observability SaaS. Integrating Cloudera Data Engineering with Cloudera Observability SaaS enables collecting service metrics, viewing resource usage for virtual clusters, optimizing Spark workloads, troubleshooting job performance through provided dashboards, and accessing historical analysis from the Cloudera Data Engineering Job runs page. For more information, see Cloudera Data Engineering with Cloudera Observability and Adding a Cloudera Data Engineering service.
- Support integrating third-party certificate manager with OpenShift Container Platform
- In Cloudera Data Services on premises 1.5.5 SP2 and higher releases, Cloudera provides out-of-the-box support for Venafi TPP as part of the OpenShift Container Platform installation. For more information, see Integrating a third-party certificate manager.
- Support Privacy Settings for Virtual Clusters with OpenShift Container Platform
- In Cloudera Data Services on premises 1.5.5 SP2 and higher releases, Cloudera Data Engineering supports Privacy Settings option for Virtual Clusters as part of the OpenShift Container Platform installations. For more information, see Privacy Settings for a Virtual Cluster.
- Support User Access Management and Artifact access management with OpenShift Container Platform
- In Cloudera Data Services on premises 1.5.5 SP2 and higher releases, Cloudera Data Engineering supports User Access Management and Artifact access management as part of the OpenShift Container Platform installations. For more information, see User Access Management and Artifact access management.
- Endpoint stability is not supported for upgrades with OpenShift Container Platform
- Upgrading Cloudera Data Engineering with OpenShift Container Platform supports endpoint stability if you are upgrading from 1.5.4 SP2 or earlier versions to 1.5.5 or 1.5.5 SP1. Endpoint stability enables you to access the Cloudera Data Engineering service of the new version with the original endpoint. Thus, you can use the existing endpoints without changing configurations at the application level. The Cloudera Data Engineering service endpoint migration process lets you migrate your resources, jobs, job run history, Spark jobs logs, and event logs from your old cluster to the new cluster.
- GPU acceleration support is disabled by default
- In Cloudera Data Services on premises 1.5.5 SP2 and higher releases, GPU acceleration support is disabled by default in Cloudera Data Engineering. If you want to use this feature, contact Cloudera Support. For more information, see Using GPUs in Cloudera Data Engineering (Technical Preview).
- The restore-service command changed to clone-service in Cloudera Data Engineering upgrade with OpenShift Container Platform
- If you are upgrading to Cloudera Data Services on premises 1.5.5 SP2 or higher versions with OpenShift Container Platform, the restore-service command is changed to clone-service. For more information, see Restoring a Cloudera Data Engineering service.
- Support for dedicated Cloudera Embedded Container Service nodes for Cloudera Data Engineering
- You can now dedicate Cloudera Embedded Container Service nodes for Cloudera Data Engineering using Cloudera Manager so that only Cloudera Data Engineering can deploy pods on those nodes. For more information, see Dedicating Cloudera Embedded Container Service nodes for Cloudera Data Engineering.
- Cloudera Data Engineering on premises does not support Spark 2.4.7 and Spark 3.2.3
- With the End of Support (EoS) for Cloudera 7.1.7, Cloudera Data Engineering on premises no longer supports Spark 2.4.7 and Spark 3.2.3. If you are using Spark 2.4.7 or Spark 3.2.3, migrate to Spark 2.4.8 or Spark 3.3.2, respectively, for continued support in future Cloudera Data Services on premises releases. For more information, see Compatibility for Cloudera Data Engineering and Cloudera Runtime components.
- Support for Spark 3.5.x with Cloudera Runtime 7.1.9 SP1
- In Cloudera Data Engineering on premises 1.5.5 SP2 and higher releases, Spark 3.5.x is supported with Cloudera Runtime 7.1.9 SP1 if you are using a security-hardened image. For more information, see Compatibility for Cloudera Data Engineering and Cloudera Runtime components.
- Kubernetes version upgrade to 1.32
- The Kubernetes version that Cloudera Data Engineering uses is upgraded to Kubernetes 1.32. For more information, see Compatibility for Cloudera Data Engineering and Cloudera Runtime components.
