Review the list of Spark issues that are resolved in Cloudera Runtime
7.3.2, its service packs and cumulative hotfixes.
Cloudera Runtime 7.3.2.10000 SP1
- CDPD-101584: Spark History Server fails on FIPS 140-3 clusters
when default
cacerts is BCFKS
- 7.3.2.10000 SP1
- Fixed failures when the JVM default trust store is BCFKS
and no trust store password is set. Spark 3 adds support for
spark.ssl.defaultTrustStorePassword, which sets
javax.net.ssl.trustStorePassword on driver and executor JVMs and when
Spark History Server starts. Cloudera Manager may set this property on managed
clusters.
- CDPD-87793: Knox redirection fails for running Spark 3
applications when the Spark UI is disabled
- 7.3.2.10000 SP1
- Fixed Knox redirection to Spark History Server tracking
URLs when the Spark UI is disabled and Encrypt all ports defaults apply.
Cloudera Runtime 7.3.2.100 CHF1
There are no fixed issues in this release.
Cloudera Runtime 7.3.2
Cloudera Runtime 7.3.2 resolves Spark issues and incorporates fixes from
the service packs and cumulative hotfixes from 7.3.1.100 through 7.3.1.706. For a
comprehensive record of all fixes in Cloudera Runtime 7.3.1.x, see Spark Fixed Issues.
- CDPD-90482: Jersey Library update
- 7.3.2.0
- Updated the Jersey suite of libraries to versions 2.0 and above to ensure JDK 17 compatibility.
- CDPD-82979: Added Cloudera-specific configuration for
MapPartitionsRDD default determinacy
- 7.3.2.0
- To avoid failures during error handling, a Cloudera-specific configuration was added to change the default determinacy level for any
MapPartitionsRDD.
- CDPD-80239: Fixed handling of non-deterministic SQL expressions
- 7.3.2.0
- Fixed a bug related to intermittent data corruption and record loss during task failures. Spark now honors the
deterministic=false attribute for non-deterministic data types.
- CDPD-67187: Fixed missing JVM class imports for Spark
- 7.3.2.0
- JVM classes are now imported during session initialization.
- OPSAPS-74346, OPSAPS-74346, OPSAPS-74316: Enhancing Spark security
- 7.3.2.0
- Changed the generation of the
spark.yarn.historyServer.address value to use the HTTPS address when SSL/TLS is enabled. The spark3.network.crypto.enabled new configuration property is now available to enable AES-based encryption.
- OPSAPS-72254: UCL | FIPS Failed to upload Spark example jar to HDFS in cluster mode
- 7.3.2.0
Fixed an issue with deploying the Spark 3 Client Advanced Configuration Snippet (Safety Valve) for spark3-conf/spark-env.sh.
For more information, see New Cloudera Manager configuration parameter spark_pyspark_executable_path has been added to the Livy for Spark 3 in Behavioral Changes In Cloudera Manager 7.13.2.
- OPSAPS-76539: Cloudera Manager UI allowed adding multiple Spark 3 service instances within a single cluster
- 7.3.2.0
- Prevented multiple Spark 3 service instances within a single cluster by implementing a
maxInstances flag in Cloudera Manager.
- OPSAPS-74281: Disabled the live Spark UI when the
ENCRYPT_ALL_PORTS feature flag is enabled
- 7.3.2.0
Enhanced the security of Spark by implementing new default configuration settings:
spark.ui.enabled=false: preventing initiation of an HTTP service that can be accessed from external hosts.spark.io.encryption.keySizeBits=256: the default Spark keySizeBits has been increased from 128 to 256
