Known Issues in Spark

Known issues and technical limitations for Spark are addressed in Cloudera Runtime 7.1.9 SP2.

Known issues identified in Cloudera Runtime 7.1.9 SP2

CDPD-108994: Oozie Spark 3 jobs fail on 7.1.9 SP2 with Commons Lang errors when the CDS parcel predates 7.1.9 SP2: When Spark 3 is launched from an Oozie workflow on Cloudera Runtime 7.1.9 SP2 (including any CHF) with a CDS parcel released before CDS 3.3 for 7.1.9 SP2 or CDS 3.5 for 7.1.9 SP2, jobs can fail with NoSuchMethodError on org.apache.commons.lang.StringUtils.join or ClassNotFoundException for org.apache.commons.lang.StringEscapeUtils. Direct spark3-submit jobs are not affected by this classpath mismatch.; Activate CDS 3.3 for 7.1.9 SP2 or CDS 3.5 for 7.1.9 SP2. For version pairing details, see Commons Lang compatibility for Oozie Spark 3.