Cloudera Upgrade Guide Companion

Use this guide to follow the main steps for a successful in-place cluster upgrade.

Before Upgrade

This upgrade guide companion will help you in achieving the key milestones for successfully completing an in-place upgrade of your cluster.

Use the My Environment section to define your cluster upgrade configuration. Use the tabs below to access key checkpoints for completing your upgrade and as a jumping off point for each individual milestone.

Review the complete upgrade documentation | Review supported upgrade paths | Review supported Cloudera Manager information

Upgrade Path

Select your upgrade path to filter content

Loading Filters...
Loading Filters... 7.11.3 7.7.3 7.7.1 7.6.7 7.6.1 7.4.4 7.3.1 7.2.4 7.1.4 7.1.3 7.1.2 7.1.1 7.0.3 6.3.4 6.3.3 6.3.1 6.3.0 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 7.13.1 7.11.3 7.7.3 7.7.1 7.6.7 7.6.1 7.4.4 7.3.1 7.2.4 7.1.4 7.1.3 7.1.2 7.1.1 7.0.3 6.3.4 6.3.3 6.3.1 6.3.0 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 6.3.4 6.3.3 6.3.2 6.3.0 6.2.1 6.2.0 6.1.1 6.1.0 6.0.1 6.0.0 5.16 5.15 5.14 5.13 7.1.9.1000 7.1.9 7.1.8 7.1.7.3000 7.1.7.2000 7.1.7.1000 7.1.7 7.1.6 7.1.5 7.1.4 7.1.3 7.1.2 7.1.1 7.0.3
Loading Filters...
Loading Filters... 7.1.9.1000 7.1.9 7.1.8 7.1.7.3000 7.1.7.2000 7.1.7.1000 7.1.7 7.1.6 7.1.5 7.1.4 7.1.3 7.1.2 7.1.1 7.0.3 7.3.1 7.1.9.1000 7.1.9 7.1.8 7.1.7.3000 7.1.7.2000 7.1.7.1000 7.1.7 7.1.6 7.1.5 7.1.4 7.1.3 7.1.2 7.1.1
Loading Filters... 7.1.7.3000 7.1.7.2000 7.1.7.1000 7.1.9.1000 7.1.7.3000 7.1.7.2000

Upgrading HDP to Cloudera on premises

To get started with upgrading from HDP to Cloudera on premises, first learn about the overall upgrade process and the impact of an upgrade on your deployment.

HDP 2 to Cloudera on premises

Preparing your upgrade Cloudera on premises Upgrade Readiness

HDP 3 to Cloudera on premises

Preparing your upgrade Cloudera on premises Upgrade Readiness

Upgrading CDH to Cloudera on premises

To get started with upgrading from CDH to Cloudera on premises, first learn about the overall upgrade process and the impact of an upgrade on your deployment.

CDH 5 to Cloudera on premises

When upgrading from CDH 5.x, you should review:

Preparing for your upgrade Assessing the Impact of an Upgrade About using this online Upgrade Guide

CDH 6 to Cloudera on premises

When upgrading from CDH 6.x, you should review:

Preparing for your upgrade Assessing the Impact of an Upgrade About using this online Upgrade Guide

Upgrading Cloudera on premises to a higher version

To get started with upgrading to a newer version of Cloudera on premises, first learn about the overall upgrade process and the impact of an upgrade on your deployment.

Upgrading to Cloudera on premises 7.3.1

You can upgrade to Cloudera on premises 7.3.1 from the following Cloudera on premises versions:

Upgrading to Cloudera on premises 7.1.9 SP1

You can upgrade to Cloudera on premises 7.1.9 SP1 from the following Cloudera on premises versions:

Upgrading to Cloudera on premises 7.1.9

You can upgrade to Cloudera on premises 7.1.9 from the following Cloudera on premises versions:

Upgrading to Cloudera on premises 7.1.7 SP3

You can upgrade to Cloudera on premises 7.1.7 SP3 from the following Cloudera on premises versions:

  • Cloudera on premises 7.1.7 SP2
  • Cloudera on premises 7.1.7 SP1
  • Cloudera on premises 7.1.7
  • Cloudera on premises 7.1.6

For more information on the supported upgrade paths, see Supported in-place upgrade paths.

You must also make sure you upgrade to the supported version of Cloudera Manager before you perform the upgrade process. For more information, see Upgrading Cloudera Manager.

To get started with upgrading to a newer version of Cloudera on premises, first learn about the overall upgrade process and the impact of an upgrade on your deployment. See Preparing for your upgrade

Upgrading Cloudera on premises to a Service Pack

To get started with upgrading to a Service Pack, first review the upgrade requirements and supported versions for your target Service Pack release.

Upgrades to Cloudera on premises 7.1.9 Service Pack 1 (SP1)

Note the following requirements for upgrades to Cloudera on premises 7.1.9 SP1:

  • You can upgrade to Cloudera on premises 7.1.9 SP1 (7.1.9.1000) from Cloudera on premises 7.1.9 or Cloudera on premises 7.1.8 latest cumulative hotfix or Cloudera on premises 7.1.7 SP3 (7.1.7.3000).
  • You must first upgrade to Cloudera Manager 7.11.3 CHF7 or higher, which is the supported version for the Cloudera on premises 7.1.9 SP1.
  • You can use Cloudera Manager 7.11.3 CHF7 or higher to manage deployments where Cloudera Manager is managing a Cloudera on premises Data Services cluster.

Upgrades to Cloudera on premises 7.1.7 Service Pack 3 (SP3)

Note the following requirements for upgrades to Cloudera on premises 7.1.7 SP3:

  • You can upgrade to Cloudera on premises 7.1.7 SP3 (7.1.7.3000) from Cloudera on premises 7.1.7 SP2 (7.1.7.2000) or Cloudera on premises 7.1.7 SP1 (7.1.7.1000) or Cloudera on premises 7.1.7 (log4j2) or Cloudera on premises 7.1.6.
  • You must first upgrade to Cloudera Manager 7.11.3 CHF4 or higher, which is the supported version for the Cloudera on premises 7.1.7 SP3.
  • You can use Cloudera Manager 7.11.3 CHF4 or higher to manage deployments where Cloudera Manager is managing a Cloudera on premises Data Services cluster.

Upgrades to Cloudera on premises 7.1.7 Service Pack 2 (SP2)

Note the following requirements for upgrades to Cloudera on premises 7.1.7 SP2:

  • You can upgrade to Cloudera Runtime 7.1.7 Service Pack 2 (7.1.7.2000) from Cloudera Runtime 7.1.7 Service Pack 1 (7.1.7.1000) or Cloudera Runtime 7.1.7 (log4j2) or Cloudera Runtime 7.1.6.
  • You must first upgrade to Cloudera Manager 7.6.7, which is the supported version for Service Pack 2.
  • You cannot use Cloudera Manager 7.6.7 to manage deployments where Cloudera Manager is managing a Cloudera on premises Data Services cluster.

HDP to Cloudera on premises Pre-upgrade Tasks

Gather information on your deployment and verify cluster environment readiness. You can prepare a sample data for ingestion and validate the Apache Hive constructs before you upgrade.

HDP 2 to Cloudera on premises

Cluster Environment Readiness Sample Data Ingestion Hive Validations Ambari and HDP Checklists

HDP 3 to Cloudera on premises

Cluster Environment Readiness Sample Data Ingestion Merge Independent Hive and Spark Catalogs Ambari and HDP Checklists

CDH to Cloudera on premises Pre-upgrade Tasks

The following procedures must be completed before performing a cluster upgrade to Cloudera on premises (Cloudera Runtime 7.1.1 or higher). Only complete the procedures for services running in your source cluster.

CDH 5 to Cloudera on premises

Transitioning from MapReduce 1 to MapReduce 2 Transitioning Cloudera Search configuration before upgrading to Cloudera Runtime Transitioning from Sentry Policy Files to the Sentry Service Transitioning the Sentry service to Apache Ranger Transitioning Navigator content to Atlas Migrating Hive 1-2 to Hive 3 Checking Apache HBase CDH cluster upgrade requirements for Replication Manager Installing dependencies for Hue before upgrading to Cloudera on premises

JDK and Operating System Upgrades (CDH 5)

In addition to the pre-upgrade steps for Cloudera Runtime services, you may also need to upgrade your JDK or operating system. See the Cloudera support Matrix for more information.

Upgrade the JDK Upgrade the Operating System

You must install Python 3.8 (or 3.9 for RHEL 9.1) on all hosts before upgrading to Cloudera Manager 7.11.3. See Installing Python 3.

After you complete these steps, continue with your upgrade by Upgrading Cloudera Manager.

CDH 6 to Cloudera on premises

The following procedures must be completed before performing a cluster upgrade to Cloudera on premises (Cloudera Runtime 7.1.1 or higher). Only complete the procedures for services running in your source cluster:

Transitioning from Sentry Policy Files to the Sentry Service Transitioning the Sentry service to Apache Ranger Transitioning Navigator content to Atlas Migrating Hive 1-2 to Hive 3 Checking Apache HBase CDH cluster upgrade requirements for Replication Manager Installing dependencies for Hue before upgrading to Cloudera on premises

JDK and Operating System Upgrades (CDH 6)

In addition to the pre-upgrade steps for Cloudera Runtime services, you may also need to upgrade your JDK or operating system. See the Cloudera support Matrix for more information.

Upgrade the JDK Upgrade the Operating System

You must install Python 3.8 (or 3.9 for RHEL 9.1) on all hosts before upgrading to Cloudera Manager 7.11.3. See Installing Python 3.

After you complete these steps, continue with your upgrade by Upgrading Cloudera Manager.

Cloudera on premises Pre-upgrade Tasks

You may need to upgrade your JDK or operating system. See the Cloudera support Matrix for more information on supported operating systems and JDKs.

Upgrade the JDK Upgrade the Operating System

Python Requirements

  • You must install Python 3.8 or 3.9 for RHEL 8.8 or RHEL 8.10 on all hosts before upgrading to Cloudera Manager 7.13.1. See Installing Python 3.
  • You must install Python 3.9 for RHEL 9.2 on all hosts before upgrading to Cloudera Manager 7.13.1. See Installing Python 3.
  • You must install Python 3.10 for SLES 15 SP4 and SLES 15 SP5 on all hosts before upgrading to Cloudera Manager 7.13.1. See Installing Python 3.
  • You must install Python 3.8 (or 3.9 for RHEL 9.1) on all hosts before upgrading to Cloudera Manager 7.11.3. See Installing Python 3.
  • You must install Python 3.10 for SLES 15 SP4 and SLES 15 SP5 on all hosts before upgrading to Cloudera Manager 7.11.3 CHF7. See Installing Python 3.

After you complete these steps, continue with your upgrade by Upgrading Cloudera Manager.

Service Pack Pre-upgrade Tasks

You may need to upgrade your JDK or operating system. See the Cloudera support Matrix for more information on supported operating systems and JDKs.

Upgrade the JDK Upgrade the Operating System

You must install Python 3.8 (or 3.9 for RHEL 9.1) on all hosts before upgrading to Cloudera Manager 7.11.3. See Installing Python 3.

After you complete these steps, continue with your upgrade by Upgrading Cloudera Manager.

HDP to Cloudera on premises Cluster Upgrade

Plan how and when to begin your upgrade. Check readiness and upgrade Ambari and HDP. Transition from HDP to Cloudera Manager.

HDP 2 to Cloudera on premises

Upgrade Ambari Upgrade HDP Pre-Transition steps Cloudera Manager Installation Installing JDBC driver Transitioning HDP 2.6.5 cluster to Cloudera on premises cluster using the AM2CM tool

HDP 3 to Cloudera on premises

Upgrade Ambari Upgrade HDP Pre-Transition steps Cloudera Manager Installation Installing JDBC driver Transitioning HDP 3.1.5 cluster to Cloudera on premises cluster using the AM2CM tool

CDH to Cloudera on premises Cluster Upgrade

Upgrading a cluster requires some preparatory steps such as backing up your cluster and Cloudera Manager, establishing access to the software, and other tasks.

CDH 5 to Cloudera on premises

Before upgrading your cluster(s) you must first upgrade Cloudera Manager to a version that supports your planned cluster upgrades. See Upgrade Cloudera Manager.

For detailed procedures, see Upgrading a Cluster.

CDH 6 to Cloudera on premises

Before upgrading your cluster(s) you must first upgrade Cloudera Manager to a version that supports your planned cluster upgrades. See Upgrade Cloudera Manager.

For detailed procedures, see Upgrading a Cluster.

Cloudera on premises Cluster Upgrade

Upgrading a cluster requires some preparatory steps such as backing up your cluster and Cloudera Manager, establishing access to the software, and other tasks.

Before upgrading your cluster(s) you must first upgrade Cloudera Manager to a version that supports your planned cluster upgrades. See Upgrade Cloudera Manager.

For detailed procedures, see Upgrading a Cluster.

Service Pack Cluster Upgrade

Upgrading a cluster requires some preparatory steps such as backing up your cluster and Cloudera Manager, establishing access to the software, and other tasks.

Before upgrading your cluster(s) you must first upgrade Cloudera Manager to a version that supports your planned cluster upgrades. See Upgrade Cloudera Manager.

For detailed procedures, see Upgrading a Cluster.

HDP Post-Upgrade Tasks

If you have LDAP configured on your cluster, configure external authentication with Cloudera Manager. You can also install additional services like DAS and Hue. Test your applications and services on your cluster.

HDP 2 to Cloudera on premises

Post Transition Steps Configuring External Authentication for Cloudera Manager Installing DAS on Cloudera on premises Adding Hue service with Cloudera Manager

HDP 3 to Cloudera on premises

Post Transition Steps Configuring External Authentication for Cloudera Manager Installing DAS on Cloudera on premises Adding Hue service with Cloudera Manager

CDH Post-Upgrade Tasks

The following procedures must be completed after performing a cluster upgrade to Cloudera on premises (Cloudera Runtime 7.1.1 or higher). Only complete the procedures for services running in your source cluster.

CDH 5 to Cloudera on premises

Update permissions for Replication Manager service Apache Hive Expedited Migration Tasks Apache Hive Changes in Cloudera on premises Apache Hive Post-Upgrade Tasks Hue Post-Upgrade Tasks Apache Impala changes in Cloudera on premises Sentry to Ranger Permissions Migrating ACLs from Key Trustee KMS to Ranger KMS Cloudera Search post-upgrade tasks Apache Hadoop YARN default value changes Upgrade Notes for Apache Kudu 1.12 / Cloudera on premises 7.1 Apache HBase post-upgrade tasks Configure SMM to monitor SRM replications Configure SMM's service dependency on Schema Registry Apache Sqoop Changes

CDH 6 to Cloudera on premises

Update permissions for Replication Manager service Apache Hive Expedited Migration Tasks Apache Hive Changes in Cloudera on premises Apache Hive Post-Upgrade Tasks Hue Post-Upgrade Tasks Apache Impala changes in Cloudera on premises Sentry to Ranger Permissions Apache Hadoop YARN default value changes Upgrade Notes for Apache Kudu 1.12 / Cloudera on premises 7.1 Apache HBase post-upgrade tasks Configure SMM to monitor SRM replications Configure SMM's service dependency on Schema Registry Apache Sqoop Changes

Cloudera on premises Post-Upgrade Tasks

Complete Post-Upgrade steps for upgrades to Cloudera on premises

Service Pack Post-Upgrade Tasks

Complete Post-Upgrade steps for upgrades to Cloudera on premises

HDP Troubleshooting

You can review a list of possible causes and solutions to debug and resolve issues that you might face.

Post-upgrade HDP 2 - Troubleshooting DAS Installation Post-upgrade HDP 3 - Troubleshooting DAS Installation

CDH Troubleshooting

If you need to review common things to check that you may have missed during your upgrade, review troubleshooting suggestions.

CDH 5 to Cloudera on premises - Troubleshooting CDH 6 to Cloudera on premises - Troubleshooting

Cloudera on premises Troubleshooting

You can review a list of possible causes and solutions to debug and resolve issues that you might face.

Troubleshooting

Service Pack Troubleshooting

You can review a list of possible causes and solutions to debug and resolve issues that you might face.

Troubleshooting