How to finalize an HDFS upgrade
If the HDFS upgrade worked out fine and your subsequent testing was successful, you may want to finalize the HDFS upgrade.
Warning: Finalizing an HDFS upgrade is a point of no return. You cannot perform a rollback once the cluster is finalized!
You can check with the following command whether the cluster needs to be finalized:
1. $ hadoop dfsadmin -upgradeProgress status
Run the actual finalize command to make the HDFS upgrade permanent:
| 1 | $ hadoop dfsadmin -finalizeUpgrade |
|---|---|
The -finalizeUpgrade command removes the previous version of the NameNode’s and DataNodes’ storage directories.
In the NameNode Web UI you should see a message "Upgrades: There are no upgrades in progress".
The NameNode logs should contain entries similar to the following:
| 1 | # NameNode log file |
|---|---|
Also, running hadoop dfsadmin will now report that there are no pending upgrades.
| 1 | $ hadoop dfsadmin -upgradeProgress status |
|---|---|
Note: The finalize upgrade procedure can run in the background without disrupting the performance of the Hadoop cluster.
It is worth mentioning that deleting files that existed before the upgrade does not free up real disk space on the DataNodes until the HDFS cluster is finalized.
The official HDFS User Guide in the Hadoop documentation provides the full instructions for HDFS Upgrade and Rollback.