目录
一、实验
1.环境
（1）主机
表1 主机
| 主机 | 架构 | 软件 | 版本 | IP | 备注 |
| hadoop | NameNode （已部署） SecondaryNameNode （已部署） ResourceManager（已部署） |
hadoop |
2.7.7 | 192.168.204.50 | |
| node01 | DataNode（已部署） NodeManager（已部署） |
hadoop |
2.7.7 | 192.168.204.51 | |
| node02 | DataNode（已部署） NodeManager（已部署） |
hadoop |
2.7.7 | 192.168.204.52 | |
| node03 | DataNode（已部署） NodeManager（已部署） |
hadoop |
2.7.7 | 192.168.204.53 | |
| node04 | DataNode（已部署） |
hadoop |
2.7.7 | 192.168.204.54 | 需求迁移主机 |
（2）查看jps
hadoop节点
[root@hadoop ~]# jps
node01节点
node02节点
node03节点
node04节点
2.HDFS 集群节点缩容
（1）查看节点状态 （已使用）
node01: 176KB (约等于0.18M )
node02: 280KB (约等于0.28M )
node03: 260KB (约等于0.26M )
node04: 8KB （约等于0.008M）
[root@hadoop hadoop]# ./bin/hdfs dfsadmin -report
Configured Capacity: 822126559232 (765.67 GB)
Present Capacity: 798788423680 (743.93 GB)
DFS Remaining: 798787682304 (743.93 GB)
DFS Used: 741376 (724 KB)
DFS Used%: 0.00%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (4):
Name: 192.168.204.54:50010 (node04)
Hostname: node04
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 8192 (8 KB)
Non DFS Used: 5658730496 (5.27 GB)
DFS Remaining: 199872901120 (186.15 GB)
DFS Used%: 0.00%
DFS Remaining%: 97.25%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 15:16:33 CST 2024
Name: 192.168.204.53:50010 (node03)
Hostname: node03
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 266240 (260 KB)
Non DFS Used: 5620936704 (5.23 GB)
DFS Remaining: 199910436864 (186.18 GB)
DFS Used%: 0.00%
DFS Remaining%: 97.27%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 15:16:33 CST 2024
Name: 192.168.204.51:50010 (node01)
Hostname: node01
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 180224 (176 KB)
Non DFS Used: 6029176832 (5.62 GB)
DFS Remaining: 199502282752 (185.80 GB)
DFS Used%: 0.00%
DFS Remaining%: 97.07%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 15:16:34 CST 2024
Name: 192.168.204.52:50010 (node02)
Hostname: node02
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 286720 (280 KB)
Non DFS Used: 6029291520 (5.62 GB)
DFS Remaining: 199502061568 (185.80 GB)
DFS Used%: 0.00%
DFS Remaining%: 97.07%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 15:16:34 CST 2024
（2）查看
[root@hadoop ~]# cd /usr/local/hadoop/
[root@hadoop hadoop]# ./bin/hadoop fs -ls /
（3）上传第一个测试文件
[root@hadoop ~]# /usr/local/hadoop/bin/hadoop fs -put /root/jenkins.tar.gz /jenkins.tar.gz
观察上传变化
[root@hadoop hadoop]# ./bin/hadoop fs -ls /
（4）上传第二个测试文件
[root@hadoop ~]# /usr/local/hadoop/bin/hadoop fs -put /root/pinpoint-image.tar.gz /pinpoint.tar.gz
观察上传变化
[root@hadoop hadoop]# ./bin/hadoop fs -ls /
（5）web查看
http://192.168.204.50:50070/
（6）清空本地文件
[root@hadoop ~]# rm -rf jenkins.tar.gz pinpoint-image.tar.gz
（7）再次查看节点状态 （已使用）
node01: 2.41GB
node02: 2.27GB
node03: 2.15GB
node04: 2.26GB
[root@hadoop hadoop]# ./bin/hdfs dfsadmin -report
Configured Capacity: 822126559232 (765.67 GB)
Present Capacity: 798784053248 (743.93 GB)
DFS Remaining: 789019389952 (734.83 GB)
DFS Used: 9764663296 (9.09 GB)
DFS Used%: 1.22%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (4):
Name: 192.168.204.54:50010 (node04)
Hostname: node04
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2424492032 (2.26 GB)
Non DFS Used: 5662638080 (5.27 GB)
DFS Remaining: 197444509696 (183.88 GB)
DFS Used%: 1.18%
DFS Remaining%: 96.07%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 15:57:57 CST 2024
Name: 192.168.204.53:50010 (node03)
Hostname: node03
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2312085504 (2.15 GB)
Non DFS Used: 5620936704 (5.23 GB)
DFS Remaining: 197598617600 (184.03 GB)
DFS Used%: 1.12%
DFS Remaining%: 96.14%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 15:57:58 CST 2024
Name: 192.168.204.51:50010 (node01)
Hostname: node01
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2592927744 (2.41 GB)
Non DFS Used: 6029148160 (5.62 GB)
DFS Remaining: 196909563904 (183.39 GB)
DFS Used%: 1.26%
DFS Remaining%: 95.80%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 15:57:57 CST 2024
Name: 192.168.204.52:50010 (node02)
Hostname: node02
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2435158016 (2.27 GB)
Non DFS Used: 6029783040 (5.62 GB)
DFS Remaining: 197066698752 (183.53 GB)
DFS Used%: 1.18%
DFS Remaining%: 95.88%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 15:57:59 CST 2024
（8）计算总存储使用量
9.09 G （jenkins 1.24 GB,pinpoint 3.27 GB ,存储副本量为2）
(9)修改配置文件（hadoop节点）
#添加代码
[root@hadoop hadoop]# vim hdfs-site.xml
修改前：
修改后：
……
<property>
<name>dfs.hosts.exclude</name>
<value>/usr/local/hadoop/etc/hadoop/exclude</value>
</property>
……
（10）配置移除主机
[root@hadoop hadoop]# vim /usr/local/hadoop/etc/hadoop/exclude
node04
（11）数据刷新
[root@hadoop hadoop]# ./bin/hdfs dfsadmin -refreshNodes
（12）查看节点状态
node04节点状态变为Decommissioned in Program
[root@hadoop hadoop]# ./bin/hdfs dfsadmin -report
Configured Capacity: 619019411456 (576.51 GB)
Present Capacity: 600866701319 (559.60 GB)
DFS Remaining: 590966771712 (550.38 GB)
DFS Used: 9899929607 (9.22 GB)
DFS Used%: 1.65%
Under replicated blocks: 2
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (4):
Name: 192.168.204.54:50010 (node04)
Hostname: node04
Decommission Status : Decommission in progress
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2424492032 (2.26 GB)
Non DFS Used: 5663162368 (5.27 GB)
DFS Remaining: 197443985408 (183.88 GB)
DFS Used%: 1.18%
DFS Remaining%: 96.06%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 16:34:33 CST 2024
Name: 192.168.204.53:50010 (node03)
Hostname: node03
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2312085504 (2.15 GB)
Non DFS Used: 5756112896 (5.36 GB)
DFS Remaining: 197463441408 (183.90 GB)
DFS Used%: 1.12%
DFS Remaining%: 96.07%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 2
Last contact: Thu Mar 14 16:34:31 CST 2024
Name: 192.168.204.51:50010 (node01)
Hostname: node01
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2592927744 (2.41 GB)
Non DFS Used: 6232051712 (5.80 GB)
DFS Remaining: 196706660352 (183.20 GB)
DFS Used%: 1.26%
DFS Remaining%: 95.71%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 3
Last contact: Thu Mar 14 16:34:33 CST 2024
Name: 192.168.204.52:50010 (node02)
Hostname: node02
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2570424327 (2.39 GB)
Non DFS Used: 6164545529 (5.74 GB)
DFS Remaining: 196796669952 (183.28 GB)
DFS Used%: 1.25%
DFS Remaining%: 95.75%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 2
Last contact: Thu Mar 14 16:34:32 CST 2024
Decommissioning datanodes (1):
Name: 192.168.204.54:50010 (node04)
Hostname: node04
Decommission Status : Decommission in progress
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2424492032 (2.26 GB)
Non DFS Used: 5663162368 (5.27 GB)
DFS Remaining: 197443985408 (183.88 GB)
DFS Used%: 1.18%
DFS Remaining%: 96.06%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 16:34:33 CST 2024
(13) 再次查看节点
node04节点状态变为Decommissioned
[root@hadoop hadoop]# ./bin/hdfs dfsadmin -report
Configured Capacity: 619019411456 (576.51 GB)
Present Capacity: 601338761216 (560.04 GB)
DFS Remaining: 589149618176 (548.69 GB)
DFS Used: 12189143040 (11.35 GB)
DFS Used%: 2.03%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (4):
Name: 192.168.204.54:50010 (node04)
Hostname: node04
Decommission Status : Decommissioned
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2424492032 (2.26 GB)
Non DFS Used: 5662666752 (5.27 GB)
DFS Remaining: 197444481024 (183.88 GB)
DFS Used%: 1.18%
DFS Remaining%: 96.07%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 16:42:33 CST 2024
Name: 192.168.204.53:50010 (node03)
Hostname: node03
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 3123707904 (2.91 GB)
Non DFS Used: 5620969472 (5.23 GB)
DFS Remaining: 196786962432 (183.27 GB)
DFS Used%: 1.52%
DFS Remaining%: 95.75%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 16:42:31 CST 2024
Name: 192.168.204.51:50010 (node01)
Hostname: node01
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2998738944 (2.79 GB)
Non DFS Used: 6029717504 (5.62 GB)
DFS Remaining: 196503183360 (183.01 GB)
DFS Used%: 1.46%
DFS Remaining%: 95.61%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 16:42:33 CST 2024
Name: 192.168.204.52:50010 (node02)
Hostname: node02
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 3642204160 (3.39 GB)
Non DFS Used: 6029963264 (5.62 GB)
DFS Remaining: 195859472384 (182.41 GB)
DFS Used%: 1.77%
DFS Remaining%: 95.29%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 16:42:32 CST 2024
(14) 重启
[root@hadoop hadoop]# ssh node04
[root@node04 ~]# reboot
（15）查看节点（已使用）
node01: 2.79GB
node02: 3.39GB
node03: 2.91GB
[root@hadoop hadoop]# ./bin/hdfs dfsadmin -report
Configured Capacity: 619019411456 (576.51 GB)
Present Capacity: 601338118144 (560.04 GB)
DFS Remaining: 589148975104 (548.69 GB)
DFS Used: 12189143040 (11.35 GB)
DFS Used%: 2.03%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (3):
Name: 192.168.204.53:50010 (node03)
Hostname: node03
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 3123707904 (2.91 GB)
Non DFS Used: 5621551104 (5.24 GB)
DFS Remaining: 196786380800 (183.27 GB)
DFS Used%: 1.52%
DFS Remaining%: 95.75%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 17:00:14 CST 2024
Name: 192.168.204.51:50010 (node01)
Hostname: node01
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2998738944 (2.79 GB)
Non DFS Used: 6029742080 (5.62 GB)
DFS Remaining: 196503158784 (183.01 GB)
DFS Used%: 1.46%
DFS Remaining%: 95.61%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 17:00:12 CST 2024
Name: 192.168.204.52:50010 (node02)
Hostname: node02
Decommission Status : Normal
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 3642204160 (3.39 GB)
Non DFS Used: 6030000128 (5.62 GB)
DFS Remaining: 195859435520 (182.41 GB)
DFS Used%: 1.77%
DFS Remaining%: 95.29%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 17:00:12 CST 2024
Dead datanodes (1):
Name: 192.168.204.54:50010 (node04)
Hostname: node04
Decommission Status : Decommissioned
Configured Capacity: 205531639808 (191.42 GB)
DFS Used: 2424492032 (2.26 GB)
Non DFS Used: 5663240192 (5.27 GB)
DFS Remaining: 197443907584 (183.88 GB)
DFS Used%: 1.18%
DFS Remaining%: 96.06%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Thu Mar 14 16:47:27 CST 2024
计算总使用量
（16）回顾使用量变化
表2 数据迁移
| 主机 | 初始 | 上传 | 迁移前 | 迁移后 | IP | 备注 |
| node01 | 76KB | jenkins 1.24 GB pinpoint 3.27 GB |
2.41GB | 2.79GB | 192.168.204.51 | replication 2 (副本备份量) |
| node02 | 280KB | 2.27GB | 3.39GB | 192.168.204.52 | ||
| node03 | 260KB | 2.15GB | 2.91GB | 192.168.204.53 | ||
| node04 | 8KB | 2.26GB | / | 192.168.204.54 | 迁移主机 | |
| 总大小 | 9.09GB | 9.09GB |
二、问题
1.数据迁移有哪些状态
（1）状态
1）正常状态
Normal
2)数据正在迁移
Decommissioned in Progress
3)数据迁移完成 （仅变成当前状态才能down机下线）
Decommissioned
2.数据迁移失败
（1）失败
迁移node04状态依然为Normal
（2）原因分析
配置错误。
（3）解决方法
修改配置。
修改前：
修改后：
成功：
