博客
关于我
强烈建议你试试无所不能的chatGPT,快点击我
二:HDFS 命令指南
阅读量:5154 次
发布时间:2019-06-13

本文共 3295 字,大约阅读时间需要 10 分钟。

命令具体选项请参考:
 

1.HDFS命令通过bin/hdfs执行,语法如下:

 
hdfs [--config confdir] [COMMAND] [GENERIC_OPTIONS] [COMMAND_OPTIONS]
        配置文件          用户命令   基本选项           命令选项
COMMAND_OPTION Description
--config confdir Overwrites the default Configuration directory. Default is $HADOOP_HOME/conf.
GENERIC_OPTIONS The common set of options supported by multiple commands. Full list is .
COMMAND_OPTIONS Various commands with their options are described in the following sections. The commands have been grouped into and .
来源: 
 
2.GENREIC_OPTIONS基本命令如下:
GENERIC_OPTION Description
-conf <configuration file> Specify an application configuration file.  指定配置文件
-D <property>=<value> Use value for given property.                   指定属性
-jt <local> or <resourcemanager:port> Specify a ResourceManager. Applies only to job.   指定资源管理器
-files <comma separated list of files> Specify comma separated files to be copied to the map reduce cluster. Applies only to job.
-libjars <comma seperated list of jars> Specify comma separated jar files to include in the classpath. Applies only to job.
-archives <comma separated list of archives> Specify comma separated archives to be unarchived on the compute machines. Applies only to job.
 
3.用户命令
文件系统命令:
hdfs dfs [GENERIC_OPTIONS] [COMMAND_OPTIONS] 参考:
 
.
口令命令   :
hdfs fetchdt [GENERIC_OPTIONS] [--webservice <namenode_http_addr>] <path>
文件检查命令:
hdfs fsck [GENERIC_OPTIONS] <path> [-list-corruptfileblocks | [-move | -delete | -openforwrite] [-files [-
    
    
    
locks   -locations | -racks]]]] [-includeSnapshots]
版本信息
   :
hdfs version
 
4.管理命令
dfsadmi
平衡
    
    :
hdfs balancer [-threshold <threshold>] [-policy <policy>]
启动回滚撤销datanode :
 
hdfs datanode [-regular | -rollback | -rollingupgrace rollback]
dfsadmin管理:
hdfs dfsadmin [GENERIC_OPTIONS]
mover数据迁移工具
   
:
hdfs mover [-p <files/dirs> | -f <local file name>] 类似于balancer,会检查块是否满足存储要求
namenode管理:
Usage: hdfs namenode [-backup] |          [-checkpoint] |          [-format [-clusterid cid ] [-force] [-nonInteractive] ] |          [-upgrade [-clusterid cid] [-renameReserved
] ] | [-upgradeOnly [-clusterid cid] [-renameReserved
] ] | [-rollback] | [-rollingUpgrade
] | [-finalize] | [-importCheckpoint] | [-initializeSharedEdits] | [-bootstrapStandby] | [-recover [-force] ] | [-metadataVersion ]
 
secondary namenode:

hdfs secondarynamenode [-checkpoint [force]] | [-format] | [-geteditsize]

COMMAND_OPTION Description
-checkpoint [force] Checkpoints the SecondaryNameNode if EditLog size >= fs.checkpoint.size. If force is used, checkpoint irrespective of EditLog size.
-format Format the local storage during startup.
-geteditsize Prints the number of uncheckpointed transactions on the NameNode.
 
 
注意:
sbin/start-dfs.sh实际上就是启动了namenode datanode secondarynamenode zooker等
  --script "$bin/hdfs" start namenode $nameStartOpt
  --script "$bin/hdfs" start datanode $dataStartOpt
  --script "$bin/hdfs" start secondarynamenode
  --script "$bin/hdfs" start zkfc
通过SSH连接到nanenode上启动
5.存储策略
hdfs storagepolicies 列出策略
hdfs dfsadmin -setStoragePolicy 
设置策略
hdfs dfsadmin -getStoragePolicy 
获取策略
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

<wiz_tmp_tag id="wiz-table-range-border" contenteditable="false" style="display: none;">

 
 
 
 

转载于:https://www.cnblogs.com/skyrim/p/7455532.html

你可能感兴趣的文章
2012 Multi-University Training Contest 1
查看>>
推荐几本Python书
查看>>
Django基础五之django模型层(一)单表操作
查看>>
命令模式/command模式/行为型模式
查看>>
【pyhon】理想论坛爬虫1.05版,将读取和写DB分离成两个文件
查看>>
w3c提供的performance.timing各阶段api图
查看>>
python笔记:python的表达式
查看>>
如鹏网学习笔记(五)MySql基础
查看>>
Luogu P1886 滑动窗口 题解报告
查看>>
hosts文件修改
查看>>
POJ 3048 线性筛法求素数
查看>>
BZOJ 2120 带修莫队
查看>>
BZOJ 4867 分块+神tm卡常
查看>>
最小割 + 网络流变体
查看>>
模板中用url_for的好处
查看>>
PostgreSQL and bloat
查看>>
H5音频处理的一些小知识
查看>>
转:MFC消息大全
查看>>
HTML5 摇一摇加强版之一次失败的探索
查看>>
SSL 1589——火车票
查看>>