我的Hexo博客

部署ELK日志平台7.11.1

发表于2022-06-17|大数据ELK

ES部署创建数据目录12mkdir -pv /data/elasticsearch/{config,data,logs}chown 1000 /data/elasticsearch/{data,logs} 修改主机配置123456789vim /etc/sysctl.conf加入vm.max_map_count=655360sysctl -pvim /etc/security/limits.conf加入* soft memlock unlimited* hard memlock unlimited 配置文件123456789101112131415161718192021222324252627282930cat > /data/elasticsearch/config/elasticsearch.yml << 'EOF'cluster.name: git_es_clusternode.name: node-3network.host: 3.1.101.35http.port: 9200bootst...

Spark3.X分布式集群部署

发表于2021-09-29|大数据Spark

一、部署规划hadoop高可用集群部署参考: Hadoop3.X分布式高可用集群部署 1.1 版本说明软件版本操作系统 CentOS Linux release 7.8.2003 (Core) JAVA jdk-8u271-linux-x64 Hadoop hadoop-3.2.2 Scala Scala2.12.15 Spark spark-3.1.2-bin-hadoop3.2 1.2 集群规划 hostname IP 组件 master 172.16.20.200 NameNode Spark-Master secondmaster 172.16.20.201 NameNode Spark-Master slave1 172.16.20.202 Zookeeper DataNode NodeManage Spark-Worker slave2 172.16.20.203 Zookeeper DataNode NodeManage Spark-Worker slave3 172.16.20....

Hadoop3.X分布式高可用集群部署

发表于2021-09-28|大数据Hadoop

一、部署规划1.1 版本说明软件版本操作系统 CentOS Linux release 7.8.2003 (Core) hadoop hadoop-3.2.2 JAVA jdk-8u271-linux-x64 1.2 集群规划 hostname IP 组件 master 172.16.20.200 NameNode ZKFailoverController secondmaster 172.16.20.201 NameNode ZKFailoverController slave1 172.16.20.202 Zookeeper journalnode DataNode NodeManage ResourceManager slave2 172.16.20.203 Zookeeper journalnode DataNode NodeManage ResourceManager slave3 172.16.20.204 Zookeeper journalnode DataNode N...

Hbase2.3.X高可用部署

发表于2021-09-26|大数据Hbase

一、部署规划hadoop高可用集群部署参考: Hadoop3.X分布式高可用集群部署 1.1 版本说明软件版本操作系统 CentOS Linux release 7.8.2003 (Core) JAVA jdk-8u271-linux-x64 Hadoop hadoop-3.2.2 Hbase hbase-2.3.6 1.2 集群规划 hostname IP 组件 master 172.16.20.200 NameNode Hbase-HMaster secondmaster 172.16.20.201 NameNode Hbase-HMaster slave1 172.16.20.202 Zookeeper DataNode NodeManage Hbase-HRegionServer slave2 172.16.20.203 Zookeeper DataNode NodeManage Hbase-HRegionServer slave3 172.16.20.204 Zookeeper Data...

Hive3.X高可用部署

发表于2021-09-15|大数据Hive

一、部署规划hadoop高可用集群部署参考: Hadoop3.X分布式高可用集群部署 1.1 版本说明软件版本操作系统 CentOS Linux release 7.8.2003 (Core) JAVA jdk-8u271-linux-x64 Hadoop hadoop-3.2.2 Hive hive-3.1.2 1.2 集群规划 hive远程模式 && hiveserver2高可用 hostname IP 组件 master 172.16.20.200 NameNode hive-metastore secondmaster 172.16.20.201 NameNode hive-metastore slave1 172.16.20.202 Zookeeper DataNode NodeManage hiveserver2 slave2 172.16.20.203 Zookeeper DataNode NodeManage hiveserver2 slave3 172.16.2...