在docker中模拟不同主机快速搭建GBase 8c集群环境
导读：环境准备申请环境宿主机：32g内存，8g swap，需要保证每台机器至少4g内存+8g swap，否则不能安装。。。IPhostname角色172.72.3.30gbase8c_1gha_server（高可用服务）、dcs（分布式配置存储）...
环境准备
申请环境
宿主机：32g内存，8g swap，需要保证每台机器至少4g内存+8g swap，否则不能安装。。。
IP | hostname | 角色 |
|---|---|---|
172.72.3.30 | gbase8c_1 | gha_server（高可用服务）、dcs（分布式配置存储）、gtm（全局事务管理）、coordinator（协调器） |
172.72.3.31 | gbase8c_2 | datanode（数据节点） 1 |
172.72.3.32 | gbase8c_3 | datanode（数据节点） 2 |
-- 网卡
docker network create --subnet=172.72.0.0/16 lhrnw
docker rm -f gbase8c_1
docker run -itd --name gbase8c_1 -h gbase8c_1 \
--net=lhrnw --ip 172.72.3.30 \
-p 63330:5432 \
-v /sys/fs/cgroup:/sys/fs/cgroup \
--privileged=true \
--add-host='gbase8c_1:172.72.3.30' \
--add-host='gbase8c_2:172.72.3.31' \
--add-host='gbase8c_3:172.72.3.32' \
lhrbest/lhrcentos76:9.0 \
/usr/sbin/init
docker rm -f gbase8c_2
docker run -itd --name gbase8c_2 -h gbase8c_2 \
--net=lhrnw --ip 172.72.3.31 \
-v /sys/fs/cgroup:/sys/fs/cgroup \
--privileged=true \
--add-host='gbase8c_1:172.72.3.30' \
--add-host='gbase8c_2:172.72.3.31' \
--add-host='gbase8c_3:172.72.3.32' \
lhrbest/lhrcentos76:9.0 \
/usr/sbin/init
docker rm -f gbase8c_3
docker run -itd --name gbase8c_3 -h gbase8c_3 \
--net=lhrnw --ip 172.72.3.32 \
-v /sys/fs/cgroup:/sys/fs/cgroup \
--privileged=true \
--add-host='gbase8c_1:172.72.3.30' \
--add-host='gbase8c_2:172.72.3.31' \
--add-host='gbase8c_3:172.72.3.32' \
lhrbest/lhrcentos76:9.0 \
/usr/sbin/init
docker cp GBase8cV5_S3.0.0B76_centos7.8_x86_64.tar.gz gbase8c_1:/soft/
docker cp sshUserSetup.sh gbase8c_1:/soft/
增加swap空间并调整swap配置
若内存为4g或更小，则需要增加swap空间，并增加swappiness参数，否则内存耗尽，系统会很卡，导致集群状态不对：
dd if=/dev/zero of=/root/swapfile bs=10M count=3200
chmod -R 0600 /root/swapfile
mkswap /root/swapfile
swapon /root/swapfile
echo '/root/swapfile swap swap defaults 0 0' >
>
/etc/fstab
swapon -s
echo 20 >
/proc/sys/vm/swappiness
echo 'vm.swappiness=20' >
>
/etc/sysctl.conf
cat /proc/sys/vm/swappiness
cat /etc/sysctl.conf | grep swappiness
每台机器内存至少4G，且需要配置swap 8G，否则会报错“Failed to initialize the memory protect for g_instance.attr.attr_storage.cstore_buffers (1024 Mbytes) or shared memory (4496 Mbytes) is larger.” 、“could not create shared memory segment: Cannot allocate memory”、“This error usually means that openGauss's request for a shared memory segment exceeded available memory or swap space, or exceeded your kernel's SHMALL parameter. You can either reduce the request size or reconfigure the kernel with larger SHMALL. To reduce the request size (currently 1841225728 bytes), reduce openGauss's shared memory usage, perhaps by reducing shared_buffers.”
shared_buffers = 126MB
也可以修改参数：
sudo sed -i '/shared_buffers = 1GB/c shared_buffers = 256MB' /home/gbase/data/gtm/gtm1/postgresql.conf
sudo sed -i '/shared_buffers = 1GB/c shared_buffers = 256MB' /home/gbase/data/coord/cn1/postgresql.conf
sudo sed -i '/shared_buffers = 1GB/c shared_buffers = 256MB' /home/gbase/data/dn1/dn1_1/postgresql.conf
sudo sed -i '/shared_buffers = 1GB/c shared_buffers = 256MB' /home/gbase/data/dn2/dn2_1/postgresql.conf
sudo sed -i '/cstore_buffers = 1GB/c cstore_buffers = 32MB' /home/gbase/data/gtm/gtm1/postgresql.conf
sudo sed -i '/cstore_buffers = 1GB/c cstore_buffers = 32MB' /home/gbase/data/coord/cn1/postgresql.conf
sudo sed -i '/cstore_buffers = 1GB/c cstore_buffers = 32MB' /home/gbase/data/dn1/dn1_1/postgresql.conf
sudo sed -i '/cstore_buffers = 1GB/c cstore_buffers = 32MB' /home/gbase/data/dn2/dn2_1/postgresql.conf
所有节点安装依赖包
yum install -y libaio-devel flex bison ncurses-devel \
glibc-devel patch readline-devel bzip2 firewalld \
crontabs net-tools openssh-server openssh-clients which sshpass \
ntp chrony
systemctl disable firewalld
systemctl stop firewalld
systemctl status firewalld
-- 重启OS
sed -i s/SELINUX=enforcing/SELINUX=disabled/g /etc/selinux/config
setenforce 0
getenforce
-- 安装 ntp 组件或 chronyd 组件，确保集群各个节点之间的时间同步
systemctl unmask ntpd
systemctl enable ntpd
systemctl start ntpd
systemctl status ntpd
systemctl status ntpd
systemctl disable chronyd
systemctl stop chronyd
systemctl status chronyd
修改主机名
注意修改三个节点的 IP 地址，这里我使用如下三个 IP，并分别修改 hostname。
node 1: hostnamectl set-hostname gbase8c_1.lhr.com
node 2: hostnamectl set-hostname gbase8c_2.lhr.com
node 3: hostnamectl set-hostname gbase8c_3.lhr.com
docker的话可以不用管。
修改内核参数
cat >
>
/etc/sysctl.conf "EOF"
kernel.shmmax = 4398046511104
kernel.shmmni = 4096
kernel.shmall = 4000000000
kernel.sem = 32000 1024000000 500 32000
EOF
sysctl -p
若kernel.shmmax配置过小，会报共享内存相关的错误：
FATAL: could not create shared memory segment: Invalid argument DETAIL: Failed system call was shmget(key=6666001, size=4714683328, 03600). HINT: This error usually means that openGauss's request for a shared memory segment exceeded your kernel's SHMMAX parameter. You can either reduce the request size or reconfigure the kernel with larger SHMMAX. To reduce the request size (currently 4714683328 bytes), reduce openGauss's shared memory usage, perhaps by reducing shared_buffers. If the request size is already small, it's possible that it is less than your kernel's SHMMIN parameter, in which case raising the request size or reconfiguring SHMMIN is called for. The openGauss documentation contains more information about shared memory configuration.
所有节点创建用户
useradd gbase
echo "gbase:lhr" | chpasswd
echo "gbase ALL=(ALL) NOPASSWD: ALL" >
>
/etc/sudoers
配置互信
以root用户只在主节点操作：
./sshUserSetup.sh -user gbase -hosts "gbase8c_1 gbase8c_2 gbase8c_3" -advanced exverify –confirm
-- 所有节点
chmod 600 /home/gbase/.ssh/config
解压安装包
只在主节点操作，解压安装包 GBase8cV5_S3.0.0B76_centos7.8_x86_64.tar.gz：
su - gbase
mkdir -p /home/gbase/gbase_package
cp /soft/GBase8cV5_S3.0.0B76_centos7.8_x86_64.tar.gz /home/gbase/gbase_package
cd /home/gbase/gbase_package
tar -zxvf GBase8cV5_S3.0.0B76_centos7.8_x86_64.tar.gz
tar zxf GBase8cV5_S3.0.0B76_CentOS_x86_64_om.tar.gz
示例：
[gbase@gbase8c_1 gbase_package]$ ll -h
total 514M
drwxrwxr-x 5 gbase gbase 165 Feb 27 16:48 dependency
-rw-r--r-- 1 root root 257M Mar 17 16:36 GBase8cV5_S3.0.0B76_centos7.8_x86_64.tar.gz
-rw-rw-r-- 1 gbase gbase 65 Feb 27 16:48 GBase8cV5_S3.0.0B76_CentOS_x86_64_om.sha256
-rw-rw-r-- 1 gbase gbase 99M Feb 27 16:48 GBase8cV5_S3.0.0B76_CentOS_x86_64_om.tar.gz
-rw-rw-r-- 1 gbase gbase 1012K Feb 27 16:48 GBase8cV5_S3.0.0B76_CentOS_x86_64_pgpool.tar.gz
-rw-rw-r-- 1 gbase gbase 65 Feb 27 16:48 GBase8cV5_S3.0.0B76_CentOS_x86_64.sha256
-rw-rw-r-- 1 gbase gbase 158M Feb 27 16:48 GBase8cV5_S3.0.0B76_CentOS_x86_64.tar.bz2
-rw-rw-r-- 1 gbase gbase 2.6K Feb 27 16:48 gbase.yml
drwxrwxr-x 11 gbase gbase 4.0K Feb 27 16:48 gha
-rw-rw-r-- 1 gbase gbase 188 Feb 27 16:48 gha_ctl.ini
drwxrwxr-x 2 gbase gbase 96 Feb 27 16:48 lib
-rw-rw-r-- 1 gbase gbase 729 Feb 27 16:48 package_info.json
drwxr-xr-x 4 gbase gbase 28 Mar 16 2021 python3.8
drwxrwxr-x 10 gbase gbase 4.0K Feb 27 16:48 script
drwxrwxr-x 2 gbase gbase 330 Feb 27 16:48 simpleInstall
-rw-rw-r-- 1 gbase gbase 118 Feb 27 16:48 ubuntu_version.json
drwx------ 6 gbase gbase 87 Jul 2 2022 venv
-rw-rw-r-- 1 gbase gbase 36 Feb 27 16:48 version.cfg
[gbase@gbase8c_1 gbase_package]$
开始安装
只在主节点操作。
编辑集群部署文件 gbase8c.yml
cd /home/gbase/gbase_package
mv gbase.yml gbase.yml.bak
- gbase.yml 修改如下
cat >
/home/gbase/gbase_package/gbase.yml "EOF"
gha_server:
- gha_server1:
host: 172.72.3.30
port: 20001
dcs:
- host: 172.72.3.30
port: 2379
gtm:
- gtm1:
host: 172.72.3.30
agent_host: 172.72.3.30
role: primary
port: 6666
agent_port: 8001
work_dir: /home/gbase/data/gtm/gtm1
coordinator:
- cn1:
host: 172.72.3.30
agent_host: 172.72.3.30
role: primary
port: 5432
agent_port: 8003
work_dir: /home/gbase/data/coord/cn1
datanode:
- dn1:
- dn1_1:
host: 172.72.3.31
agent_host: 172.72.3.31
role: primary
port: 15432
agent_port: 8005
work_dir: /home/gbase/data/dn1/dn1_1
- dn2:
- dn2_1:
host: 172.72.3.32
agent_host: 172.72.3.32
role: primary
port: 20010
agent_port: 8007
work_dir: /home/gbase/data/dn2/dn2_1
env:
# cluster_type allowed values: multiple-nodes, single-inst, default is multiple-nodes
cluster_type: multiple-nodes
pkg_path: /home/gbase/gbase_package # 安装包所在路径
prefix: /home/gbase/gbase_db # 运行目录
version: V5_S3.0.0B76 # 与安装包版本一致 GBase8cV5_S3.0.0B76
user: gbase
port: 22
# constant:
# virtual_ip: 172.72.3.36/24
EOF
执行安装脚本
su - gbase
cd /home/gbase/gbase_package/script
./gha_ctl install -c gbase -p /home/gbase/gbase_package -f
注释：
-c 参数：数据库名称，默认 gbase -p 参数：配置文件路径，默认 /home/gbase -f参数：若集群已存在，可以强制安装
安装日志位置
tailf /tmp/gha_ctl/gha_ctl.log
cd /home/gbase/gbase_db/log/om/
执行时间约 5 分钟，安装结束后，脚本会提示：
{
"ret":0,
"msg":"Success"
}
集群安装成功！
状态检查
执行
/home/gbase/gbase_package/script/gha_ctl monitor -l http://172.72.3.30:2379
/home/gbase/gbase_package/script/gha_ctl monitor -l http://172.72.3.30:2379 -H
结果如下，说明集群安装正常，数据服务启动中
[gbase@gbase8c_1 ~]$ /home/gbase/gbase_package/script/gha_ctl monitor -l http://172.72.3.30:2379 -H
+----+-------------+-------------+-------+---------+--------+
| No | name | host | port | state | leader |
+----+-------------+-------------+-------+---------+--------+
| 0 | gha_server1 | 172.72.3.30 | 20001 | running | True |
+----+-------------+-------------+-------+---------+--------+
+----+------+-------------+------+---------------------------+---------+---------+
| No | name | host | port | work_dir | state | role |
+----+------+-------------+------+---------------------------+---------+---------+
| 0 | gtm1 | 172.72.3.30 | 6666 | /home/gbase/data/gtm/gtm1 | running | primary |
+----+------+-------------+------+---------------------------+---------+---------+
+----+------+-------------+------+----------------------------+---------+---------+
| No | name | host | port | work_dir | state | role |
+----+------+-------------+------+----------------------------+---------+---------+
| 0 | cn1 | 172.72.3.30 | 5432 | /home/gbase/data/coord/cn1 | running | primary |
+----+------+-------------+------+----------------------------+---------+---------+
+----+-------+-------+-------------+-------+----------------------------+---------+---------+
| No | group | name | host | port | work_dir | state | role |
+----+-------+-------+-------------+-------+----------------------------+---------+---------+
| 0 | dn1 | dn1_1 | 172.72.3.31 | 15432 | /home/gbase/data/dn1/dn1_1 | running | primary |
| 1 | dn2 | dn2_1 | 172.72.3.32 | 20010 | /home/gbase/data/dn2/dn2_1 | running | primary |
+----+-------+-------+-------------+-------+----------------------------+---------+---------+
+----+-------------------------+--------+---------+----------+
| No | url | name | state | isLeader |
+----+-------------------------+--------+---------+----------+
| 0 | http://172.72.3.30:2379 | node_0 | healthy | True |
+----+-------------------------+--------+---------+----------+
[gbase@gbase8c_1 ~]$ /home/gbase/gbase_package/script/gha_ctl monitor -l http://172.72.3.30:2379
{
"cluster": "gbase",
"version": "V5_S3.0.0B76",
"server": [
{
"name": "gha_server1",
"host": "172.72.3.30",
"port": "20001",
"state": "running",
"isLeader": true
}
],
"gtm": [
{
"name": "gtm1",
"host": "172.72.3.30",
"port": "6666",
"workDir": "/home/gbase/data/gtm/gtm1",
"agentPort": "8001",
"state": "running",
"role": "primary",
"agentHost": "172.72.3.30"
}
],
"coordinator": [
{
"name": "cn1",
"host": "172.72.3.30",
"port": "5432",
"workDir": "/home/gbase/data/coord/cn1",
"agentPort": "8003",
"state": "running",
"role": "primary",
"agentHost": "172.72.3.30",
"central": true
}
],
"datanode": {
"dn1": [
{
"name": "dn1_1",
"host": "172.72.3.31",
"port": "15432",
"workDir": "/home/gbase/data/dn1/dn1_1",
"agentPort": "8005",
"state": "running",
"role": "primary",
"agentHost": "172.72.3.31"
}
],
"dn2": [
{
"name": "dn2_1",
"host": "172.72.3.32",
"port": "20010",
"workDir": "/home/gbase/data/dn2/dn2_1",
"agentPort": "8007",
"state": "running",
"role": "primary",
"agentHost": "172.72.3.32"
}
]
}
,
"dcs": {
"clusterState": "healthy",
"members": [
{
"url": "http://172.72.3.30:2379",
"id": "b337bb545c63ee23",
"name": "node_0",
"isLeader": true,
"state": "healthy"
}
]
}
}
[gbase@gbase8c_1 ~]$
数据库启停
停止数据库服务
su - gbase
/home/gbase/gbase_package/script/gha_ctl stop all -l http://172.72.3.30:2379
-- 或
systemctl stop coordinator_gbase_cn1.service
systemctl stop datanode_gbase_dn1_1.service
systemctl stop gtm_gbase_gtm1.service
systemctl stop server_gbase_gha_server1.service
systemctl stop etcd.service
启动数据库服务
systemctl start etcd.service
su - gbase
/home/gbase/gbase_package/script/gha_ctl start all -l http://172.72.3.30:2379
连接和 SQL 测试
在主节点 gbase8c_1 执行 $ gsql -d postgres -p 5432，出现 postgres=# 操作符说明客户端工具 gsql 成功连接 GBase 8c 数据库。
[gbase@gbase8c_1 script]$ gsql -d postgres -p 5432
gsql ((multiple_nodes GBase8cV5 3.0.0B76 build 47948f99) compiled at 2023-02-27 16:04:20 commit 0 last mr 1232 )
Non-SSL connection (SSL connection is recommended when requiring high-security)
Type "help" for help.
testdb=# select version();
version
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
PostgreSQL 9.2.4 (multiple_nodes GBase8cV5 3.0.0B76 build 47948f99) compiled at 2023-02-27 16:04:20 commit 0 last mr 1232 on x86_64-unknown-linux-gnu, compiled by g++ (GCC) 7.3.0, 64-bit
(1 row)
testdb=#
postgres=# create database testdb;
CREATE DATABASE
postgres=# \c testdb
Non-SSL connection (SSL connection is recommended when requiring high-security)
You are now connected to database "testdb" as user "gbase".
testdb=# create table student(ID int, Name varchar(10));
CREATE TABLE
testdb=# insert into student values(1, 'Mike'),(2,'John');
INSERT 0 2
testdb=# select * from student;
id | name
----+------
1 | Mike
2 | John
(2 rows)
testdb=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+-------+----------+---------+-------+-------------------
postgres | gbase | UTF8 | C | C |
template0 | gbase | UTF8 | C | C | =c/gbase +
| | | | | gbase=CTc/gbase
template1 | gbase | UTF8 | C | C | =c/gbase +
| | | | | gbase=CTc/gbase
testdb | gbase | UTF8 | C | C |
(4 rows)
testdb=# \d student
Table "public.student"
Column | Type | Modifiers
--------+-----------------------+-----------
id | integer |
name | character varying(10) |
testdb=#
卸载集群
在主节点（172.72.3.30）执行以下命令
-- 1、停止所有节点的集群服务
gha_ctl stop all -l http://172.72.3.30:2379
-- 2、集群程序的卸载：
gha_ctl uninstall -l http://172.72.3.30:2379
-- 3、移除 dcs 集群：
cd /home/gbase/gbase_package/script
./gha_ctl destroy dcs -l http://172.72.3.30:2379
netstat -tulnp
systemctl list-unit| grep gbase
systemctl stop coordinator_gbase_cn1.service
systemctl stop datanode_gbase_dn1_1.service
systemctl stop gtm_gbase_gtm1.service
systemctl stop server_gbase_gha_server1.service
systemctl stop etcd.service
systemctl disable coordinator_gbase_cn1.service
systemctl disable datanode_gbase_dn1_1.service
systemctl disable gtm_gbase_gtm1.service
systemctl disable server_gbase_gha_server1.service
systemctl disable etcd.service
pkill -9 python3
pkill -9 gaussdb
rm -rf /home/gbase/data
rm -rf /home/gbase/gbase_db
环境变量
[gbase@gbase8c_1 om]$ cat /home/gbase/.bashrc
export GPHOME=/home/gbase/gbase_db/om
export PATH=$GPHOME/script/gspylib/pssh/bin:$GPHOME/venv/bin:$GPHOME/script:$PATH
export LD_LIBRARY_PATH=$GPHOME/lib:$LD_LIBRARY_PATH
export PYTHONPATH=$GPHOME/lib
export GAUSSHOME=/home/gbase/gbase_db/app
export PATH=$GAUSSHOME/bin:$PATH
export LD_LIBRARY_PATH=$GAUSSHOME/lib:$LD_LIBRARY_PATH
export S3_CLIENT_CRT_FILE=$GAUSSHOME/lib/client.crt
export GAUSS_VERSION=V5_S3.0.0B76
export PGHOST=/home/gbase/gbase_db/tmp
export GAUSSLOG=/home/gbase/gbase_db/log
umask 077
export GAUSS_ENV=2
export GAUSS_PSSH_PORT=22
export GS_CLUSTER_NAME=gbase
安装脚本会自动帮我们配置，无需我们手动配置。
修改密码
postgres=# select * from pg_roles;
rolname | rolsuper | rolinherit | rolcreaterole | rolcreatedb | rolcatupdate | rolcanlogin | rolreplication | rolauditadmin | rolsystemadmin | rolconnlimit | rolpassword | rolvalidbegin | rolvaliduntil | rolrespool | rolparentid | roltabspace | rolconfig | oid | roluseft | rolkind | nodegroup | roltempspace | rolspillspace | rolmonitoradmin | roloperatoradmin | rolpolicyadmin
--------------------------+----------+------------+---------------+-------------+--------------+-------------+----------------+---------------+----------------+--------------+-------------+---------------+---------------+--------------+-------------+-------------+-----------+-------+----------+---------+-----------+--------------+---------------+-----------------+------------------+----------------
gbase | t | t | t | t | t | t | t | t | t | -1 | ******** | | | default_pool | 0 | | | 10 | t | n | | | | t | t | t
gs_role_directory_drop | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1059 | f | n | | | | f | f | f
gs_role_directory_create | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1056 | f | n | | | | f | f | f
gs_role_pldebugger | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1055 | f | n | | | | f | f | f
gs_role_account_lock | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1048 | f | n | | | | f | f | f
gs_role_replication | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1047 | f | n | | | | f | f | f
gs_role_tablespace | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1046 | f | n | | | | f | f | f
gs_role_signal_backend | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1045 | f | n | | | | f | f | f
gs_role_copy_files | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1044 | f | n | | | | f | f | f
gha | f | t | f | f | f | t | f | f | t | -1 | ******** | | | default_pool | 0 | | | 16965 | f | n | | | | f | f | f
(10 rows)
postgres=# alter system set password_policy=0;
ALTER SYSTEM SET
postgres=# alter system set password_encryption_type=1;
ALTER SYSTEM SET
postgres=# ALTER ROLE gbase PASSWORD 'lhr';
ALTER ROLE
postgres=# ALTER ROLE gha PASSWORD 'lhr';
配置远程登录
cat >
>
/home/gbase/data/coord/cn1/postgresql.conf "EOF"
listen_addresses = '*'
EOF
echo "host all all 0.0.0.0/0 md5" >
>
/home/gbase/data/coord/cn1/pg_hba.conf
-- 初始化数据库的用户gbase，是不能通过IP远程连接的，所以需要创建另外一个用户才能远程连接
gsql -p 5432 -d postgres
create user lhr with password 'lhr' sysadmin ;
grant all PRIVILEGES to lhr;
postgres=# select * from pg_roles;
rolname | rolsuper | rolinherit | rolcreaterole | rolcreatedb | rolcatupdate | rolcanlogin | rolreplication | rolauditadmin | rolsystemadmin | rolconnlimit | rolpassword | rolvalidbegin | rolvaliduntil | rolrespool | rolparentid | roltabspace | rolconfig | oid | roluseft | rolkind | nodegroup | roltempspace | rolspillspace | rolmonitoradmin | roloperatoradmin | rolpolicyadmin
--------------------------+----------+------------+---------------+-------------+--------------+-------------+----------------+---------------+----------------+--------------+-------------+---------------+---------------+--------------+-------------+-------------+-----------+-------+----------+---------+-----------+--------------+---------------+-----------------+------------------+----------------
gbase | t | t | t | t | t | t | t | t | t | -1 | ******** | | | default_pool | 0 | | | 10 | t | n | | | | t | t | t
gs_role_directory_drop | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1059 | f | n | | | | f | f | f
gs_role_directory_create | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1056 | f | n | | | | f | f | f
gs_role_pldebugger | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1055 | f | n | | | | f | f | f
gs_role_account_lock | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1048 | f | n | | | | f | f | f
gs_role_replication | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1047 | f | n | | | | f | f | f
gs_role_tablespace | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1046 | f | n | | | | f | f | f
gs_role_signal_backend | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1045 | f | n | | | | f | f | f
gs_role_copy_files | f | t | f | f | f | f | f | f | f | -1 | ******** | | | default_pool | 0 | | | 1044 | f | n | | | | f | f | f
lhr | f | t | f | f | f | t | f | f | t | -1 | ******** | | | default_pool | 0 | | | 16975 | f | n | | | | f | f | f
gha | f | t | f | f | f | t | f | f | t | -1 | ******** | | | default_pool | 0 | | | 16965 | f | n | | | | f | f | f
(11 rows)
postgres=#
-- 重启集群后 远程登录
gsql -U gha -h 172.72.3.30 -p 5432 -d postgres
C:\Users\lhr>
psql -U gha -h 192.168.88.80 -p 5432 -d postgres
Password for user gha:
psql (14.0, server 9.2.4)
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, bits: 256, compression: off)
Type "help" for help.
postgres=>
\l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+-------+----------+---------+-------+-------------------
postgres | gbase | UTF8 | C | C |
template0 | gbase | UTF8 | C | C | =c/gbase +
| | | | | gbase=CTc/gbase
template1 | gbase | UTF8 | C | C | =c/gbase +
| | | | | gbase=CTc/gbase
testdb | gbase | UTF8 | C | C |
(4 rows)
postgres=>
gbase 8c修改参数的命令
gs_guc reload -Z coordinator -N all -I all -c "enable_gbase_mot=true"
gs_guc reload -Z GTM -N all -I all -c "enable_gbase_mot=true"
gs_guc reload -Z datanode -N all -I all -c "enable_gbase_mot=true"
安装错误解决
gaussDB state is Coredump
原因：CPU不支持。。。
请检查是否含有avx、rdtscp指令集，若没有则不能安装。。。
lscpu | grep avx
lscpu | grep rdtscp
AVX（Advanced Vector Extensions）是一组x86 CPU指令集扩展，用于加速向量和矩阵操作。AVX指令集在Intel Sandy Bridge微架构的CPU中首次引入，可用于执行单指令多数据（SIMD）计算。 AVX指令集扩展增加了256位和128位宽的SIMD指令，用于加速浮点运算、整数运算和逻辑运算。这些指令可用于执行并行计算，因为它们能够在一次操作中处理多个数据元素，而不是逐个处理。这使得AVX指令集非常适合执行科学计算、图形处理、媒体处理和数据分析等计算密集型任务。 AVX指令集扩展还提供了一些新的高级指令，例如FMA指令（Fused Multiply-Add），可以将一个乘法和一个加法操作结合起来，从而提高计算速度。另外，AVX指令集还增加了一些新的控制指令，用于支持更灵活的条件分支和循环结构。 总之，AVX指令集扩展可以大幅提高计算机系统的运算速度，特别是对于需要大量浮点运算和向量处理的应用程序来说。
rdtscp 是一个 x86 CPU 指令，它的作用是读取一个 64 位的时间戳计数器（TSC）的值，并且返回当前的处理器周期计数（即时钟周期数）和时间戳计数器的值。
时间戳计数器是一个单调递增的计数器，它记录了 CPU 自从上电以来所经过的时钟周期数。它通常被用来测量代码执行时间、性能分析和同步等方面。而 rdtscp 指令可以在多核 CPU 环境下提供一致的计数器值，这对于跨核心的时间测量和调试是非常有用的。
需要注意的是，rdtscp 指令可能会受到时钟频率调整和 CPU 切换等因素的影响，因此在一些特殊情况下，它可能不够精确。
这个问题在opengauss、Gaussdb和GBase 8c中都存在。。。。。
不支持的CPU列表：
[root@lhrdb greenplum]# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 1
Core(s) per socket: 16
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 15
Model name: Intel(R) Core(TM)2 Duo CPU T7700 @ 2.40GHz
Stepping: 11
CPU MHz: 2593.912
BogoMIPS: 5187.82
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 4096K
NUMA node0 CPU(s): 0-15
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl eagerfpu pni ssse3 cx16 pcid sse4_2 x2apic hypervisor lahf_lm rsb_ctxsw
[root@lhrdb greenplum]# lscpu | grep Flags
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good nopl eagerfpu pni ssse3 cx16 pcid sse4_2 x2apic hypervisor lahf_lm rsb_ctxsw
[root@lhrdb greenplum]#
支持的CPU列表：
[root@lhrxxt ~]# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 16
On-line CPU(s) list: 0-15
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 141
Model name: 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz
Stepping: 1
CPU MHz: 2304.009
BogoMIPS: 4608.01
Virtualization: VT-x
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 48K
L1i cache: 32K
L2 cache: 1280K
L3 cache: 24576K
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid movdiri movdir64b fsrm avx512_vp2intersect flush_l1d arch_capabilities
[root@lhrxxt ~]# lscpu | grep Flags
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid movdiri movdir64b fsrm avx512_vp2intersect flush_l1d arch_capabilities
[root@lhrxxt ~]#
[root@mdw ~]# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 2
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 79
Model name: Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz
Stepping: 1
CPU MHz: 2397.223
BogoMIPS: 4794.44
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 25600K
NUMA node0 CPU(s): 0-7
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology tsc_reliable nonstop_tsc eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 invpcid rtm rdseed adx smap xsaveopt arat
[root@mdw ~]#
error while loading shared libraries: libcjson.so.1
安装完成后需要退出gbase用户，然后重新进入一次 gbase 用户，或者重新引入环境变量，否则无法找到二进制文件或者类包。
[gbase@gbase8c_1 script]$ gsql -d postgres -p 5432
bash: gsql: command not found...
[gbase@gbase8c_1 bin]$ ./gsql -d postgres -p 5432
./gsql: error while loading shared libraries: libcjson.so.1: cannot open shared object file: No such file or directory
[gbase@gbase8c_1 bin]$ ldd gsql
...
libcjson.so.1 =>
not found
libcurl.so.4 =>
/lib64/libcurl.so.4 (0x00007fccb7acb000)
libgssapi_krb5_gauss.so.2 =>
not found
libgssrpc_gauss.so.4 =>
not found
libkrb5_gauss.so.3 =>
not found
libkrb5support_gauss.so.0 =>
not found
libk5crypto_gauss.so.3 =>
not found
libcom_err_gauss.so.3 =>
not found
libpq_ce.so.5 =>
not found
-->
ldconfig, source ~/.bashrc
Current gtm center group num 1 is out of range [0, 0]
[BACKEND] FATAL: Invalid attribute for gtm server.
[BACKEND] DETAIL: Current gtm center group num 1 is out of range [0, 0].
原因：和底层的虚拟化环境有关系，因为我用的是wls2环境，所以报错，若使用VMware workstation或其它环境安装不报错。
Configuration is completed.
Using gbase:gbase to install database.
Using installation program path : /home/gbase/gbase_db/app_89583529
$GAUSSHOME points to /home/gbase/gbase_db/app_89583529, no need to create symbolic link.
Traceback (most recent call last):
File "/home/gbase/gbase_db/om/script/local/Install.py", line 784, in module>
functionDict[g_opts.action]()
File "/home/gbase/gbase_db/om/script/local/Install.py", line 708, in startCluster
gtm.start(self.time_out)
File "/home/gbase/gbase_db/om_89583529/script/local/../gspylib/component/Kernel/Kernel.py", line 109, in start
raise Exception(ErrorCode.GAUSS_516["GAUSS_51607"] % "instance"
Exception: [GAUSS-51607] : Failed to start instance. Error: Please check the gs_ctl log for failure details.
[2023-03-20 09:11:25.006][10270][][gs_ctl]: gs_ctl started,datadir is /home/gbase/data/gtm/gtm1
[2023-03-20 09:11:25.049][10270][][gs_ctl]: waiting for server to start...
.0 LOG: [Alarm Module]can not read GAUSS_WARNING_TYPE env.
0 LOG: [Alarm Module]Host Name: lhrxxt
0 LOG: [Alarm Module]Host IP: 192.168.8.8
0 LOG: [Alarm Module]Cluster Name: gbase
0 LOG: [Alarm Module]Invalid data in AlarmItem file! Read alarm English name failed! line: 57
0 WARNING: failed to open feature control file, please check whether it exists: FileName=gaussdb.version, Errno=2, Errmessage=No such file or directory.
0 WARNING: failed to parse feature control file: gaussdb.version.
0 WARNING: Failed to load the product control file, so gaussdb cannot distinguish product version.
0 LOG: bbox_dump_path is set to /mnt/wslg/dumps/
2023-03-20 09:11:25.129 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm DB010 0 [REDO] LOG: Recovery parallelism, cpu count = 16, max = 4, actual = 4
2023-03-20 09:11:25.129 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm DB010 0 [REDO] LOG: ConfigRecoveryParallelism, true_max_recovery_parallelism:4, max_recovery_parallelism:4
gaussdb.state does not exist, and skipt setting since it is optional.2023-03-20 09:11:25.135 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm 00000 0 [BACKEND] LOG: [Alarm Module]can not read GAUSS_WARNING_TYPE env.
2023-03-20 09:11:25.135 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm 00000 0 [BACKEND] LOG: [Alarm Module]Host Name: lhrxxt
2023-03-20 09:11:25.135 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm 00000 0 [BACKEND] LOG: [Alarm Module]Host IP: 192.168.8.8
2023-03-20 09:11:25.135 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm 00000 0 [BACKEND] LOG: [Alarm Module]Cluster Name: gbase
2023-03-20 09:11:25.135 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm 00000 0 [BACKEND] LOG: [Alarm Module]Invalid data in AlarmItem file! Read alarm English name failed! line: 57
2023-03-20 09:11:25.141 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm 00000 0 [BACKEND] LOG: loaded library "security_plugin"
2023-03-20 09:11:25.146 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm 01000 0 [BACKEND] WARNING: could not create any HA TCP/IP sockets
2023-03-20 09:11:25.146 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm 01000 0 [BACKEND] WARNING: could not create any HA TCP/IP sockets
2023-03-20 09:11:25.160 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm OP0A3 0 [BACKEND] FATAL: Invalid attribute for gtm server.
2023-03-20 09:11:25.160 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm OP0A3 0 [BACKEND] DETAIL: Current gtm center group num 1 is out of range [0, 0].
2023-03-20 09:11:25.160 6417b2bd.1 [unknown] 140672756703680 [unknown] 0 gtm OP0A3 0 [BACKEND] BACKTRACELOG: tid[10273]'s backtrace:
/home/gbase/gbase_db/app/bin/gaussdb(+0xf21792) [0x55a626f21792]
/home/gbase/gbase_db/app/bin/gaussdb(_Z9errfinishiz+0x391) [0x55a626f179d1]
/home/gbase/gbase_db/app/bin/gaussdb(_ZN18GtmServerControler9ParseAttrEv+0x2a3) [0x55a62759e193]
/home/gbase/gbase_db/app/bin/gaussdb(_ZN18GtmServerControler16SetGtmServerInfoEv+0x5f) [0x55a62759f45f]
/home/gbase/gbase_db/app/bin/gaussdb(_Z14PostmasterMainiPPc+0x172f) [0x55a62748c66f]
/home/gbase/gbase_db/app/bin/gaussdb(main+0x3ef) [0x55a626a2906f]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7ff0e6e02555]
/home/gbase/gbase_db/app/bin/gaussdb(+0xa9dea7) [0x55a626a9dea7]
Use addr2line to get pretty function name and line
[2023-03-20 09:11:26.050][10270][][gs_ctl]: waitpid 10273 failed, exitstatus is 256, ret is 2
[2023-03-20 09:11:26.050][10270][][gs_ctl]: stopped waiting
[2023-03-20 09:11:26.050][10270][][gs_ctl]: could not start server
Examine the log output.
Rpc request failed:dn1_1 save node info
{
"ret":80000116,
"msg":"Rpc request failed:dn1_1 save node info"
}
原因：系统资源不足，可以尝试增加内存，或减小shared_buffers参数，然后重启集群。
OPENSSL_1_1_1 not defined in file libcrypto.so.1.1
/home/gbase/gbase_db/app/bin/gaussdb: relocation error: /home/gbase/gbase_db/app/bin/gaussdb: symbol EVP_sm4_ctr, version OPENSSL_1_1_1 not defined in file libcrypto.so.1.1 with link time reference
no data was returned by command \"\"/home/gbase/gbase_db/app/bin/gaussdb\" -V\"
The program \"gaussdb\" is needed by gs_initdb but was not found in the
same directory as \"/home/gbase/gbase_db/app/bin/gs_initdb\".
Check your installation.
Exception ignored in: function Pool.__del__ at 0x7fbf2be67af0>
Traceback (most recent call last):
File \"./lib/python3.8/multiprocessing/pool.py\", line 268, in __del__
File \"./lib/python3.8/multiprocessing/queues.py\", line 368, in put
File \"./lib/python3.8/multiprocessing/connection.py\", line 200, in send_bytes
File \"./lib/python3.8/multiprocessing/connection.py\", line 411, in _send_bytes
File \"./lib/python3.8/multiprocessing/connection.py\", line 368, in _send
OSError: [Errno 9] Bad file descriptor
原因：openssl版本太高导致，需要使用yum安装回默认版本，请参考：https://www.xmmup.com/opensslshengji.html
find / -name openssl
rm -rf /usr/local/src/openssl-1.1.1
rm -rf /usr/local/bin/openssl
rm -rf /usr/local/share/doc/openssl
rm -rf /usr/local/include/openssl
yum install -y openssl openssl-libs openssl-devel openssl-static
yum reinstall -y openssl openssl-libs openssl-devel openssl-static
ln -sf /usr/local/lib64/libssl.so.1.1 /usr/lib64/libssl.so.1.1
ln -sf /usr/local/lib64/libcrypto.so.1.1 /usr/lib64/libcrypto.so.1.1
Exception: Failed to obtain host name. The cmd is hostname
原因：互信没有配置好。
install or upgrade dependency { 'patch' : Mone} failed
原因：patch包没有安装
解决：
yum install -y patch
巡检脚本
更多请参考：https://www.xmmup.com/nandatongyonggbaseshujukuxunjianjiaoben.html
其它不再截图。
总结
1、内存要大，否则安装不起来
2、openssl不能升级，需要默认的版本
3、CPU需要支持avx、rdtscp指令集
4、虚拟化底层不能是wls2
参考
https://www.modb.pro/db/618721
https://blog.csdn.net/wiserhowe/article/details/128833346
https://blog.csdn.net/wiserhowe/article/details/126725440
https://mp.weixin.qq.com/s/yT-v0f5raST8pw9Fiu_HLA
声明：本文内容由网友自发贡献，本站不承担相应法律责任。对本内容有异议或投诉，请联系2913721942#qq.com核实处理，我们将尽快回复您，谢谢合作！
若转载请注明出处： 在docker中模拟不同主机快速搭建GBase 8c集群环境
本文地址： https://pptw.com/jishu/290757.html