Hyperledger Fabric远程多节点环境部署过程

程序员文章站 2022-03-17 23:28:54

...

很多Hyperledger Fabric的初学者会从官方的github上clone一份代码到本地，运行其中的demo（fabric下的e2e_cli或者fabric-samples下的first-network），在本地利用虚拟机搭建fabric网络来进行开发测试，那种方式按照官方的wiki以及md教程比较容易，只需要安装好必要的基础软件如docker，docker-compose，go...即可通过提供的.yaml和.sh脚本快速的运行起来。但是对于准生产或者生产环境就不能这么简单的完成了，如果不购买第三方提供的Baas服务，则需要自己一个个的进行安装配置，在这个手动操作的过程中也会遇到不少问题，在此将我在某家银行现场一步步安装配置搭建Fabric网络的准生产环境的过程分享出来，希望能够对有需要的朋友有所帮助。

首先说明下硬件环境的一个基本配置：4台CentOS 7.5云主机，4核CPU+16G内存，5M带宽。

如果将4台机器分别编号为A，B，C，D，那么他们各自需要安装的软件及充当的角色如下：

Hyperledger Fabric远程多节点环境部署过程

博客分类： HyperledgerFabric HyperledgerFabric

下面依次介绍需要安装的这些软件安装及配置

1、安装JDK-1.8

关于JDK的安装配置，这里不做过多阐述

2、安装Zookeeper-3.4.12（以机器A为例）

将安装包zookeeper-3.4.12.tar.gz上传到当前登录用户工作空间soft目录下；
解压缩安装包到目标安装目录下并进入到conf目录，复制一份zoo_sample.cfg改名为zoo.cfg；
编辑zoo.cfg配置文件，对配置信息做如下修改
dataDir=/appdata/zookeeper

server.1=A:2888:3888

server.2=B:2888:3888

server.3=C:2888:3888

说明：

2181是对外通讯的端口

2888是集群内机器通讯使用的端口（Leader监听此端口）

3888是选举leader时监听的端口
创建/appdata/zookeeper目录并授权（chown -R user:group /appdata/zookeeper）给当前登录用户，在该目录下创建myid配置文件，写入对应的值（机器A写入1，机器B写入2，机器C写入3）
切换到zookeeper安装目下的bin目录，执行如下命令启动server
zkServer.sh start
默认的日志文件存放在/appdata/zookeeper目录下面
tail -f /appdata/zookeeper/version-2/log.100000001

3、安装kafka-2.11-1.0.1.tgz（以机器A为例）

将安装包kafka_2.11-1.0.1.tgz上传到当前登录用户工作空间soft目录下；
解压缩安装包到目标安装目录下并进入到config目录；

编辑server.properties，对配置信息做如下修改

#kafka broker id，4台主机分别为1,2,3,4

broker.id=1

listeners=PLAINTEXT://A:9092

#kafka主机名

host.name=A

#zookeeper server列表

zookeeper.connect=A:2181,B:2181,C:2181

切换到kafka安装目录下，执行如下命令启动kafka

nohup bin/kafka-server-start.sh config/server.properties &

默认的日志文件存放在/tmp/kafka-logs目录下，如果想要自己指定日志目录，用如下命令启动

nohup ./bin/kafka-server-start.sh ./config/server.properties >logs/kafka.log 2>1 & 
# 注意logs/kafka.log需要手动创建好，不然会报 -bash: logs/kafka.log: No such file or directory错误

4、安装Docker

可以参考我的另一篇博文《Ubuntu 16.04环境下安装Docker17.12.0-ce详细步骤》

5、安装Go

说明：要通过哪台机器连接peer节点进行chaincode的安装才需要在哪台机器上安装go

将安装包go1.9.linux-amd64.tar.gz上传到当前登录用户工作空间soft目录下；
解压缩安装包到目标安装目录下，如：/opt/go
编辑当前用户的环境变量 vi ~/.bash_profile，在文件最后追加如下环境变量配置
```
export GOPATH=/opt/gopath
export GOROOT=/opt/go
export PATH=$PATH:/opt/go/bin
```
重新编译载入环境变量
```
source ~/.bash_profile
```
执行go version检验go是否安装成功

6、安装Fabric Binaries

将安装包hyperledger-fabric-linux-amd64-1.1.0.tar.gz上传到当前登录用户工作空间soft目录下；
解压缩安装包到/home/username/fabric目录下，Fabric binaries文件列表如下：
fabric

├── bin

│ ├── configtxgen //创始区块生成工具

│ ├── configtxlator //channel修改工具

│ ├── cryptogen //pki证书生成工具

│ ├── get-docker-images.sh

│ ├── orderer //orderer节点运行程序

│ └── peer //peer节点运行程序

└── config

├── configtx.yaml //创始区块定义文件

├── core.yaml //peer配置文件

└── orderer.yaml //orderer配置文件

7、生成msp证书文件

定义部署orderer和peer各机构的msp结构模板crypto-config.yaml，可以参考fabric官方源码下的e2e_cli示例代码进行修改配置

# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#

# ---------------------------------------------------------------------------
# "OrdererOrgs" - Definition of organizations managing orderer nodes
# ---------------------------------------------------------------------------
OrdererOrgs:
  # ---------------------------------------------------------------------------
  # Orderer
  # ---------------------------------------------------------------------------
  - Name: Orderer #orderer机构定义
    Domain: example.com  
    # ---------------------------------------------------------------------------
    # "Specs" - See PeerOrgs below for complete description
    # ---------------------------------------------------------------------------
    Specs:
      - Hostname: orderer  #orderer节点主机名定义
      - Hostname: orderer1 #orderer节点主机名定义
# ---------------------------------------------------------------------------
# "PeerOrgs" - Definition of organizations managing peer nodes
# ---------------------------------------------------------------------------
PeerOrgs:
  # ---------------------------------------------------------------------------
  # Org1
  # ---------------------------------------------------------------------------
#  - Name: Org1
#    Domain: org1.example.com
#    Template:
#      Count: 2
#   Users:
#      Count: 1
  - Name: Org2 #peer机构定义
    Domain: org2.example.com
    Template:
      Count: 2 #peer节点数量
    Users:
      Count: 2 #为客户端sdk生成证书的数量

#  - Name: Org3
#    Domain: org3.example.com
#    Template:
#      Count: 2
#    Users:
#      Count: 1

生成orderer和peer的msp文件

./bin/cryptogen generate --config=./crypto-config.yaml

8、生成创始区块和channel定义

创始区块和channel定义在configtx.yaml文件中，内容如下：

# Copyright IBM Corp. All Rights Reserved.
#
# SPDX-License-Identifier: Apache-2.0
#

---
################################################################################
#
#   Profile
#
#   - Different configuration profiles may be encoded here to be specified
#   as parameters to the configtxgen tool
#
################################################################################
Profiles:

    OrgsOrdererGenesis:
        Orderer:
            <<: *OrdererDefaults
            Organizations:
                - *OrdererOrg
        Consortiums:
            SampleConsortium:
                Organizations:
                   # - *Org1
                    -  *Org2
                   # - *Org3
    OrgsChannel:
        Consortium: SampleConsortium
        Application:
            <<: *ApplicationDefaults
            Organizations:
               # - *Org1
                - *Org2
               # - *Org3

################################################################################
#
#   Section: Organizations
#
#   - This section defines the different organizational identities which will
#   be referenced later in the configuration.
#
################################################################################
Organizations:

    # SampleOrg defines an MSP using the sampleconfig.  It should never be used
    # in production but may be used as a template for other definitions
    - &OrdererOrg
        # DefaultOrg defines the organization which is used in the sampleconfig
        # of the fabric.git development environment
        Name: OrdererOrg

        # ID to load the MSP definition as
        ID: OrdererMSP

        # MSPDir is the filesystem path which contains the MSP configuration
        MSPDir: crypto-config/ordererOrganizations/example.com/msp

  #  - &Org1
        # DefaultOrg defines the organization which is used in the sampleconfig
        # of the fabric.git development environment
 #       Name: Org1MSP

        # ID to load the MSP definition as
 #       ID: Org1MSP

 #       MSPDir: crypto-config/peerOrganizations/org1.example.com/msp

 #       AnchorPeers:
            # AnchorPeers defines the location of peers which can be used
            # for cross org gossip communication.  Note, this value is only
            # encoded in the genesis block in the Application section context
#            - Host: peer0.org1.example.com
#              Port: 7051

    - &Org2
        # DefaultOrg defines the organization which is used in the sampleconfig
        # of the fabric.git development environment
        Name: Org2MSP

        # ID to load the MSP definition as
        ID: Org2MSP

        MSPDir: crypto-config/peerOrganizations/org2.example.com/msp

        AnchorPeers:
            # AnchorPeers defines the location of peers which can be used
            # for cross org gossip communication.  Note, this value is only
            # encoded in the genesis block in the Application section context
            - Host: peer0.org2.example.com
              Port: 7051

#    - &Org3
        # DefaultOrg defines the organization which is used in the sampleconfig
        # of the fabric.git development environment
#        Name: Org3MSP

        # ID to load the MSP definition as
#        ID: Org3MSP

#        MSPDir: crypto-config/peerOrganizations/org3.example.com/msp

#        AnchorPeers:
            # AnchorPeers defines the location of peers which can be used
            # for cross org gossip communication.  Note, this value is only
            # encoded in the genesis block in the Application section context
#            - Host: peer0.org3.example.com
#              Port: 7051

################################################################################
#
#   SECTION: Orderer
#
#   - This section defines the values to encode into a config transaction or
#   genesis block for orderer related parameters
#
################################################################################
Orderer: &OrdererDefaults

    # Orderer Type: The orderer implementation to start
    # Available types are "solo" and "kafka"
    OrdererType: kafka

    Addresses:
        - orderer.example.com:7050
        - orderer1.example.com:7050

    # Batch Timeout: The amount of time to wait before creating a batch
    BatchTimeout: 2s

    # Batch Size: Controls the number of messages batched into a block
    BatchSize:

        # Max Message Count: The maximum number of messages to permit in a batch
        MaxMessageCount: 10

        # Absolute Max Bytes: The absolute maximum number of bytes allowed for
        # the serialized messages in a batch.
        AbsoluteMaxBytes: 10 MB

        # Preferred Max Bytes: The preferred maximum number of bytes allowed for
        # the serialized messages in a batch. A message larger than the preferred
        # max bytes will result in a batch larger than preferred max bytes.
        PreferredMaxBytes: 512 KB

    Kafka:
        # Brokers: A list of Kafka brokers to which the orderer connects
        # NOTE: Use IP:port notation
        Brokers:
            - A:9092
            - B:9092
            - C:9092
            - D:9092

    # Organizations is the list of orgs which are defined as participants on
    # the orderer side of the network
    Organizations:

################################################################################
#
#   SECTION: Application
#
#   - This section defines the values to encode into a config transaction or
#   genesis block for application related parameters
#
################################################################################
Application: &ApplicationDefaults

    # Organizations is the list of orgs which are defined as participants on
    # the application side of the network
    Organizations:

手动创建一个文件夹channel-artifacts
```
sudo mkdir channel-artifacts
```

执行命令生成创世区块

sudo bin/configtxgen -profile OrgsOrdererGenesis -outputBlock ./channel-artifacts/genesis.block

执行命令生成channel定义

sudo bin/configtxgen -profile OrgsChannel -outputCreateChannelTx ./channel-artifacts/channel.tx -channelID mychannel

9、配置orderer.yaml

拷贝orderer机器A上的crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/和crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tls/目录到fabric/bin目录下
拷贝orderer机器B上的crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/msp/和crypto-config/ordererOrganizations/example.com/orderers/orderer1.example.com/tls/目录到fabric/bin目录下
分别拷贝orderer机器A，B上的channel-artifacts/genesis.block到fabric/bin目录下

分别拷贝orderer机器A，B上的config/orderer.yaml到fabric/bin目录下，进行编辑

32行 Enabled: false	改为：true		

64行 GenesisMethod: provisional	改为：file

76行 GenesisFile: genesisblock	改为：genesis.block

137行 Location: /var/hyperledger/production/orderer	改为：/data/hyperledger/production/orderer

10、启动orderer节点

nohup ./orderer [./orderer.yaml] >logs/orderer.log 2>1 &

11、导入Fabric chaincode运行环境镜像

在C，D两台peer机器上导入chaincode运行环境相关的基础镜像。

docker load --input fabric-ccenvx86_64-1.1.0.tar
docker load --input fabric-javaenvx86_64-1.1.0.tar
docker load --input fabric-baseos.tar
docker load --input fabric-baseimage.tar

查看已安装
docker images | grep -E *\(1.1.0\|0.4.6\)

12、配置core.yaml

拷贝peer机器C上的crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/msp/和crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/目录到fabric/bin目录下
拷贝peer机器D上的crypto-config/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/msp/和crypto-config/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/tls/目录到fabric/bin目录下

分别拷贝peer机器C，D上的config/core.yaml到fabric/bin目录下，进行编辑

262行 enabled:  false		改为：true

300行 fileSystemPath: /var/hyperledger/production	改为：/data/hyperledger/production

328行 localMspId: DEFAULT	改为：Org2MSP

13、启动peer节点

nohup ./peer [./core.yaml] node start >logs/peer.log 2>1 &

14、Fabric网络初始化-创建通道

以在orderer机器A为例

切换当前目录到fabric/bin下面
拷贝orderer机器A上的crypto-config目录到fabric/bin下面
拷贝orderer机器A上的channel-artifacts/channel.tx到fabric/bin目录下

执行如下命令设置环境变量

#### 命令行执行
#### 环境变量设置
export CORE_PEER_ADDRESS=peer0.org2.example.com:7051
export CORE_PEER_LOCALMSPID=Org2MSP
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_TLS_CERT_FILE=crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/server.crt
export CORE_PEER_TLS_KEY_FILE=crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/server.key
export CORE_PEER_TLS_ROOTCERT_FILE=crypto-config/peerOrganizations/org2.example.com/peers/peer0.org2.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp

执行如下命令创建通道

./peer channel create -o orderer.example.com:7050 -c mychannel -f ./channel.tx --tls true --cafile crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem

15、将peer节点加入通道

当前环境变量配置连接的是peer0节点，直接执行如下命令将peer0节点加入通道
```
./peer channel join -b mychannel.block
```

执行如下命令设置环境变量连接到peer1节点

#### 命令行执行
#### 环境变量设置
export CORE_PEER_ADDRESS=peer1.org2.example.com:7051
export CORE_PEER_LOCALMSPID=Org2MSP
export CORE_PEER_TLS_ENABLED=true
export CORE_PEER_TLS_CERT_FILE=crypto-config/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/tls/server.crt
export CORE_PEER_TLS_KEY_FILE=crypto-config/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/tls/server.key
export CORE_PEER_TLS_ROOTCERT_FILE=crypto-config/peerOrganizations/org2.example.com/peers/peer1.org2.example.com/tls/ca.crt
export CORE_PEER_MSPCONFIGPATH=crypto-config/peerOrganizations/org2.example.com/users/Admin@org2.example.com/msp

执行如下命令将peer1节点加入通道
```
./peer channel join -b mychannel.block
```

16、安装并实例化chaincode

准备go环境
配置环境变量以连接到某个peer节点上

切换到fabric/bin目录下安装chaincode

./peer chaincode install -n acctflow -v 1.0 -p github.com/hyperledger/fabric/chaincode/go/acctflow

实例化chaincode

./peer chaincode instantiate -o orderer.example.com:7050 --tls true --cafile crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n acctflow -v 1.0 -c '{"Args":["init","a","100","b","200"]}' -P "OR ('Org2MSP.member')"

17、连接到某个peer节点进行chaincode调用

设置环境变量以连接到某个peer节点

进行chaincode写入、查询的调用

./peer chaincode invoke -o orderer.example.com:7050 --tls --cafile crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n acctflow -c '{"Args":["insert","{\"loanNumber\":\"1219171448700375376293400660\",\"happenDate\":\"2017/12/26\",\"currency\":\"156\",\"businessCode\":\"50099\",\"businessDesc\":\" 车款分期\",\"loanFlag\":\"D\",\"businessAmount\":950.00,\"branchNo\":\"\",\"productNo\":\"F02100900001\",\"subject\":\"1301\"}"]}'

./peer chaincode invoke -o orderer.example.com:7050 --tls --cafile crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem -C mychannel -n acctflow -c '{"Args":["query", "1219171448700375376293400660", "2017/12/26"]}'

至此，Fabric远程多节点环境手动部署过程全部结束！

查看图片附件

Hyperledger Fabric远程多节点环境部署过程

1、安装JDK-1.8

2、安装Zookeeper-3.4.12（以机器A为例）

3、安装kafka-2.11-1.0.1.tgz（以机器A为例）

4、安装Docker

5、安装Go

6、安装Fabric Binaries

7、生成msp证书文件

8、生成创始区块和channel定义

9、配置orderer.yaml

10、启动orderer节点

11、导入Fabric chaincode运行环境镜像

12、配置core.yaml

13、启动peer节点

14、Fabric网络初始化-创建通道

15、将peer节点加入通道

16、安装并实例化chaincode

17、连接到某个peer节点进行chaincode调用

Hyperledger Fabric远程多节点环境部署过程

Hyperledger Fabric2.0环境部署和部署过程中的问题解决以及fabric-samples的启动