Docker and VirtualBox(1)Set up Shared Disk for Virtual Box
程序员文章站
2022-03-30 19:23:52
...
Docker and VirtualBox(1)Set up Shared Disk for Virtual Box
I have a lot of things on my VirtualBox. But I am think to change the way I deploy, I should use docker more that I thought.
I plan to share the disk across some virtual machines and deploy only docker images.
Here is a list of what I have on these machines.
ubuntu-master
1. Spark
cd /opt/spark
sbin/start-all.sh
visit the page
http://ubuntu-master:8080/
2. opscenter
cd /opt/opscenter
bin/opscenter -f #start in the front ground
bin/opscenter #start in the back ground
visit the page
http://ubuntu-master:8888/opscenter/
3. tomcat
cd /opt/tomcat
bin/startup.sh
visit the page
http://ubuntu-master:8081/bldinfo.txt
http://ubuntu-master:8081/LMMAdmin
4. kafka Monitor
cd /opt/kafka-monitor
nohup bin/startup.sh &
visit the page
http://ubuntu-master:8082/
5. memcached Monitor
memcache-top --sleep 3 --instance ubuntu-dev1,ubuntu-dev2
6. Switch Java Version
sudo update-alternatives --config java
sudo update-alternatives --config javac
7. Hadoop hdfs and yarn
cd /opt/hadoop
sbin/start-dfs.sh
sbin/start-yarn.sh
visit the page
http://ubuntu-master:50070/dfshealth.html#tab-overview
http://ubuntu-master:8088/cluster
8. Haproxy
cd /opt/haproxy
sudo sbin/haproxy -f conf/haproxy.conf
Visit the page
http://ubuntu-master/haproxy-status
9. Mysql cluster database
mysql -u cluster -pcluster
10. hbase
cd /opt/hbase
bin/start-hbase.sh
visit the page
http://ubuntu-master:16030/master-status
11. solr
cd /opt/solr/example
java -jar start.jar
Visit Page
http://ubuntu-master:8983/solr/#/jobs
ubuntu-dev1
1. cassandra
cd /opt/cassandra
nohup cassandra -f conf/cassandra.yaml &
2. zookeeper
cd /opt/zookeeper
zkServer.sh start zoo-dev1.cfg
3. kafka
cd /opt/kafka
nohup bin/kafka-server-start.sh config/server-dev1.properties &
4. rabbitmq
cd /opt/rabbitmq
sudo RABBITMQ_NODE_PORT=5672 RABBITMQ_SERVER_START_ARGS="-rabbitmq_management listener [{port,15672}]" RABBITMQ_NODENAME=rabbit1 sbin/rabbitmq-server -detached
visit page
http://ubuntu-dev1:15672 sillycat/kaishi
5. memcached
cd /opt/memcached
memcached -d -m 50 -p 11211 -u carl -l 0.0.0.0
6. Switch Java Version
sudo update-alternatives --config java
sudo update-alternatives --config javac
ubuntu-dev2
1. cassandra
cd /opt/cassandra
nohup cassandra -f conf/cassandra.yaml &
2. zookeeper
cd /opt/zookeeper
zkServer.sh start zoo-dev2.cfg
3. kafka
cd /opt/kafka
nohup bin/kafka-server-start.sh config/server-dev2.properties &
4. rabbitmq
cd /opt/rabbitmq
sudo RABBITMQ_NODE_PORT=5672 RABBITMQ_SERVER_START_ARGS="-rabbitmq_management listener [{port,15672}]" RABBITMQ_NODENAME=rabbit2 sbin/rabbitmq-server -detached
enable cluster
sudo sbin/rabbitmqctl -n rabbit2 stop_app
sudo sbin/rabbitmqctl -n rabbit2 join_cluster rabbit1@ubuntu-dev1
sudo sbin/rabbitmqctl -n rabbit2 start_app
visit page
http://ubuntu-dev2:15672 sillycat/kaishi
5. memcached
cd /opt/memcached
memcached -d -m 50 -p 11211 -u carl -l 0.0.0.0
6. Switch Java Version
sudo update-alternatives --config java
sudo update-alternatives --config javac
I will export all of them and backup them into my home NAS. I will delete all of them on my local and set up fresh ubuntu LTS for that.
1 Set up Share Driver
I create virtual machine ubuntu-master, ubuntu-dev1, ubuntu-dev2 and all of them have their own driver, and a shared driver with name shared-nas.
> sudo fdisk -l
Disk /dev/sda: 85.9 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000645fe
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 159582207 79790080 83 Linux
/dev/sda2 159584254 167770111 4092929 5 Extended
/dev/sda5 159584256 167770111 4092928 82 Linux swap / Solaris
Disk /dev/sdb: 85.9 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdb doesn't contain a valid partition table
Solution:
> sudo mkfs -t ext3 /dev/sdb
It does not work.
Initiate the disk with following
> sudo fdisk /dev/sdb
>m >n >p >1
default for other information, then >w
It works.
> sudo fdisk -l
Disk /dev/sda: 85.9 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000645fe
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 159582207 79790080 83 Linux
/dev/sda2 159584254 167770111 4092929 5 Extended
/dev/sda5 159584256 167770111 4092928 82 Linux swap / Solaris
Disk /dev/sdb: 85.9 GB, 85899345920 bytes
86 heads, 10 sectors/track, 195083 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x070ebcd0
Device Boot Start End Blocks Id System
/dev/sdb1 2048 167772159 83885056 83 Linux
Format to ext3
> sudo mkfs -t ext3 /dev/sdb1
It works. Then I will configure the docker to use this share driver.
2 Auto Mount the Driver to System
Create the folder
>sudo mkdir /mnt/nas
List the UUID of my disk
> ls -l /dev/disk/by-uuid
total 0
lrwxrwxrwx 1 root root 10 Dec 11 17:45 6624adb3-86bc-4801-8275-063cdfae706b -> ../../sda5
lrwxrwxrwx 1 root root 9 Dec 11 17:45 75595e30-261b-4ee1-a855-4b4f5428d583 -> ../../sdb
lrwxrwxrwx 1 root root 10 Dec 11 17:45 8e864fd6-914f-489f-abf7-85744496c664 -> ../../sda1
lrwxrwxrwx 1 root root 10 Dec 11 17:45 adcfd350-219a-414f-92a6-e175b334d970 -> ../../sdb1
Copy the one of sdb1, edit the file fstab
> sudo vi /etc/fstab
UUID=adcfd350-219a-414f-92a6-e175b334d970 /mnt/nas auto defaults 0 0
Amount the disk
> sudo mount -a
Check the amount
> df -hl /mnt/nas/
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 79G 56M 75G 1% /mnt/nas
3 Install and Set up Docker
> sudo mkdir /mnt/nas/docker
Installation
> sudo curl -sSL https://get.docker.com/ | sh
> sudo usermod -aG docker carl
or
> sudo gpasswd -a carl docker
Change the configuration
> sudo vi /etc/default/docker
DOCKER_OPTS="-g /mnt/nas/docker"
> docker info
Containers: 0
Images: 0
Server Version: 1.9.1
Storage Driver: aufs
Root Dir: /mnt/nas/docker/aufs
Done and try
> docker pull centos:7
> df -hl
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 75G 1.3G 70G 2% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 1.9G 4.0K 1.9G 1% /dev
tmpfs 386M 484K 386M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 1.9G 0 1.9G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sdb1 79G 242M 75G 1% /mnt/nas
File are stored in the shared NAS disk.
The problem is that I try to create a shared vdi and ask all the clients ubuntu system to load the same NAS disk. It does not work. I will try shared folder.
> sudo apt-get install virtualbox-guest-additions-iso
> sudo apt-get install virtualbox-guest-utils
> sudo mount -t vboxsf nas /mnt/nas
It works. But when I start the docker, docker throw exception in logging
> sudo less /var/log/upstart/docker.log
ESC[31mERROESC[0m[0000] [graphdriver] prior storage driver "aufs" failed: invalid argument
ESC[31mFATAESC[0m[0000] Error starting daemon: error initializing graphdriver: invalid argument
> VBoxManage modifymedium --type shareable nas.vdi
It works.
References:
https://help.ubuntu.com/community/InstallingANewHardDrive
http://tutorialforlinux.com/2014/08/01/ubuntu-how-to-automatically-mount-a-ext2-ext3-ext4-linux-partition-on-startup-easy-guide/
https://forums.docker.com/t/how-do-i-change-the-docker-image-installation-directory/1169
http://www.oracledistilled.com/virtualbox/creating-shared-drives-in-oracle-vm-virtualbox/
http://www.giannistsakiris.com/2008/04/09/virtualbox-access-windows-host-shared-folders-from-ubuntu-guest/
http://superuser.com/questions/745868/mount-failed-unknown-file-system-in-virtualbox-in-ubuntu-14-04-and-windows-7-a
http://*.com/questions/30969435/where-is-the-docker-daemon-log
I have a lot of things on my VirtualBox. But I am think to change the way I deploy, I should use docker more that I thought.
I plan to share the disk across some virtual machines and deploy only docker images.
Here is a list of what I have on these machines.
ubuntu-master
1. Spark
cd /opt/spark
sbin/start-all.sh
visit the page
http://ubuntu-master:8080/
2. opscenter
cd /opt/opscenter
bin/opscenter -f #start in the front ground
bin/opscenter #start in the back ground
visit the page
http://ubuntu-master:8888/opscenter/
3. tomcat
cd /opt/tomcat
bin/startup.sh
visit the page
http://ubuntu-master:8081/bldinfo.txt
http://ubuntu-master:8081/LMMAdmin
4. kafka Monitor
cd /opt/kafka-monitor
nohup bin/startup.sh &
visit the page
http://ubuntu-master:8082/
5. memcached Monitor
memcache-top --sleep 3 --instance ubuntu-dev1,ubuntu-dev2
6. Switch Java Version
sudo update-alternatives --config java
sudo update-alternatives --config javac
7. Hadoop hdfs and yarn
cd /opt/hadoop
sbin/start-dfs.sh
sbin/start-yarn.sh
visit the page
http://ubuntu-master:50070/dfshealth.html#tab-overview
http://ubuntu-master:8088/cluster
8. Haproxy
cd /opt/haproxy
sudo sbin/haproxy -f conf/haproxy.conf
Visit the page
http://ubuntu-master/haproxy-status
9. Mysql cluster database
mysql -u cluster -pcluster
10. hbase
cd /opt/hbase
bin/start-hbase.sh
visit the page
http://ubuntu-master:16030/master-status
11. solr
cd /opt/solr/example
java -jar start.jar
Visit Page
http://ubuntu-master:8983/solr/#/jobs
ubuntu-dev1
1. cassandra
cd /opt/cassandra
nohup cassandra -f conf/cassandra.yaml &
2. zookeeper
cd /opt/zookeeper
zkServer.sh start zoo-dev1.cfg
3. kafka
cd /opt/kafka
nohup bin/kafka-server-start.sh config/server-dev1.properties &
4. rabbitmq
cd /opt/rabbitmq
sudo RABBITMQ_NODE_PORT=5672 RABBITMQ_SERVER_START_ARGS="-rabbitmq_management listener [{port,15672}]" RABBITMQ_NODENAME=rabbit1 sbin/rabbitmq-server -detached
visit page
http://ubuntu-dev1:15672 sillycat/kaishi
5. memcached
cd /opt/memcached
memcached -d -m 50 -p 11211 -u carl -l 0.0.0.0
6. Switch Java Version
sudo update-alternatives --config java
sudo update-alternatives --config javac
ubuntu-dev2
1. cassandra
cd /opt/cassandra
nohup cassandra -f conf/cassandra.yaml &
2. zookeeper
cd /opt/zookeeper
zkServer.sh start zoo-dev2.cfg
3. kafka
cd /opt/kafka
nohup bin/kafka-server-start.sh config/server-dev2.properties &
4. rabbitmq
cd /opt/rabbitmq
sudo RABBITMQ_NODE_PORT=5672 RABBITMQ_SERVER_START_ARGS="-rabbitmq_management listener [{port,15672}]" RABBITMQ_NODENAME=rabbit2 sbin/rabbitmq-server -detached
enable cluster
sudo sbin/rabbitmqctl -n rabbit2 stop_app
sudo sbin/rabbitmqctl -n rabbit2 join_cluster rabbit1@ubuntu-dev1
sudo sbin/rabbitmqctl -n rabbit2 start_app
visit page
http://ubuntu-dev2:15672 sillycat/kaishi
5. memcached
cd /opt/memcached
memcached -d -m 50 -p 11211 -u carl -l 0.0.0.0
6. Switch Java Version
sudo update-alternatives --config java
sudo update-alternatives --config javac
I will export all of them and backup them into my home NAS. I will delete all of them on my local and set up fresh ubuntu LTS for that.
1 Set up Share Driver
I create virtual machine ubuntu-master, ubuntu-dev1, ubuntu-dev2 and all of them have their own driver, and a shared driver with name shared-nas.
> sudo fdisk -l
Disk /dev/sda: 85.9 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000645fe
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 159582207 79790080 83 Linux
/dev/sda2 159584254 167770111 4092929 5 Extended
/dev/sda5 159584256 167770111 4092928 82 Linux swap / Solaris
Disk /dev/sdb: 85.9 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdb doesn't contain a valid partition table
Solution:
> sudo mkfs -t ext3 /dev/sdb
It does not work.
Initiate the disk with following
> sudo fdisk /dev/sdb
>m >n >p >1
default for other information, then >w
It works.
> sudo fdisk -l
Disk /dev/sda: 85.9 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000645fe
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 159582207 79790080 83 Linux
/dev/sda2 159584254 167770111 4092929 5 Extended
/dev/sda5 159584256 167770111 4092928 82 Linux swap / Solaris
Disk /dev/sdb: 85.9 GB, 85899345920 bytes
86 heads, 10 sectors/track, 195083 cylinders, total 167772160 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x070ebcd0
Device Boot Start End Blocks Id System
/dev/sdb1 2048 167772159 83885056 83 Linux
Format to ext3
> sudo mkfs -t ext3 /dev/sdb1
It works. Then I will configure the docker to use this share driver.
2 Auto Mount the Driver to System
Create the folder
>sudo mkdir /mnt/nas
List the UUID of my disk
> ls -l /dev/disk/by-uuid
total 0
lrwxrwxrwx 1 root root 10 Dec 11 17:45 6624adb3-86bc-4801-8275-063cdfae706b -> ../../sda5
lrwxrwxrwx 1 root root 9 Dec 11 17:45 75595e30-261b-4ee1-a855-4b4f5428d583 -> ../../sdb
lrwxrwxrwx 1 root root 10 Dec 11 17:45 8e864fd6-914f-489f-abf7-85744496c664 -> ../../sda1
lrwxrwxrwx 1 root root 10 Dec 11 17:45 adcfd350-219a-414f-92a6-e175b334d970 -> ../../sdb1
Copy the one of sdb1, edit the file fstab
> sudo vi /etc/fstab
UUID=adcfd350-219a-414f-92a6-e175b334d970 /mnt/nas auto defaults 0 0
Amount the disk
> sudo mount -a
Check the amount
> df -hl /mnt/nas/
Filesystem Size Used Avail Use% Mounted on
/dev/sdb1 79G 56M 75G 1% /mnt/nas
3 Install and Set up Docker
> sudo mkdir /mnt/nas/docker
Installation
> sudo curl -sSL https://get.docker.com/ | sh
> sudo usermod -aG docker carl
or
> sudo gpasswd -a carl docker
Change the configuration
> sudo vi /etc/default/docker
DOCKER_OPTS="-g /mnt/nas/docker"
> docker info
Containers: 0
Images: 0
Server Version: 1.9.1
Storage Driver: aufs
Root Dir: /mnt/nas/docker/aufs
Done and try
> docker pull centos:7
> df -hl
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 75G 1.3G 70G 2% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 1.9G 4.0K 1.9G 1% /dev
tmpfs 386M 484K 386M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 1.9G 0 1.9G 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sdb1 79G 242M 75G 1% /mnt/nas
File are stored in the shared NAS disk.
The problem is that I try to create a shared vdi and ask all the clients ubuntu system to load the same NAS disk. It does not work. I will try shared folder.
> sudo apt-get install virtualbox-guest-additions-iso
> sudo apt-get install virtualbox-guest-utils
> sudo mount -t vboxsf nas /mnt/nas
It works. But when I start the docker, docker throw exception in logging
> sudo less /var/log/upstart/docker.log
ESC[31mERROESC[0m[0000] [graphdriver] prior storage driver "aufs" failed: invalid argument
ESC[31mFATAESC[0m[0000] Error starting daemon: error initializing graphdriver: invalid argument
> VBoxManage modifymedium --type shareable nas.vdi
It works.
References:
https://help.ubuntu.com/community/InstallingANewHardDrive
http://tutorialforlinux.com/2014/08/01/ubuntu-how-to-automatically-mount-a-ext2-ext3-ext4-linux-partition-on-startup-easy-guide/
https://forums.docker.com/t/how-do-i-change-the-docker-image-installation-directory/1169
http://www.oracledistilled.com/virtualbox/creating-shared-drives-in-oracle-vm-virtualbox/
http://www.giannistsakiris.com/2008/04/09/virtualbox-access-windows-host-shared-folders-from-ubuntu-guest/
http://superuser.com/questions/745868/mount-failed-unknown-file-system-in-virtualbox-in-ubuntu-14-04-and-windows-7-a
http://*.com/questions/30969435/where-is-the-docker-daemon-log