Python人工智能之sg2im文字转图像

程序员文章站 2022-06-30 23:34:25

【人工智能项目】sg2im文字转图像本次主要对github上的sg2im源码进行执行训练，得到结果。1.从github上下载源码!git clone https://github.com/google...

【人工智能项目】sg2im文字转图像

本次主要对github上的sg2im源码进行执行训练，得到结果。

1.从github上下载源码

!git clone https://github.com/google/sg2im.git

cloning into 'sg2im'...
remote: enumerating objects: 85, done.[k
remote: total 85 (delta 0), reused 0 (delta 0), pack-reused 85[k
unpacking objects: 100% (85/85), done.

! cp -r sg2im/sg2im sg2im/scripts/

!pip install -r sg2im/requirements.txt

collecting cloudpickle==0.5.3
downloading https://files.pythonhosted.org/packages/e7/bf/60ae7ec1e8c6742d2abbb6819c39a48ee796793bcdb7e1d5e41a3e379ddd/cloudpickle-0.5.3-py2.py3-none-any.whl
requirement already satisfied: cycler==0.10.0 in /usr/local/lib/python3.6/dist-packages (from -r sg2im/requirements.txt (line 2)) (0.10.0)
collecting cython==0.28.3
[?25l downloading https://files.pythonhosted.org/packages/6f/79/d8e2cd00bea8156a995fb284ce7b6677c49eccd2d318f73e201a9ce560dc/cython-0.28.3-cp36-cp36m-manylinux1_x86_64.whl (3.4mb)
[k |████████████████████████████████| 3.4mb 8.6mb/s
[?25hcollecting dask==0.17.5
[?25l downloading https://files.pythonhosted.org/packages/91/1a/71be14f468f8f3f94e708afd5662cf75a0ca33a78924ca9f129a9c45c66b/dask-0.17.5-py3-none-any.whl (598kb)
[k |████████████████████████████████| 604kb 30.6mb/s
[?25hcollecting decorator==4.3.0
downloading https://files.pythonhosted.org/packages/bc/bb/a24838832ba35baf52f32ab1a49b906b5f82fb7c76b2f6a7e35e140bac30/decorator-4.3.0-py2.py3-none-any.whl
collecting h5py==2.8.0
[?25l downloading https://files.pythonhosted.org/packages/8e/cb/726134109e7bd71d98d1fcc717ffe051767aac42ede0e7326fd1787e5d64/h5py-2.8.0-cp36-cp36m-manylinux1_x86_64.whl (2.8mb)
[k |████████████████████████████████| 2.8mb 57.5mb/s
[?25hcollecting imageio==2.3.0
[?25l downloading https://files.pythonhosted.org/packages/a7/1d/33c8686072148b3b0fcc12a2e0857dd8316b8ae20a0fa66c8d6a6d01c05c/imageio-2.3.0-py2.py3-none-any.whl (3.3mb)
[k |████████████████████████████████| 3.3mb 59.0mb/s
[?25hcollecting kiwisolver==1.0.1
[?25l downloading https://files.pythonhosted.org/packages/69/a7/88719d132b18300b4369fbffa741841cfd36d1e637e1990f27929945b538/kiwisolver-1.0.1-cp36-cp36m-manylinux1_x86_64.whl (949kb)
[k |████████████████████████████████| 952kb 56.0mb/s
[?25hcollecting matplotlib==2.2.2
[?25l downloading https://files.pythonhosted.org/packages/49/b8/89dbd27f2fb171ce753bb56220d4d4f6dbc5fe32b95d8edc4415782ef07f/matplotlib-2.2.2-cp36-cp36m-manylinux1_x86_64.whl (12.6mb)
[k |████████████████████████████████| 12.6mb 238kb/s
[?25hcollecting networkx==2.1
[?25l downloading https://files.pythonhosted.org/packages/11/42/f951cc6838a4dff6ce57211c4d7f8444809ccbe2134179950301e5c4c83c/networkx-2.1.zip (1.6mb)
[k |████████████████████████████████| 1.6mb 49.4mb/s
[?25hcollecting numpy==1.14.4
[?25l downloading https://files.pythonhosted.org/packages/4b/3d/9c0a34ad8544abef864714840fb8954d630b04433f00881bc8fde7b2ab27/numpy-1.14.4-cp36-cp36m-manylinux1_x86_64.whl (12.2mb)
[k |████████████████████████████████| 12.2mb 149kb/s
[?25hcollecting pillow==5.1.0
[?25l downloading https://files.pythonhosted.org/packages/5f/4b/8b54ab9d37b93998c81b364557dff9f61972c0f650efa0ceaf470b392740/pillow-5.1.0-cp36-cp36m-manylinux1_x86_64.whl (2.0mb)
[k |████████████████████████████████| 2.0mb 53.7mb/s
[?25hcollecting pyparsing==2.2.0
[?25l downloading https://files.pythonhosted.org/packages/6a/8a/718fd7d3458f9fab8e67186b00abdd345b639976bc7fb3ae722e1b026a50/pyparsing-2.2.0-py2.py3-none-any.whl (56kb)
[k |████████████████████████████████| 61kb 9.3mb/s
[?25hcollecting python-dateutil==2.7.3
[?25l downloading https://files.pythonhosted.org/packages/cf/f5/af2b09c957ace60dcfac112b669c45c8c97e32f94aa8b56da4c6d1682825/python_dateutil-2.7.3-py2.py3-none-any.whl (211kb)
[k |████████████████████████████████| 215kb 49.8mb/s
[?25hcollecting pytz==2018.4
[?25l downloading https://files.pythonhosted.org/packages/dc/83/15f7833b70d3e067ca91467ca245bae0f6fe56ddc7451aa0dc5606b120f2/pytz-2018.4-py2.py3-none-any.whl (510kb)
[k |████████████████████████████████| 512kb 56.7mb/s
[?25hcollecting pywavelets==0.5.2
[?25l downloading https://files.pythonhosted.org/packages/32/c0/3646053c0ce297686da524bc968bff6017151a9089d16c33afe7d330a48b/pywavelets-0.5.2-cp36-cp36m-manylinux1_x86_64.whl (5.7mb)
[k |████████████████████████████████| 5.7mb 29.6mb/s
[?25hcollecting scikit-image==0.14.0
[?25l downloading https://files.pythonhosted.org/packages/34/79/cefff573a53ca3fb4c390739d19541b95f371e24d2990aed4cd8837971f0/scikit_image-0.14.0-cp36-cp36m-manylinux1_x86_64.whl (25.3mb)
[k |████████████████████████████████| 25.3mb 115kb/s
[?25hcollecting scipy==1.1.0
[?25l downloading https://files.pythonhosted.org/packages/a8/0b/f163da98d3a01b3e0ef1cab8dd2123c34aee2bafbb1c5bffa354cc8a1730/scipy-1.1.0-cp36-cp36m-manylinux1_x86_64.whl (31.2mb)
[k |████████████████████████████████| 31.2mb 97kb/s
[?25hcollecting six==1.11.0
downloading https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
collecting toolz==0.9.0
[?25l downloading https://files.pythonhosted.org/packages/14/d0/a73c15bbeda3d2e7b381a36afb0d9cd770a9f4adc5d1532691013ba881db/toolz-0.9.0.tar.gz (45kb)
[k |████████████████████████████████| 51kb 8.4mb/s
[?25hcollecting torch==0.4.0
[?25l downloading https://files.pythonhosted.org/packages/69/43/380514bd9663f1bf708abeb359b8b48d3fabb1c8e95bb3427a980a064c57/torch-0.4.0-cp36-cp36m-manylinux1_x86_64.whl (484.0mb)
[k |████████████████████████████████| 484.0mb 33kb/s
[?25hcollecting torchvision==0.2.1
[?25l downloading https://files.pythonhosted.org/packages/ca/0d/f00b2885711e08bd71242ebe7b96561e6f6d01fdb4b9dcf4d37e2e13c5e1/torchvision-0.2.1-py2.py3-none-any.whl (54kb)
[k |████████████████████████████████| 61kb 9.8mb/s
[?25hrequirement already satisfied: setuptools in /usr/local/lib/python3.6/dist-packages (from kiwisolver==1.0.1->-r sg2im/requirements.txt (line 8)) (47.1.1)
building wheels for collected packages: networkx, toolz
building wheel for networkx (setup.py) ... [?25l[?25hdone
created wheel for networkx: filename=networkx-2.1-py2.py3-none-any.whl size=1447765 sha256=4e89cc8350ab7270295c4e879190531eee2b1205e4a7b0c073ed8fe950717a25
stored in directory: /root/.cache/pip/wheels/44/c0/34/6f98693a554301bdb405f8d65d95bbcd3e50180cbfdd98a94e
building wheel for toolz (setup.py) ... [?25l[?25hdone
created wheel for toolz: filename=toolz-0.9.0-cp36-none-any.whl size=53240 sha256=eb0e9434019a90c774ffcbfb077542b8688b43df4895b0c5c57204702dadc064
stored in directory: /root/.cache/pip/wheels/f4/0c/f6/ce6b2d1aa459ee97cc3c0f82236302bd62d89c86c700219463
successfully built networkx toolz
[31merror: xarray 0.15.1 has requirement numpy>=1.15, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: umap-learn 0.4.3 has requirement numpy>=1.17, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: umap-learn 0.4.3 has requirement scipy>=1.3.1, but you'll have scipy 1.1.0 which is incompatible.[0m
[31merror: tifffile 2020.5.30 has requirement numpy>=1.15.1, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: tensorflow 2.2.0 has requirement h5py<2.11.0,>=2.10.0, but you'll have h5py 2.8.0 which is incompatible.[0m
[31merror: tensorflow 2.2.0 has requirement numpy<2.0,>=1.16.0, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: tensorflow 2.2.0 has requirement scipy==1.4.1; python_version >= "3", but you'll have scipy 1.1.0 which is incompatible.[0m
[31merror: tensorflow 2.2.0 has requirement six>=1.12.0, but you'll have six 1.11.0 which is incompatible.[0m
[31merror: tensorflow-probability 0.10.0 has requirement cloudpickle>=1.2.2, but you'll have cloudpickle 0.5.3 which is incompatible.[0m
[31merror: tensorflow-hub 0.8.0 has requirement six>=1.12.0, but you'll have six 1.11.0 which is incompatible.[0m
[31merror: spacy 2.2.4 has requirement numpy>=1.15.0, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: plotnine 0.6.0 has requirement matplotlib>=3.1.1, but you'll have matplotlib 2.2.2 which is incompatible.[0m
[31merror: plotnine 0.6.0 has requirement numpy>=1.16.0, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: plotnine 0.6.0 has requirement scipy>=1.2.0, but you'll have scipy 1.1.0 which is incompatible.[0m
[31merror: numba 0.48.0 has requirement numpy>=1.15, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: mizani 0.6.0 has requirement matplotlib>=3.1.1, but you'll have matplotlib 2.2.2 which is incompatible.[0m
[31merror: imgaug 0.2.9 has requirement numpy>=1.15.0, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: gym 0.17.2 has requirement cloudpickle<1.4.0,>=1.2.0, but you'll have cloudpickle 0.5.3 which is incompatible.[0m
[31merror: google-colab 1.0.0 has requirement six~=1.12.0, but you'll have six 1.11.0 which is incompatible.[0m
[31merror: featuretools 0.4.1 has requirement dask>=0.19.4, but you'll have dask 0.17.5 which is incompatible.[0m
[31merror: fbprophet 0.6 has requirement python-dateutil>=2.8.0, but you'll have python-dateutil 2.7.3 which is incompatible.[0m
[31merror: fastai 1.0.61 has requirement numpy>=1.15, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: fastai 1.0.61 has requirement torch>=1.0.0, but you'll have torch 0.4.0 which is incompatible.[0m
[31merror: distributed 1.25.3 has requirement dask>=0.18.0, but you'll have dask 0.17.5 which is incompatible.[0m
[31merror: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.[0m
[31merror: cvxpy 1.0.31 has requirement numpy>=1.15, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: blis 0.4.1 has requirement numpy>=1.15.0, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: astropy 4.0.1.post1 has requirement numpy>=1.16, but you'll have numpy 1.14.4 which is incompatible.[0m
[31merror: albumentations 0.1.12 has requirement imgaug<0.2.7,>=0.2.5, but you'll have imgaug 0.2.9 which is incompatible.[0m
installing collected packages: cloudpickle, cython, dask, decorator, six, numpy, h5py, pillow, imageio, kiwisolver, python-dateutil, pytz, pyparsing, matplotlib, networkx, pywavelets, scipy, scikit-image, toolz, torch, torchvision
found existing installation: cloudpickle 1.3.0
uninstalling cloudpickle-1.3.0:
successfully uninstalled cloudpickle-1.3.0
found existing installation: cython 0.29.19
uninstalling cython-0.29.19:
successfully uninstalled cython-0.29.19
found existing installation: dask 2.12.0
uninstalling dask-2.12.0:
successfully uninstalled dask-2.12.0
found existing installation: decorator 4.4.2
uninstalling decorator-4.4.2:
successfully uninstalled decorator-4.4.2
found existing installation: six 1.12.0
uninstalling six-1.12.0:
successfully uninstalled six-1.12.0
found existing installation: numpy 1.18.4
uninstalling numpy-1.18.4:
successfully uninstalled numpy-1.18.4
found existing installation: h5py 2.10.0
uninstalling h5py-2.10.0:
successfully uninstalled h5py-2.10.0
found existing installation: pillow 7.0.0
uninstalling pillow-7.0.0:
successfully uninstalled pillow-7.0.0
found existing installation: imageio 2.4.1
uninstalling imageio-2.4.1:
successfully uninstalled imageio-2.4.1
found existing installation: kiwisolver 1.2.0
uninstalling kiwisolver-1.2.0:
successfully uninstalled kiwisolver-1.2.0
found existing installation: python-dateutil 2.8.1
uninstalling python-dateutil-2.8.1:
successfully uninstalled python-dateutil-2.8.1
found existing installation: pytz 2018.9
uninstalling pytz-2018.9:
successfully uninstalled pytz-2018.9
found existing installation: pyparsing 2.4.7
uninstalling pyparsing-2.4.7:
successfully uninstalled pyparsing-2.4.7
found existing installation: matplotlib 3.2.1
uninstalling matplotlib-3.2.1:
successfully uninstalled matplotlib-3.2.1
found existing installation: networkx 2.4
uninstalling networkx-2.4:
successfully uninstalled networkx-2.4
found existing installation: pywavelets 1.1.1
uninstalling pywavelets-1.1.1:
successfully uninstalled pywavelets-1.1.1
found existing installation: scipy 1.4.1
uninstalling scipy-1.4.1:
successfully uninstalled scipy-1.4.1
found existing installation: scikit-image 0.16.2
uninstalling scikit-image-0.16.2:
successfully uninstalled scikit-image-0.16.2
found existing installation: toolz 0.10.0
uninstalling toolz-0.10.0:
successfully uninstalled toolz-0.10.0
found existing installation: torch 1.5.0+cu101
uninstalling torch-1.5.0+cu101:
successfully uninstalled torch-1.5.0+cu101
found existing installation: torchvision 0.6.0+cu101
uninstalling torchvision-0.6.0+cu101:
successfully uninstalled torchvision-0.6.0+cu101
successfully installed cython-0.28.3 pillow-5.1.0 pywavelets-0.5.2 cloudpickle-0.5.3 dask-0.17.5 decorator-4.3.0 h5py-2.8.0 imageio-2.3.0 kiwisolver-1.0.1 matplotlib-2.2.2 networkx-2.1 numpy-1.14.4 pyparsing-2.2.0 python-dateutil-2.7.3 pytz-2018.4 scikit-image-0.14.0 scipy-1.1.0 six-1.11.0 toolz-0.9.0 torch-0.4.0 torchvision-0.2.1

!bash sg2im/scripts/download_models.sh

--2020-06-05 08:11:22-- https://storage.googleapis.com/sg2im-data/small/coco64.pt
resolving storage.googleapis.com (storage.googleapis.com)... 173.194.79.128, 2a00:1450:4013:c05::80
connecting to storage.googleapis.com (storage.googleapis.com)|173.194.79.128|:443... connected.
http request sent, awaiting response... 200 ok
length: 119806264 (114m) [application/octet-stream]
saving to: ‘sg2im-models/coco64.pt'

sg2im-models/coco64 100%[===================>] 114.26m 38.5mb/s in 3.0s

2020-06-05 08:11:25 (38.5 mb/s) - ‘sg2im-models/coco64.pt' saved [119806264/119806264]

--2020-06-05 08:11:25-- https://storage.googleapis.com/sg2im-data/small/vg64.pt
resolving storage.googleapis.com (storage.googleapis.com)... 108.177.119.128, 2a00:1450:4013:c00::80
connecting to storage.googleapis.com (storage.googleapis.com)|108.177.119.128|:443... connected.
http request sent, awaiting response... 200 ok
length: 119873465 (114m) [application/octet-stream]
saving to: ‘sg2im-models/vg64.pt'

sg2im-models/vg64.p 100%[===================>] 114.32m 44.0mb/s in 2.6s

2020-06-05 08:11:29 (44.0 mb/s) - ‘sg2im-models/vg64.pt' saved [119873465/119873465]

--2020-06-05 08:11:29-- https://storage.googleapis.com/sg2im-data/small/vg128.pt
resolving storage.googleapis.com (storage.googleapis.com)... 74.125.128.128, 2a00:1450:4013:c02::80
connecting to storage.googleapis.com (storage.googleapis.com)|74.125.128.128|:443... connected.
http request sent, awaiting response... 200 ok
length: 129319241 (123m) [application/octet-stream]
saving to: ‘sg2im-models/vg128.pt'

sg2im-models/vg128. 100%[===================>] 123.33m 54.2mb/s in 2.3s

2020-06-05 08:11:32 (54.2 mb/s) - ‘sg2im-models/vg128.pt' saved [129319241/129319241]

2.训练与结果展示

!python3 sg2im/scripts/run_model.py --checkpoint sg2im-models/vg128.pt --scene_graphs sg2im/scene_graphs/figure_6_sheep.json --output_dir outputs

import matplotlib.pyplot as plt
import cv2
%matplotlib inline

img0 = cv2.imread("outputs/img000000.png")
img1 = cv2.imread("outputs/img000001.png")
img2 = cv2.imread("outputs/img000002.png")
img3 = cv2.imread("outputs/img000003.png")
img4 = cv2.imread("outputs/img000004.png")
img5 = cv2.imread("outputs/img000005.png")
img6 = cv2.imread("outputs/img000006.png")


plt.figure()
plt.subplot(3,3,1)
plt.imshow(img0)
plt.subplot(3,3,2)
plt.imshow(img1)
plt.subplot(3,3,3)
plt.imshow(img2)
plt.subplot(3,3,4)
plt.imshow(img3)
plt.subplot(3,3,5)
plt.imshow(img4)
plt.subplot(3,3,6)
plt.imshow(img5)
plt.subplot(3,3,7)
plt.imshow(img6)

<matplotlib.image.axesimage at 0x7fa2bdfb36d8>

Python人工智能之sg2im文字转图像

!python3 sg2im/scripts/run_model.py --checkpoint sg2im-models/vg128.pt --scene_graphs sg2im/scene_graphs/figure_6_street.json --output_dir outputs

import matplotlib.pyplot as plt
import cv2
%matplotlib inline

img0 = cv2.imread("outputs/img000000.png")
img1 = cv2.imread("outputs/img000001.png")
img2 = cv2.imread("outputs/img000002.png")
img3 = cv2.imread("outputs/img000003.png")
img4 = cv2.imread("outputs/img000004.png")
img5 = cv2.imread("outputs/img000005.png")
img6 = cv2.imread("outputs/img000006.png")


plt.figure()
plt.subplot(3,3,1)
plt.imshow(img0)
plt.subplot(3,3,2)
plt.imshow(img1)
plt.subplot(3,3,3)
plt.imshow(img2)
plt.subplot(3,3,4)
plt.imshow(img3)
plt.subplot(3,3,5)
plt.imshow(img4)
plt.subplot(3,3,6)
plt.imshow(img5)
plt.subplot(3,3,7)
plt.imshow(img6)

<matplotlib.image.axesimage at 0x7fa2be14d1d0>

Python人工智能之sg2im文字转图像

!python3 sg2im/scripts/run_model.py --checkpoint sg2im-models/vg128.pt --scene_graphs sg2im/scene_graphs/figure_5_vg.json --output_dir outputs

import matplotlib.pyplot as plt
import cv2
%matplotlib inline

img0 = cv2.imread("outputs/img000000.png")
img1 = cv2.imread("outputs/img000001.png")
img2 = cv2.imread("outputs/img000002.png")
img3 = cv2.imread("outputs/img000003.png")
img4 = cv2.imread("outputs/img000004.png")
img5 = cv2.imread("outputs/img000005.png")
img6 = cv2.imread("outputs/img000006.png")
img7 = cv2.imread("outputs/img000007.png")

plt.figure()
plt.subplot(3,3,1)
plt.imshow(img0)
plt.subplot(3,3,2)
plt.imshow(img1)
plt.subplot(3,3,3)
plt.imshow(img2)
plt.subplot(3,3,4)
plt.imshow(img3)
plt.subplot(3,3,5)
plt.imshow(img4)
plt.subplot(3,3,6)
plt.imshow(img5)
plt.subplot(3,3,7)
plt.imshow(img6)
plt.subplot(3,3,8)
plt.imshow(img7)