ARCore Augmented Image+marker传送门

想要的效果就是将marker变成一个窗口，通过这个窗口可以看见里面的虚拟内容

做的东西有点类似于传送门，但与网上常见的AR传送门又有一点不同，网上常见的AR传送门其中“门”的位置通常是任意的，b站上有一个老哥的视频，可以看看AR传送门的效果。

【附源码】Unity+ARcore 实现传送门 小意思VR AR教程

 

我想做的这个“门”是和marker同样大小的，有点像官方的一个演示视频（官方的这个示例里从marker到虚拟物体的过渡做的很流畅）

还有之前看到的一个国外网友做的东西。

效果大概就是这个样子，最终做出的demo：

识别marker

ARCore组件中的Example下有一个名为增强图像（Augmented Image）的示例，展示了ARCore如何识别marker并在上面显示模型。初学记录一下做的过程和理解。

Augmented Image示例

代码主要有两部分AugmentedImageExampleController和AugmentedImageVisualizer。AugmentedImageVisualizer部分是ARCore检测到marker图像之后，负责在其上显示模型的组件。就像HelloAR中显示模型一样。

能够发现在AugmentedVisualizer脚本上留有左下角、右下角、左上角和右上角这四个部分的模型等待绑定。

Prefab目录下的AugmentedImageVisualizer上绑定了同名的script，并为其绑定了对应的四个模型。 在prefab的Inspector中我们看到的如下。

prefab与实际使用时看到的不一样是因为在Visualizer的script中对4部分的位置都分别做了调整。 

 然后在AugmentedImageExampleController中绑定了上面的prefab

初见遇到的坑

这些坑太蠢了，笑笑跳过去吧。

1、换成自己的模型之后，在Visualizer脚本中试图修改模型的缩放和位置，但是没有效果，模型显示的时候还是出现在初始位置且是初始的缩放比例。

最初的示例Augmented Image中，当改变marker的大小时，模型的位置和大小也会发生改变，也就是说，从一开始模型和marker的相对大小就是固定的，而脚本中有修改模型位置的代码。我的问题就在于，这个修改位置的代码不起作用。

是因为我在Inspector上脚本处绑定物体不对。我绑定的是模型文件夹下的，而不是prefab内的模型？二者是不同的，相互独立的，虽然它们大小坐标等等是一样的，但是在场景中显示的是prefab内的模型，我们修改的也应当是prefab的组成部分，也就是说之前修改的统统没有使用。

对！就是这里。

2、关于localscale缩放大小的问题，是我蠢逼了，要么在script中设置，要么在Inspector中设置，不要两头同时做。被自己蠢到了，计算了一晚上找不出到底哪里错了，还搁这研究坐标变换呢....

坐标系

就算不熟悉ARCore、unity的坐标系，通过Vector3的forward、back等6个向量加上prefab中四个模型和Visualizer脚本中的代码，我们也可以推出来坐标系的样子。如下图：

unity中的世界坐标系是左手坐标系，用左手做出一个手枪枪毙的动作，将中指指向掌心方向，大拇指代表X轴，食指代表Y轴，中指代表Z轴。

6个单位向量的xyz值

Vector3.forward  (0,0,1)

Vector3.left  (-1,0,0)

Vector3.up  (0,1,0)

back，right，down分别相反

着色器Shader

网上已经有很多这种“透明物体遮挡实体”的效果了，参考：

https://jingyan.baidu.com/article/9faa7231f98071473c28cb85.html

https://blog.csdn.net/u014361280/article/details/103690369

最近正在看《Unity Shader入门精要》这本，微信读书上就有，对入门算是很友好了。

为了实现透明且同时遮挡后面物体的效果，需要单独创建一个Shader，使用VS打开，修改其代码

Shader "Unlit/MaskShader"
{
    SubShader
    {
        Tags { "Queue" = "Geometry-10"}

		Lighting off

		ZTest LEqual 

		ZWrite On

		ColorMask 0
		Pass{}
	}
}

与传送门的不同

与传送门不同的是，要更改“门”的大小到和marker一致，这个很简单，因为ARCore能够估计marker的宽高并返回值。

public AugmentedImage Image;

可通过Image的ExtentX和ExtentZ属性来获得ARCore对增强图像尺寸的估计

C#中结构体struct的坑

C#中的结构体

localScale.x = 1.0f;

这样是不行的，要修改必须将localScale整个修改，可以搜索关键字“C#结构体”来了解一下这个问题。

localScale = new Vector3(x,y,z);

代码

Visualizer：

//-----------------------------------------------------------------------
// <copyright file="AugmentedImageVisualizer.cs" company="Google LLC">
//
// Copyright 2018 Google LLC. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// </copyright>
//-----------------------------------------------------------------------

namespace GoogleARCore.Examples.AugmentedImage
{
    using System;
    using System.Collections.Generic;
    using System.Runtime.InteropServices;
    using GoogleARCore;
    using GoogleARCoreInternal;
    using UnityEngine;

    /// <summary>
    /// Uses 4 frame corner objects to visualize an AugmentedImage.
    /// </summary>
    public class MyVisualizer : MonoBehaviour
    {
        /// <summary>
        /// The AugmentedImage to visualize.
        /// </summary>
        //增强图像Image
        public AugmentedImage Image;

        //box变成marker的几倍
        //必须为大于等于1的值，最好大于等于2
        public float box_scale = 3.0f;
        //box_depth为box的深度
        //注意box要下降的深度应当为box_depth的一半，同时roof是不需要下降的
        public float box_depth = 3.0f;

        /// <summary>
        /// A model for the lower left corner of the frame to place when an image is detected.
        /// </summary>
        /// 
        public GameObject roof_left;
        public GameObject roof_right;
        public GameObject roof_up;
        public GameObject roof_down;
        public GameObject wall_forward;
        public GameObject wall_back;
        public GameObject wall_left;
        public GameObject wall_right;
        public GameObject wall_bottom;
        public GameObject ball_1;
        public GameObject ball_2;
        public GameObject ball_3;
        public GameObject ball_4;
        public GameObject Mask_roof_left;
        public GameObject Mask_roof_right;
        public GameObject Mask_roof_up;
        public GameObject Mask_roof_down;
        public GameObject Mask_wall_forward;
        public GameObject Mask_wall_back;
        public GameObject Mask_wall_left;
        public GameObject Mask_wall_right;
        public GameObject Mask_wall_bottom;

        /// <summary>
        /// The Unity Update method.
        /// </summary>
        //unity的刷新函数update是从Monobehaviour类中继承来的
        //当检测到增强图像时，这个update函数才会不断调用（因为这个函数的用处就是显示在增强图像上的模型）
        public void Update()
        {
           // UnityEngine.Debug.Log("update MyVisualizer");
            //1、当没有预先设置图像时
            //2、图像检索的状态并不是搜索中
            //将四个模型的Active都设置为False
            if (Image == null || Image.TrackingState != TrackingState.Tracking)
            {
                roof_left.SetActive(false);
                roof_right.SetActive(false);
                roof_up.SetActive(false);
                roof_down.SetActive(false);
                wall_forward.SetActive(false);
                wall_back.SetActive(false);
                wall_left.SetActive(false);
                wall_right.SetActive(false);
                wall_bottom.SetActive(false);
                ball_1.SetActive(false);
                ball_2.SetActive(false);
                ball_3.SetActive(false);
                ball_4.SetActive(false);
                Mask_wall_forward.SetActive(false);
                Mask_wall_back.SetActive(false);
                Mask_wall_left.SetActive(false);
                Mask_wall_right.SetActive(false);
                Mask_wall_bottom.SetActive(false);
                Mask_roof_left.SetActive(false);
                Mask_roof_right.SetActive(false);
                Mask_roof_up.SetActive(false);
                Mask_roof_down.SetActive(false);


                return;
            }

            float roof_thickness = 0.0001f;
            //估计图像（图像是2维的）物理宽度、高度的一半
            float halfWidth = Image.ExtentX / 2;
            float halfHeight = Image.ExtentZ / 2;

            //mask缩放和移动时使用的偏移量
            float offset_for_mask_scale = 0.005f;
            float offset_for_mask_pos = 0.001f;//此处应和墙的厚度一致，不然会出现闪烁
            //临时存储数据
            float temp_for_mask_x;
            float temp_for_mask_z;
           
            Vector3 temp;

            //ball
            float min = Math.Min(halfHeight, halfWidth);

            ball_1.transform.localScale =
                new Vector3(0.01f,0.01f,0.01f);

            temp = ball_2.transform.localScale;
            temp.x = min;
            temp.y = min ;
            temp.z = min ;
            ball_2.transform.localScale =
                new Vector3(temp.x, temp.y, temp.z);

            temp = ball_3.transform.localScale;
            temp.x = min ;
            temp.y = min ;
            temp.z = min ;
            ball_3.transform.localScale =
                new Vector3(temp.x, temp.y, temp.z);

            temp = ball_4.transform.localScale;
            temp.x = min ;
            temp.y = min ;
            temp.z = min ;
            ball_4.transform.localScale =
                new Vector3(temp.x,temp.y,temp.z);
            
            
            //roof_left和right的缩放
            temp = roof_left.transform.localScale;
            temp.x = halfWidth * (box_scale - 1);
            temp.z = halfHeight * 2 * box_scale;
            temp_for_mask_x = temp.x + offset_for_mask_scale;
            temp_for_mask_z = temp.z + offset_for_mask_scale;
            roof_left.transform.localScale = new Vector3(temp.x, roof_thickness, temp.z);
            roof_right.transform.localScale = new Vector3(temp.x, roof_thickness, temp.z);
            Mask_roof_left.transform.localScale = new Vector3(temp_for_mask_x, roof_thickness, temp_for_mask_z);
            Mask_roof_right.transform.localScale = new Vector3(temp_for_mask_x, roof_thickness, temp_for_mask_z);
            //roof up和down的缩放
            temp = roof_up.transform.localScale;
            temp.x = halfWidth * 2;
            temp.z = halfHeight *(box_scale-1);
            temp_for_mask_x = temp.x + offset_for_mask_scale;
            temp_for_mask_z = temp.z + offset_for_mask_scale;
            roof_up.transform.localScale = new Vector3(temp.x, roof_thickness, temp.z);
            roof_down.transform.localScale = new Vector3(temp.x, roof_thickness, temp.z);
            Mask_roof_up.transform.localScale = new Vector3(temp_for_mask_x, roof_thickness, temp_for_mask_z);
            Mask_roof_down.transform.localScale = new Vector3(temp_for_mask_x, roof_thickness, temp_for_mask_z);

            //forward和back的缩放
            temp = wall_forward.transform.localScale;
            temp.x = halfWidth * 2* box_scale;
            temp_for_mask_x = temp.x + offset_for_mask_scale;
            temp.y = box_depth;
            wall_forward.transform.localScale = new Vector3(temp.x, temp.y, temp.z);
            wall_back.transform.localScale = new Vector3(temp.x, temp.y, temp.z);
            Mask_wall_forward.transform.localScale = new Vector3(temp_for_mask_x, temp.y, temp.z);
            Mask_wall_back.transform.localScale = new Vector3(temp_for_mask_x, temp.y, temp.z);

            //left和right的缩放
            temp = wall_left.transform.localScale;
            temp.z = halfHeight * 2* box_scale;
            temp_for_mask_z = temp.z + offset_for_mask_scale;
            temp.y = box_depth;
            wall_left.transform.localScale = new Vector3(temp.x, temp.y, temp.z);
            wall_right.transform.localScale = new Vector3(temp.x, temp.y, temp.z);
            Mask_wall_left.transform.localScale = new Vector3(temp.x, temp.y, temp_for_mask_z);
            Mask_wall_right.transform.localScale = new Vector3(temp.x, temp.y, temp_for_mask_z);
            //bottom的缩放
            temp = wall_bottom.transform.localScale;
            temp.x = halfWidth * 2* box_scale;
            temp.z = halfHeight * 2* box_scale;
            temp.y = box_depth;
            wall_bottom.transform.localScale = new Vector3(temp.x, roof_thickness, temp.z);
            Mask_wall_bottom.transform.localScale = new Vector3(temp_for_mask_x, roof_thickness, temp_for_mask_z);

            ball_1.transform.localPosition =
                  (box_depth / 2 - ball_2.transform.localScale.y / 2) * Vector3.down;
            ball_2.transform.localPosition = 
                -halfWidth*box_depth*5/2*Vector3.left + (box_depth/2 - ball_2.transform.localScale.y/2)*Vector3.down + halfHeight*box_depth*5/2*Vector3.forward;
            ball_3.transform.localPosition =
                halfWidth*box_depth*5/2* Vector3.left + (box_depth / 2 - ball_2.transform.localScale.y / 2) * Vector3.down + halfHeight*box_depth*5/2 * Vector3.back;
            ball_4.transform.localPosition =
                 (box_depth / 2 - ball_2.transform.localScale.y / 2) * Vector3.down;
            // (halfHeight * (box_scale + 1) / 2 + offset_for_mask_pos) * Vector3.forward;
            //修改roof的位置
            roof_up.transform.localPosition =
                (halfHeight * (box_scale + 1) / 2 + offset_for_mask_pos) * Vector3.forward;
            roof_down.transform.localPosition =
                 (halfHeight * (box_scale + 1) / 2 + offset_for_mask_pos) * Vector3.back;
            roof_left.transform.localPosition =
                 (halfWidth * (box_scale + 1)/2 + offset_for_mask_pos)*Vector3.left;
            roof_right.transform.localPosition =
                 (halfWidth * (box_scale + 1)/2 + offset_for_mask_pos) * Vector3.right;
            //修改roof mask的位置
            Mask_roof_up.transform.localPosition =
                (offset_for_mask_pos * Vector3.up) + halfHeight * (box_scale + 1) / 2 * Vector3.forward;
            Mask_roof_down.transform.localPosition =
                (offset_for_mask_pos * Vector3.up) + halfHeight * (box_scale + 1) / 2 * Vector3.back;
            Mask_roof_left.transform.localPosition =
                (offset_for_mask_pos * Vector3.up) + halfWidth * (box_scale + 1) / 2 * Vector3.left;
            Mask_roof_right.transform.localPosition =
                (offset_for_mask_pos * Vector3.up) + halfWidth * (box_scale + 1) / 2 * Vector3.right;
            //修改位置，让这个box“下沉”，陷入到marker里面。
            wall_forward.transform.localPosition =
                 (box_depth/2 * Vector3.down) + (box_scale*halfHeight * Vector3.forward);
            wall_back.transform.localPosition =
                (box_depth/2 * Vector3.down) + (box_scale * halfHeight * Vector3.back);
            wall_left.transform.localPosition =
                (box_depth/2 * Vector3.down) + (box_scale * halfWidth * Vector3.left);
            wall_right.transform.localPosition =
                (box_depth/2 * Vector3.down) + (box_scale * halfWidth * Vector3.right);
            wall_bottom.transform.localPosition = box_depth/2 * Vector3.down;
            //修改mask的位置，保证能够覆盖墙面的同时，还要不与墙面重叠（否则会有闪烁）
            Mask_wall_forward.transform.localPosition =
                 (box_depth / 2 * Vector3.down) + ((box_scale * halfHeight + offset_for_mask_pos) * Vector3.forward);
            Mask_wall_back.transform.localPosition =
                (box_depth / 2 * Vector3.down) + ((box_scale * halfHeight + offset_for_mask_pos) * Vector3.back);
            Mask_wall_left.transform.localPosition =
                (box_depth / 2 * Vector3.down) + ((box_scale * halfWidth + offset_for_mask_pos) * Vector3.left);
            Mask_wall_right.transform.localPosition =
                (box_depth / 2 * Vector3.down) + ((box_scale * halfWidth + offset_for_mask_pos) * Vector3.right);
            Mask_wall_bottom.transform.localPosition = (box_depth+ offset_for_mask_pos) * Vector3.down;

            roof_left.SetActive(true);
            roof_right.SetActive(true);
            roof_up.SetActive(true);
            roof_down.SetActive(true);
            Mask_roof_left.SetActive(true);
            Mask_roof_right.SetActive(true);
            Mask_roof_up.SetActive(true);
            Mask_roof_down.SetActive(true);

            wall_forward.SetActive(true);
            wall_back.SetActive(true);
            wall_left.SetActive(true);
            wall_right.SetActive(true);
            wall_bottom.SetActive(true);
            ball_1.SetActive(true);
            ball_2.SetActive(true);
            ball_3.SetActive(true);
            ball_4.SetActive(true);
            Mask_wall_forward.SetActive(true);
            Mask_wall_back.SetActive(true);
            Mask_wall_left.SetActive(true);
            Mask_wall_right.SetActive(true);
            Mask_wall_bottom.SetActive(true);

        }
    }
}

 Controller：

//-----------------------------------------------------------------------
// <copyright file="AugmentedImageExampleController.cs" company="Google LLC">
//
// Copyright 2018 Google LLC. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
// </copyright>
//-----------------------------------------------------------------------

namespace GoogleARCore.Examples.AugmentedImage
{
    using System.Collections.Generic;
    using System.Runtime.InteropServices;
    using GoogleARCore;
    using UnityEngine;
    using UnityEngine.UI;

    /// <summary>
    /// Controller for AugmentedImage example.
    /// </summary>
    /// <remarks>
    /// In this sample, we assume all images are static or moving slowly with
    /// a large occupation of the screen. If the target is actively moving,
    /// we recommend to check <see cref="AugmentedImage.TrackingMethod"/> and
    /// render only when the tracking method equals to
    /// <see cref="AugmentedImageTrackingMethod"/>.<c>FullTracking</c>.
    /// See details in <a href="https://developers.google.com/ar/develop/c/augmented-images/">
    /// Recognize and Augment Images</a>
    /// </remarks>
    /// 一个增强图像示例类
    public class MyController : MonoBehaviour
    {
        /// <summary>
        /// A prefab for visualizing an AugmentedImage.
        /// </summary>
        // 隔壁的MyVisualizer类创建一个预制体AugmentedImageVisualizerPrefab
        public MyVisualizer AugmentedImageVisualizerPrefab;

        /// <summary>
        /// The overlay containing the fit to scan user guide.
        /// </summary>
        // 用来指引用户的覆盖层
        public GameObject FitToScanOverlay;

        //创建一个私有的字典对象，int为索引，图像为内容
        //并对每个都初始化 每一个对应一张图片
        private Dictionary<int, MyVisualizer> m_Visualizers
            = new Dictionary<int, MyVisualizer>();

        private List<AugmentedImage> m_TempAugmentedImages = new List<AugmentedImage>();

        /// <summary>
        /// The Unity Awake() method.
        /// </summary>
        public void Awake()
        {
            UnityEngine.Debug.Log("example awake");
            // Enable ARCore to target 60fps camera capture frame rate on supported devices.
            // Note, Application.targetFrameRate is ignored when QualitySettings.vSyncCount != 0.
            //垂直同步不为0时，目标帧率是无效的
            Application.targetFrameRate = 60;
        }

        /// <summary>
        /// The Unity Update method.
        /// </summary>
        public void Update()
        {
            //UnityEngine.Debug.Log("update example");
            // Exit the app when the 'back' button is pressed.
            //检查是否退出
            if (Input.GetKey(KeyCode.Escape))
            {
                Application.Quit();
            }

            // Only allow the screen to sleep when not tracking.
            //当不追踪图像时允许屏幕睡眠
            //一共有几种Status？
            if (Session.Status != SessionStatus.Tracking)
            {
                Screen.sleepTimeout = SleepTimeout.SystemSetting;
            }
            else
            {
                Screen.sleepTimeout = SleepTimeout.NeverSleep;
            }

            // Get updated augmented images for this frame.
            //GetTrackable函数返回可追踪对象，存储到列表m_TempAugmentedImage
            //类型为增强图像AugmentedImage，还可以是其他的类型（比如增强面部，检测的平面，特征点）
            //使用TrackableQueryFilter.Updated对返回值进行过滤
            Session.GetTrackables<AugmentedImage>(
                m_TempAugmentedImages, TrackableQueryFilter.Updated);

            // Create visualizers and anchors for updated augmented images that are tracking and do
            //为每一个正在追踪的增强图像创建可视化工具和锚点
            // not previously have a visualizer. Remove visualizers for stopped images.
            //
            foreach (var image in m_TempAugmentedImages)
            {
                MyVisualizer visualizer = null;
                m_Visualizers.TryGetValue(image.DatabaseIndex, out visualizer);

                //TrackingState是枚举类型吗
                if (image.TrackingState == TrackingState.Tracking && visualizer == null)
                {
                    // Create an anchor to ensure that ARCore keeps tracking this augmented image.
                    //创建锚点来使ARCore持续追踪增强图像marker
                    //在图像的中心创建锚点
                    Anchor anchor = image.CreateAnchor(image.CenterPose);
                    //强制类型转换为marker可视化对象
                    //使用之前创建的预制体+锚点的方向
                    //Instantiate实例化，这是unity的函数，将一个对象及其子物体完全复制，生成一个新的对象（包括坐标），第一个参数是被复制的对象，第二个参数就是为其赋值新的坐标
                    //将预制体进行复制，坐标为每个图形锚点的坐标
                    visualizer = (MyVisualizer)Instantiate(
                        AugmentedImageVisualizerPrefab, anchor.transform);
                    //将增强图像传给可视化组件
                    visualizer.Image = image;
                    //将可视化组件添加到 字典 m_Visualizers中存储起来
                    m_Visualizers.Add(image.DatabaseIndex, visualizer);
                }
                else if (image.TrackingState == TrackingState.Stopped && visualizer != null)
                {
                    //如果不再追踪增强图像，则将其从字典中移除。并且释放对象
                    m_Visualizers.Remove(image.DatabaseIndex);
                    GameObject.Destroy(visualizer.gameObject);
                }
            }

            // Show the fit-to-scan overlay if there are no images that are Tracking.
            foreach (var visualizer in m_Visualizers.Values)
            {
                if (visualizer.Image.TrackingState == TrackingState.Tracking)
                {
                    FitToScanOverlay.SetActive(false);
                    return;
                }
            }

            FitToScanOverlay.SetActive(true);
        }
    }
}

最后

感觉用pad来显示marker，因为有屏幕反光的原因，会有一定的抖动。改天把marker打印出来试试抖动会不会小一些。打印出的marker是黑白的也没有关系。

检测仅基于高对比度的点，所以彩色和黑白图像都会被检测到，无论使用彩色还是黑白参考图像。

很简单的东西，因为自己太蠢踩了很多坑，其中大多是因为不熟悉unity和C#绕的弯路，感觉可以做点有意思的小玩意。

本文地址：https://blog.csdn.net/Vikanill/article/details/106955314