欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

OpenGL ES for Android 相机预览

程序员文章站 2023-12-31 15:34:58
...

OpenGL ES for Android 相机预览

权限

Android上打开摄像头需要camera权限,在Android 6.0及以上的版本需要动态申请权限,在AndroidManifest.xml中添加camera权限:

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    package="com.arvr.sample">

    <uses-permission android:name="android.permission.CAMERA"/>

    <application>
		...
	</application>

<uses-permission android:name="android.permission.CAMERA"/> 是camera权限

动态申请camera权限代码如下:

class CameraActivity : AppCompatActivity(), SurfaceTexture.OnFrameAvailableListener {

    private lateinit var mRenderer: MyRenderer

    override fun onFrameAvailable(surfaceTexture: SurfaceTexture?) {
        glSurfaceView.queueEvent {
            surfaceTexture?.updateTexImage()
            glSurfaceView.requestRender()
        }
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.surface)
        glSurfaceView.setEGLContextClientVersion(2)
        mRenderer = MyRenderer(context = baseContext, listener = this)
        glSurfaceView.setRenderer(mRenderer)
        glSurfaceView.renderMode = GLSurfaceView.RENDERMODE_WHEN_DIRTY

        if (ContextCompat.checkSelfPermission(
                this,
                Manifest.permission.CAMERA
            ) != PackageManager.PERMISSION_GRANTED
        ) {
            //没有权限
            ActivityCompat.requestPermissions(this, arrayOf(Manifest.permission.CAMERA), 100)
        } else {
            mRenderer.cameraPermission = true
            mRenderer.startCamera()
        }

    }

    override fun onRequestPermissionsResult(
        requestCode: Int,
        permissions: Array<out String>,
        grantResults: IntArray
    ) {
        super.onRequestPermissionsResult(requestCode, permissions, grantResults)
        if (requestCode == 100 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
            mRenderer.cameraPermission = true
            mRenderer.startCamera()
        }
    }

    override fun onResume() {
        super.onResume()
        glSurfaceView.onResume()
    }

    override fun onPause() {
        super.onPause()
        glSurfaceView.onPause()
    }
}

在onCreate中先判断是否有camera权限,如果没有则申请权限权限 , 如果有则打开camera。弹出权限申请对话框,用户点击是否允许,不管是同意还是拒绝都会回调onRequestPermissionsResult方法,用户点击同意后打开camera,和已经有权限的操作是一样的。

创建program并获取参数句柄

顶点shader代码如下:

attribute vec4 a_Position;
attribute vec2 a_TexCoordinate;
varying vec2 v_TexCoord;

void main()
{
    v_TexCoord = a_TexCoordinate;
    gl_Position = a_Position;
}

片段shader代码如下:

#extension GL_OES_EGL_image_external : require
precision mediump float;

uniform samplerExternalOES u_Texture;
varying vec2 v_TexCoord;

void main()
{
    gl_FragColor = texture2D(u_Texture, v_TexCoord);
}
注意:顶点和片段shader是单独的文件,分别是camera_vs.glsl和camera_fs.glsl,存放于assets/glsl目录下。

onSurfaceCreated回调中创建program并获取参数句柄,创建纹理,代码如下:

 override fun onSurfaceCreated(p0: GL10?, p1: EGLConfig?) {
            createProgram()
            //获取vPosition索引
            vPositionLoc = GLES20.glGetAttribLocation(mProgramHandle, "a_Position")
            texCoordLoc = GLES20.glGetAttribLocation(mProgramHandle, "a_TexCoordinate")
            textureLoc = GLES20.glGetUniformLocation(mProgramHandle, "u_Texture")

            textureId = GLTools.createOESTextureId()
            surfaceTexture = SurfaceTexture(textureId)
            surfaceTexture?.setOnFrameAvailableListener(listener)

        }

private fun createProgram() {
            var vertexCode =
                AssetsUtils.readAssetsTxt(
                    context = context,
                    filePath = "glsl/camera_vs.glsl"
                )
            var fragmentCode =
                AssetsUtils.readAssetsTxt(
                    context = context,
                    filePath = "glsl/camera_fs.glsl"
                )
            mProgramHandle = GLTools.createAndLinkProgram(vertexCode, fragmentCode)
        }

fun createOESTextureId(): Int {
        val textures = IntArray(1)
        GLES20.glGenTextures(1, textures, 0)
        glCheck("texture generate")
        GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textures[0])
        glCheck("texture bind")

        GLES20.glTexParameterf(
            GLES11Ext.GL_TEXTURE_EXTERNAL_OES,
            GLES20.GL_TEXTURE_MIN_FILTER,
            GLES20.GL_LINEAR.toFloat()
        )
        GLES20.glTexParameterf(
            GLES11Ext.GL_TEXTURE_EXTERNAL_OES,
            GLES20.GL_TEXTURE_MAG_FILTER,
            GLES20.GL_LINEAR.toFloat()
        )
        GLES20.glTexParameteri(
            GLES11Ext.GL_TEXTURE_EXTERNAL_OES,
            GLES20.GL_TEXTURE_WRAP_S,
            GLES20.GL_CLAMP_TO_EDGE
        )
        GLES20.glTexParameteri(
            GLES11Ext.GL_TEXTURE_EXTERNAL_OES,
            GLES20.GL_TEXTURE_WRAP_T,
            GLES20.GL_CLAMP_TO_EDGE
        )

        return textures[0]
    }

GLTools 为工具类,createOESTextureId方法是其中一个方法,创建一个OES纹理,OES纹理用于渲染相机、视频。创建纹理id并创建SurfaceTexture,SurfaceTexture在打开相机方法中用到,用于预览相机。setOnFrameAvailableListener的回调是从Activity中传入,真正的实现在Activity中,

class CameraActivity : AppCompatActivity(), SurfaceTexture.OnFrameAvailableListener {

    private lateinit var mRenderer: MyRenderer

    override fun onFrameAvailable(surfaceTexture: SurfaceTexture?) {
        glSurfaceView.queueEvent {
            surfaceTexture?.updateTexImage()
            glSurfaceView.requestRender()
        }
    }

    override fun onCreate(savedInstanceState: Bundle?) {
        ...
        mRenderer = MyRenderer(context = baseContext, listener = this)
		...
	}
	...
}

当有新的一帧数据时会回调此方法,更新数据并调用glSurfaceView.requestRender() 重新绘制,也就是重新调用Renderer的onDrawFrame方法。

设置顶点坐标、纹理坐标、索引数据

设置顶点坐标,代码如下:

var vertexBuffer = GLTools.array2Buffer(
            floatArrayOf(
                -1.0f, 1.0f, 0.0f,  // top left
                -1.0f, -1.0f, 0.0f,  // bottom left
                1.0f, -1.0f, 0.0f,  // bottom right
                1.0f, 1.0f, 0.0f  // top right
            )
        )

设置纹理坐标,代码如下:

        var texBuffer = GLTools.array2Buffer(
            floatArrayOf(
                0.0f, 0.0f,
                0.0f, 1.0f,
                1.0f, 1.0f,
                1.0f, 0.0f
            )
        )

设置索引数据,代码如下:

var index = shortArrayOf(3, 2, 0, 0, 1, 2)
val indexBuffer = GLTools.array2Buffer(index)

绘制

override fun onDrawFrame(p0: GL10?) {
            GLES20.glUseProgram(mProgramHandle)
            //设置顶点数据
            vertexBuffer.position(0)
            GLES20.glEnableVertexAttribArray(vPositionLoc)
            GLES20.glVertexAttribPointer(vPositionLoc, 3, GLES20.GL_FLOAT, false, 0, vertexBuffer)
            //设置纹理顶点数据
            texBuffer.position(0)
            GLES20.glEnableVertexAttribArray(texCoordLoc)
            GLES20.glVertexAttribPointer(texCoordLoc, 2, GLES20.GL_FLOAT, false, 0, texBuffer)
            //设置纹理
            GLES20.glActiveTexture(GLES20.GL_TEXTURE0)
            GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureId)
            GLES20.glUniform1i(textureLoc, 0)

            GLES20.glDrawElements(
                GLES20.GL_TRIANGLES,
                index.size,
                GLES20.GL_UNSIGNED_SHORT,
                indexBuffer
            )
        }

打开camera

打开camera有2个条件:

  • 有相机权限。
  • SurfaceTexture已经创建完成。

相机权限申请的回调和Renderer中onSurfaceCreated(创建SurfaceTexture的方法)方法都是异步的,也就是说无法知道这2个方法回调的前后顺序,因此需要保存相机权限状态cameraPermission和SurfaceTexture变量surfaceTexture,在2个回调中都调用打开相机方法,在打开相机方法中判断相机权限和SurfaceTexture是否都已经准备完成,是则打开,不是则返回,代码如下:

override fun onSurfaceCreated(p0: GL10?, p1: EGLConfig?) {
	...
	startCamera()
}

fun startCamera() {
            if (!cameraPermission || surfaceTexture == null) {
                return
            }
            val cameraInfo = Camera.CameraInfo()
            val cameraCount = Camera.getNumberOfCameras()
            for (i in 0 until cameraCount) {
                Camera.getCameraInfo(i, cameraInfo)
                if (cameraInfo.facing == Camera.CameraInfo.CAMERA_FACING_FRONT) {
                    val mCamera = Camera.open(i)
                    mCamera.setPreviewTexture(surfaceTexture)

                    //设置分辨率
                    val parameters = mCamera.parameters
                    parameters.setPreviewSize(1280, 720)
                    mCamera.parameters = parameters

                    //开始预览
                    mCamera.startPreview()
                    return
                }
            }
        }

运行效果如下:

OpenGL ES for Android 相机预览

运行后发现相机的画面是倒的,这是因为camera本身输出的预览流就是倒的,下面通过矩阵旋转解决此问题,顶点shader修改如下:

attribute vec4 a_Position;
attribute vec2 a_TexCoordinate;
uniform mat4 mMatrix;
varying vec2 v_TexCoord;

void main()
{
    v_TexCoord = a_TexCoordinate;
    gl_Position = mMatrix * a_Position;
}

增加了mMatrix矩阵。
获取矩阵参数句柄,代码如下:

override fun onSurfaceCreated(p0: GL10?, p1: EGLConfig?) {
            createProgram()
            ...
            matrixLoc = GLES20.glGetUniformLocation(mProgramHandle, "mMatrix")
            ...

        }

旋转90度,代码如下:

var mMatrix = FloatArray(16)
override fun onSurfaceChanged(p0: GL10?, width: Int, height: Int) {
            GLES20.glViewport(0, 0, width, height)
            Matrix.setIdentityM(mMatrix, 0)
            Matrix.rotateM(mMatrix,0,90F,0F,0F,1F)
        }

设置矩阵参数,代码如下:

override fun onDrawFrame(p0: GL10?) {
           	...
            GLES20.glUniformMatrix4fv(matrixLoc, 1, false, mMatrix, 0)
            ...
        }

运行后发现画面调整正了,但左右镜像,这个时候需要画面绕y轴旋转180度,这样就解决了左右镜像问题,代码如下:

override fun onSurfaceChanged(p0: GL10?, width: Int, height: Int) {
            GLES20.glViewport(0, 0, width, height)
            Matrix.setIdentityM(mMatrix, 0)
            Matrix.rotateM(mMatrix,0,180F,0F,1F,0F)
            Matrix.rotateM(mMatrix,0,90F,0F,0F,1F)
        }
注意,对预览流的操作是先绕z轴旋转90度,使画面调正,然后再绕y轴旋转180度,但写代码的时候要绕y轴旋转180度写在前面。

最终效果如下:

OpenGL ES for Android 相机预览

更多相关阅读:

如果这篇文章有帮助到您,希望您关注我的公众号,谢谢。

OpenGL ES for Android 相机预览

上一篇:

下一篇: