使用Kalibr进行IMU+相机的标定

程序员文章站 2022-04-18 12:34:54

...

利用kalibr标定IMU和相机：　　https://github.com/ethz-asl/kalibr/wiki/Camera-IMU-calibration#2-collect-images

在开始需要明确几点的是：需要先安装kalibr，该工具主要是标定相机和IMU，在标定的过程中需要用到IMU和相机的内参文件，这里的内参文件可以用ROS 帮忙获得，或者如果已经有这个文件可以直接用。

接下要做的即安装realsense的ros接口主要是为了方便读取topic来录制rosbag，因为kalibr在标定的时候需要用rosbag中的数据进行标定；

这里的相机采用realsense d435i 因此在安装标定前要确保相关库已经安装

-----IMU内参标定-----

1. 确保realsense的SDK已经安装:

2. 安装realsense的ros warpper

https://github.com/IntelRealSense/realsense-ros 这是前两步的参考链接，包含安装realsense的SDK和roswarpper

我之前已经安装过ros kinect在Ubuntu16.04的平台上，因此这里直接从catkin的工作空间开始

3. 标定IMU

3.1 安装ceres： https://blog.csdn.net/jz1570551401/article/details/78207208

1. 预安装，其中libgoogle-glog-dev 可能需要再单独安装
sudo apt-get install liblapack-dev libsuitesparse-dev libcxsparse3.1.2 libgflags-dev 
libgoogle-glog-dev libgtest-dev   
2. 下载ceres：https://github.com/ceres-solver/ceres-solver
3. 解压文件后进入ceres-master:
    mkdir build
    cd build
    cmake ..  //如果这一步骤报错，可以检查一下是具体那个库没有安装成功，再sudo apt-get 对应的库就好
    make  //大概等几分钟
    sudo make install 
    
    安装完成后/usr/local/include/ceres 和 /usr/local/lib/ 中

3.2 下载code_utils： https://github.com/gaowenliang/code_utils

解压至catkin下的src文件，然后在code_utils下面找到sumpixel_test.cpp，修改#include "backward.hpp"为 #include “code_utils/backward.hpp”，然后运行catkin_make

错误提示：code_utils/backward.hpp:216:30: fatal error: elfutils/libdw.h: No such file or directory
compilation terminated.

　　　　　运行： apt-get install libdw-dev 安装对应的库即可，一般安装这个就可以，如果这样还不可以试试：sudo apt-get install elfutils

3.3 下载imu_utils: https://github.com/gaowenliang/imu_utils

再次catkin_make

3.4 接下来就可以就可以标定IMU了：

<launch>
    <node pkg="imu_utils" type="imu_an" name="imu_an" output="screen">
        <param name="imu_topic" type="string" value= "/camera/imu"/>  #imu topic的名字
        <param name="imu_name" type="string" value= "d435i"/>
        <param name="data_save_path" type="string" value= "$(find imu_utils)/data/d435i"/>
        <param name="max_time_min" type="int" value= "30"/> #标定的时长
        <param name="max_cluster" type="int" value= "100"/>
    </node>
</launch>

将该launch文件保存成imu.launch至src和install下的launch中，然后在两个终端中同时运行

~/catkin_ws_rs$ roslaunch realsense2_camera rs_camera.launch
~/catkin_ws_rs/install/share/realsense2_camera/launch$ roslaunch imu.launch

运行imu.launch文件的终端中会出现

process[imu_an-1]: started with pid [8274]
[ INFO] [1571305008.871258796]: Loaded imu_topic: /camera/imu
[ INFO] [1571305008.872137757]: Loaded imu_name: d435i
[ INFO] [1571305008.872762617]: Loaded data_save_path: /home/shinan/catkin_ws_rs/src/imu_utils/data/d435i
[ INFO] [1571305008.873749085]: Loaded max_time_min: 30
[ INFO] [1571305008.874493932]: Loaded max_cluster: 100
gyr x  num of Cluster 100
gyr y  num of Cluster 100
gyr z  num of Cluster 100
acc x  num of Cluster 100
acc y  num of Cluster 100
acc z  num of Cluster 100
wait for imu data.

接下来等待30分钟即可，标定完成后imu的参数将会被保存在data_save_path的路径中；

也可以录制bag保存imu数据后进行标定；https://blog.csdn.net/fang794735225/article/details/92804030

将IMU的标定结果保存到文件imu.yaml中：

#Accelerometers
accelerometer_noise_density: 2.4753056458491202e-02   #Noise density (continuous-time)
accelerometer_random_walk:   5.1092097834339175e-04   #Bias random walk

#Gyroscopes
gyroscope_noise_density:     4.0642638745600332e-03   #Noise density (continuous-time)
gyroscope_random_walk:       2.6537322064011450e-05   #Bias random walk

rostopic:                    /imu0      #the IMU ROS topic
update_rate:                 200.0      #Hz (for discretization of the values above)

相机标定：https://blog.csdn.net/heyijia0327/article/details/43538695

标定realsense的RGB相机，如果使用的是其他相机可以用opencv采集图像后进行单目标定，这里不多写，主要是标定Realsense的彩色相机：

设定好相机的相关参数后(主要是size 和fps)：目前设置图像大小为640*480 帧率为30

roslaunch realsense2_camera rs_camera.launch

启动realsense相机后，可以用rostopic list 查看对应的打开的topics

可以用ros自带的标定方法进行相机标定：

~/catkin_ws_rs$ rosrun camera_calibration cameracalibrator.py  --size 8x6 --square 0.035 image:=/camera/color/image_raw

使用Kalibr进行IMU+相机的标定

最开始的CALBRATE按钮为灰色，当采集的图像符合条件够就变成绿色，然后可以点击CALIBRATE进行标定，标定结果显示如下：

  ('D = ', [0.20848791559812113, -0.36507062055202405, 0.012079145675854016, -0.006358054188595459, 0.0])
('K = ', [653.7064773464425, 0.0, 314.4404941629716, 0.0, 654.2058371422769, 261.91335661459755, 0.0, 0.0, 1.0])
('R = ', [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0])
('P = ', [677.4393920898438, 0.0, 311.13625111082183, 0.0, 0.0, 676.0581665039062, 266.3551872155367, 0.0, 0.0, 0.0, 1.0, 0.0])

将标定的结果保存到.yaml文件中如下所示：

cam0:  #640*480
  camera_model: pinhole
  intrinsics: [653.7064773464425, 654.2058371422769,314.4404941629716, 261.91335661459755]
  distortion_model: radtan
  distortion_coeffs: [0.20848791559812113, -0.36507062055202405, 0.012079145675854016, -0.006358054188595459]
  rostopic: /camera/color/image_raw
  resolution: [640, 480]

准备标定板文件aprilgrid.yaml：

使用Kalibr进行IMU+相机的标定

相机+IMU标定： https://github.com/ethz-asl/kalibr/wiki/Camera-IMU-calibration#2-collect-images

1. 启动realsense相机和imu，所以也要先设置好rs_camera.launch文件中的相关参数；

target_type: 'aprilgrid' #gridtype
tagCols: 6               #number of apriltags
tagRows: 6               #number of apriltags
tagSize: 0.021383           #size of apriltag, edge to edge [m]
tagSpacing: 0.2853        #ratio of space between tags to tagSize 0.61cm/2.1383cm = 0.285273348
                         #example: tagSize=2m, spacing=0.5m --> tagSpacing=spacing/tagSize = 0.25[-]

2. 为相机和IMU录制数据包，

可以先运行 rosrun image_view image_view image:=/camera/color/image_raw 查看图像，保证在录制数据包的时候标定板不会超出画面

录制的时候尽量保证图像清晰，不要剧烈移动，同时尽可能地**IMU的各个角度，各个方向：

rosbag record /camera/color/image_raw  /camera/imu -O rgb_imu.bag

3. 进行标定：参数分别是包名称(不用加.bag)，相机参数文件，imu参数文件，标定板文件

~/Project/kalibr_workspace$ kalibr_calibrate_imu_camera --bag ~/Project/kalibr_workspace/rgb_imu.bag.bag --cam ~/Project/kalibr_workspace/camchain.yaml --imu ~/Project/kalibr_workspace/imu.yaml --target ~/Project/kalibr_workspace/aprilgrid.yaml

出现错误提示：

importing libraries
Traceback (most recent call last):
  File "/home/shinan/Project/kalibr_workspace/devel/bin/kalibr_calibrate_imu_camera", line 15, in <module>
    exec(compile(fh.read(), python_script, 'exec'), context)
  File "/home/shinan/Project/kalibr_workspace/src/kalibr-master/aslam_offline_calibration/kalibr/python/kalibr_calibrate_imu_camera", line 5, in <module>
    from kalibr_imu_camera_calibration import *
  File "/home/shinan/Project/kalibr_workspace/src/kalibr-master/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/__init__.py", line 4, in <module>
    import IccSensors as sens
  File "/home/shinan/Project/kalibr_workspace/src/kalibr-master/aslam_offline_calibration/kalibr/python/kalibr_imu_camera_calibration/IccSensors.py", line 16, in <module>
    import scipy.optimize
ImportError: No module named scipy.optimize

安装python相关库：

sudo apt-get install python-scipy

标定结果如下：

After Optimization (Results)
==================
Normalized Residuals
----------------------------
Reprojection error (cam0):     mean 0.552748299525, median 0.473100512902, std: 0.3640429471
Gyroscope error (imu0):        mean 0.215478847361, median 0.161409013393, std: 0.19024901177
Accelerometer error (imu0):    mean 0.165613674102, median 0.132037510374, std: 0.129116309678

Residuals
----------------------------
Reprojection error (cam0) [px]:     mean 0.552748299525, median 0.473100512902, std: 0.3640429471
Gyroscope error (imu0) [rad/s]:     mean 0.0123851576362, median 0.00927736573336, std: 0.0109350130176
Accelerometer error (imu0) [m/s^2]: mean 0.0579749018736, median 0.0462211936851, std: 0.04519859501

Transformation T_cam0_imu0 (imu0 to cam0, T_ci): 
[[ 0.9999488  -0.00279248  0.00972605  0.03555196]
 [ 0.00297818  0.99981255 -0.01913078 -0.00461578]
 [-0.0096708   0.01915877  0.99976968  0.00838297]
 [ 0.          0.          0.          1.        ]]

cam0 to imu0 time: [s] (t_imu = t_cam + shift)
-0.0280180128084

IMU0:
----------------------------
  Model: calibrated
  Update rate: 200.0
  Accelerometer:
    Noise density: 0.0247530564585 
    Noise density (discrete): 0.350061081538 
    Random walk: 0.000510920978343
  Gyroscope:
    Noise density: 0.00406426387456
    Noise density (discrete): 0.0574773709247 
    Random walk: 2.6537322064e-05
  T_i_b
    [[ 1.  0.  0.  0.]
     [ 0.  1.  0.  0.]
     [ 0.  0.  1.  0.]
     [ 0.  0.  0.  1.]]
  time offset with respect to IMU0: 0.0 [s]

  Saving camera chain calibration to file: camchain-imucam-homeshinanProjectkalibr_workspacecam_imu640_480_retrgb_imu.bag.yaml

  Saving imu calibration to file: imu-homeshinanProjectkalibr_workspacecam_imu640_480_retrgb_imu.bag.yaml
  Detailed results written to file: results-imucam-homeshinanProjectkalibr_workspacecam_imu640_480_retrgb_imu.bag.txt
Generating result report...

Timeshift表示的就是IMU相对于camera的延时；

参考博客：https://blog.csdn.net/mxdsdo09/article/details/83514310

上一篇： php 过滤敏感字符类实例代码

下一篇：方差分析实战

使用Kalibr进行IMU+相机的标定

-----IMU内参标定-----

相机标定：https://blog.csdn.net/heyijia0327/article/details/43538695

相机+IMU标定： https://github.com/ethz-asl/kalibr/wiki/Camera-IMU-calibration#2-collect-images

使用无他相机APP如何进行直播？使用无他相机进行直播的教程

Android使用系统相机进行拍照的步骤

使用无他相机APP如何进行直播？使用无他相机进行直播的教程

基于kalibr的xtion pro live 相机及与IMU的标定

使用Kalibr进行IMU+相机的标定

Android使用系统相机进行拍照的步骤