﻿ 基于场景模型的双目相机动态检校方法
 文章快速检索
 同济大学学报(自然科学版)  2018, Vol. 46 Issue (11): 1594-1600.  DOI: 10.11908/j.issn.0253-374x.2018.11.019 0

引用本文

LI Zhengning, LIU Chun, WU Hangbin. Dynamic Stereo-camera Calibration with Infrastructure Model[J]. Journal of Tongji University (Natural Science), 2018, 46(11): 1594-1600. DOI: 10.11908/j.issn.0253-374x.2018.11.019

基金项目

“十三五”国家科技支撑计划(2016YFB0502102, 2016YFB0502104)；国家自然科学基金(41771481, 41671451)

文章历史

Dynamic Stereo-camera Calibration with Infrastructure Model
LI Zhengning , LIU Chun , WU Hangbin
College of Surveying and Geo-Informatics, Tongji University, Shanghai 200092, China
Abstract: A scene model based stereo-camera calibration method is presented in this paper. The initial calibration process firstly constructs a global scene model, and based on this, both intrinsic and extrinsic parameters of the stereo-camera are estimated separately and optimized jointly. The update calibration utilizes the established scene model to restore three dimention-two dimention relation between model coordinate and image coordinate, with the relation, stereo-camera parameters are updated through joint optimization. The experiments in real-world show that the method calibrates all the parameters while SLAM-based method only solves a part, and the accuracy of the method is better than the chess-board based method.
Key words: stereo-camera    camera calibration    scene modeling    joint optimization

1 双目相机检校原理

 图 1 双目相机内、外方位元素示意图 Fig.1 Intrinsic and extrinsic parameters of stereo-camera
1.1 内方位元素初步估计

 $\mathit{\boldsymbol{x}} = \mathit{\boldsymbol{PX}}$ (1)

 $\mathit{\boldsymbol{K}} = \left[ {\begin{array}{*{20}{c}} f&s&{{u_0}}\\ 0&f&{{v_0}}\\ 0&0&1 \end{array}} \right]$ (2)

 $\mathop {\min }\limits_\mathit{\boldsymbol{K}} \sum\limits_{n, p} {\left( {{{\left\| {\mathit{\boldsymbol{K}}\left[ {{\mathit{\boldsymbol{R}}_n}|{\mathit{\boldsymbol{t}}_n}} \right]{\mathit{\boldsymbol{X}}_p} - {\mathit{\boldsymbol{x}}_{np}}} \right\|}^2}} \right)}$ (3)

1.2 外方位元素初步估计

 ${\mathit{\boldsymbol{T}}_{{\rm{L}}n}} = {\mathit{\boldsymbol{R}}_{\rm{E}}}{\mathit{\boldsymbol{T}}_{{\rm{R}}n}} + {\mathit{\boldsymbol{t}}_{\rm{E}}}$ (4)

 ${\mathit{\boldsymbol{R}}_{\rm{E}}} = {\mathit{\boldsymbol{R}}_{\rm{L}}}\mathit{\boldsymbol{R}}_{\rm{R}}^{\rm{T}}, {\mathit{\boldsymbol{t}}_{\rm{E}}} = {\mathit{\boldsymbol{t}}_{\rm{L}}}{\mathit{\boldsymbol{R}}_{\rm{E}}}{\mathit{\boldsymbol{T}}_{\rm{R}}}$ (5)

 $\overline {{\mathit{\boldsymbol{q}}_{\rm{E}}}} = \mathop {\arg \max }\limits_{{\mathit{\boldsymbol{q}}_{\rm{E}}} \in {S^3}} \mathit{\boldsymbol{q}}_{\rm{E}}^{\rm{T}}\mathit{\boldsymbol{M}}{\mathit{\boldsymbol{q}}_E}$ (6)
 $\mathit{\boldsymbol{M}} \buildrel \Delta \over = \sum\limits_{i = 1}^n {{\mathit{\boldsymbol{q}}_{{\rm{E}}i}}\mathit{\boldsymbol{q}}_{{\rm{E}}i}^{\rm{T}}}$

$\overline {{\mathit{\boldsymbol{q}}_{\rm{E}}}}$M最大特征值对应的特征向量，将其转换为罗德里格斯旋转矩阵的形式，得到旋转矩阵的平均值$\overline {{\mathit{\boldsymbol{R}}_{\rm{E}}}}$.将求得的$\overline {{\mathit{\boldsymbol{R}}_{\rm{E}}}}$代入式(5)求解tE，并求其在欧式空间内的平均值$\overline {{\mathit{\boldsymbol{t}}_{\rm{E}}}}$.最后，将得到的[$\overline {{\mathit{\boldsymbol{R}}_{\rm{E}}}}$|$\overline {{\mathit{\boldsymbol{q}}_{\rm{E}}}}$]作为外方位元素的初始值.

1.3 内、外方位元素联合优化

 $\begin{array}{l} \mathop {\min }\limits_{{\mathit{\boldsymbol{K}}_{\rm{L}}}, {\mathit{\boldsymbol{K}}_{\rm{R}}}, {\mathit{\boldsymbol{R}}_{\rm{E}}}, {\mathit{\boldsymbol{t}}_{\rm{E}}}\mathit{\boldsymbol{, }}{\mathit{\boldsymbol{R}}_{{\rm{L}}n}}, {\mathit{\boldsymbol{t}}_{{\rm{L}}n}}} \sum\limits_{n, p} {\left( {{{\left\| {{\mathit{\boldsymbol{K}}_{\rm{L}}}\left[ {{\mathit{\boldsymbol{R}}_{{\rm{L}}n}}|{\mathit{\boldsymbol{t}}_{{\rm{L}}n}}} \right]{\mathit{\boldsymbol{X}}_p} - {\mathit{\boldsymbol{x}}_{{\rm{L}}np}}} \right\|}^2}} \right.} + \\ \;\;\;\;\;\;\;\left. {{{\left\| {{\mathit{\boldsymbol{K}}_{\rm{R}}}\left[ {\mathit{\boldsymbol{R}}_{\rm{E}}^{\rm{T}}{\mathit{\boldsymbol{R}}_{{\rm{L}}n}}|\mathit{\boldsymbol{R}}_{\rm{E}}^{\rm{T}}\left( {{\mathit{\boldsymbol{t}}_{{\rm{L}}n}} - {\mathit{\boldsymbol{t}}_{\rm{E}}}} \right)} \right]{\mathit{\boldsymbol{X}}_p} - {\mathit{\boldsymbol{x}}_{{\rm{R}}np}}} \right\|}^2}} \right) \end{array}$ (7)

(KL, KR)分别为左、右相机的内方位元素，根据左相机位置以及姿态和外方位元素求得右相机的位置和姿态[RETRLn|RET(tLn-tE)].(tLnpxRnp)为三维场景点Xp在第n个左、右像对中对应的同名二维特征点.通常运动恢复结构(SFM)技术中的光束法平差是将相机作为单独个体，联合优化相机的位置、姿态和场景点，而由于本文中所采用的是双目相机，为保证相机内、外方位元素的一致性，仅优化左相机的位置、姿态以及相机的检校参数.同时，为了防止出现过拟合的情况，在联合优化过程中，不再优化与修正三维场景点坐标.本文通过梯度下降法优化相机内、外方位元素，减小三维投影与其对应的影像特征点之间的误差即重投影误差.由于内、外方位元素的初始值都已经预先求得，因此利用梯度下降优化方法可以快速收敛到最优解，得到精确的检校结果.

2 基于场景模型的双目相机检校流程

 图 2 双目相机动态检校流程 Fig.2 Calibration process of stereo-camera
2.1 基于场景模型的初始检校

2.2 基于场景模型的更新检校

2.3 本文方法与现有检校方法对比

3 实验与分析 3.1 实验数据准备

 图 3 实验平台 Fig.3 Experiment platform
 图 4 实验环境与视觉标志点 Fig.4 Experiment environment and visual marks

3.2 检校实验结果分析

4种检校方法得到的双目相机外方位元素比较如表 1所示，内方位元素比较如表 2所示.

(1) 双目影像校正结果验证

 图 5 双目影像校正结果 Fig.5 Results of stereo rectification

(2) 双目视觉定位结果验证

 图 6 双目视觉定位结果 Fig.6 Results of stereo visual odometry

 图 7 双目视觉定位累积误差对比 Fig.7 Accumulated error comparison of stereo visual odometry
4 结语

 [1] 张梁, 徐锦法, 夏青元, 等. 双目立体视觉的无人机位姿估计算法及验证[J]. 哈尔滨工业大学学报, 2014, 46(5): 66 ZHANG Liang, XU Jinfa, XIA Qingyuan, et al. Pose estimation algorithm and verification based on binocular stereo vision for unmanned aerial vehicle[J]. Journal of Harbin Institute of Technology, 2014, 46(5): 66 [2] 刘昱岗, 王卓君, 王福景, 等. 基于双目立体视觉的倒车环境障碍物测量方法[J]. 交通运输系统工程与信息, 2016, 16(4): 79 LIU Yugang, WANG Zhuojun, WANG Fujing, et al. Vehicle reversing obstacle measurement based on binocular-camera stereo vision[J]. Journal of Transportation Systems Engineering and Information Technology, 2016, 16(4): 79 DOI:10.3969/j.issn.1009-6744.2016.04.012 [3] ZHANG Z. Flexible camera calibration by viewing a plane from unknown orientations[C]// The Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra: IEEE, 1999: 666-673. [4] GEIGER A, MOOSMANN F, CARÖ, et al. Automatic camera and range sensor calibration using a single shot[C]// International Conference on Robotics and Automation. Saint Paul: IEEE, 2012: 3936-3943. [5] CARRERA G, ANGELI A, DAVISON A J. SLAM-based automatic extrinsic calibration of a multi-camera rig[C]// International Conference on Robotics and Automation. Shanghai: IEEE, 2011: 2652-2659. [6] HENG L, FURGALE P, POLLEFEYS M. Leveraging image-based localization for infrastructure-based calibration of a multi-camera rig[J]. Journal of Field Robotics, 2015, 32(5): 775 DOI:10.1002/rob.2015.32.issue-5 [7] HARTLEY R, ZISSERMAN A. Multiple view geometry in computer vision[M]. Cambridge: Cambridge University Press, 2003 [8] HARTLEY R, TRUMPF J, DAI Y, et al. Rotation averaging[J]. International Journal of Computer Vision, 2013, 103(3): 267 [9] MARKLEY F L, CHENG Y, CRASSIDIS J L, et al. Averaging quaternions[J]. Journal of Guidance Control and Dynamics, 2007, 30(4): 1193 DOI:10.2514/1.28949 [10] SCHONBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 4104-4113. [11] 李金岭, 刘鹂, 乔书波, 等. 关于三维直角坐标七参数转换模型求解的讨论[J]. 测绘科学, 2010, 35(4): 76 LI Jinling, LIU Li, QIAO Shubo, et al. Discussion on the determination of transformation parameters of 3D Cartesian coordinates[J]. Science of Surveying and Mapping, 2010, 35(4): 76 [12] GÁLVEZ-LPEZ D, TARDOS J D. Bags of binary words for fast place recognition in image sequences[J]. IEEE Transactions on Robotics, 2012, 28(5): 1188 DOI:10.1109/TRO.2012.2197158 [13] LEPETIT V, MORENO-NOGUER F, FUA P. EPnP: an accurate O(n) solution to the PnP problem[J]. International Journal of Computer Vision, 2009, 81(2): 155 [14] FISCHLER M A, BOLLES R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381 DOI:10.1145/358669.358692 [15] FUSIELLO A, TRUCCO E, VERRI A. A compact algorithm for rectification of stereo pairs[J]. Machine Vision and Applications, 2000, 12(1): 16 DOI:10.1007/s001380050120 [16] KREŠOI, ŠEGVIC S. Improving the egomotion estimation by correcting the calibration bias[C]//Proceedings of the 10th International Conference on Computer Vision Theory and Applications. Berlin: VISAPP, 2015: 347-356.