Real-time floating 3D display interaction system based on gesture recognition by leap motion

Xing-yu LIN; Yan XING; Han-le ZHANG; Qiong-hua WANG

doi:10.37188/CJLCD.2022-0060

您当前的位置：

首页 >

文章列表页 >

Real-time floating 3D display interaction system based on gesture recognition by leap motion

Near-eye Display and 3D Interaction | 更新时间：2022-06-22

- Real-time floating 3D display interaction system based on gesture recognition by leap motion
- Chinese Journal of Liquid Crystals and Displays Vol. 37, Issue 5, Pages: 654-659(2022)
- 作者机构：
  
  北京航空航天大学仪器科学与光电工程学院，北京 100191
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(U21B2034;62105014;62105016)
- DOI：10.37188/CJLCD.2022-0060
  CLC： TN27
- Received：21 February 2022，
  
  Revised：03 March 2022，
  
  Published：05 May 2022
- 稿件说明：
移动端阅览
Xing-yu LIN, Yan XING, Han-le ZHANG, et al. Real-time floating 3D display interaction system based on gesture recognition by leap motion[J]. Chinese journal of liquid crystals and displays, 2022, 37(5): 654-659.
DOI：

Xing-yu LIN, Yan XING, Han-le ZHANG, et al. Real-time floating 3D display interaction system based on gesture recognition by leap motion[J]. Chinese journal of liquid crystals and displays, 2022, 37(5): 654-659. DOI： 10.37188/CJLCD.2022-0060.

摘要

提出基于Leap Motion手势识别的悬浮真3D显示实时交互系统，可实现对多个悬浮3D图像的独立交互。系统由显示和交互两个模块组成，显示模块利用集成成像3D显示屏再现3D图像，再通过二面角反射镜阵列实现悬浮真3D显示；交互模块通过Leap Motion识别手部各关节空间位置，提取手势信息并判断用户操作意图，利用提出的通道机制，独立改变3D场景内多个物体的空间坐标，渲染对应的3D片源并刷新，完成实时悬浮真3D显示交互，构成人机交互闭环系统。实验结果表明，该系统在3 840×2 160分辨率下，可实现30帧/s的交互帧率。所提系统通过预设手势可实现多物体的独立实时交互，提高了交互自由度。

Abstract

A real-time floating 3D display interaction system is proposed， which can realize independent interaction of multiple floating 3D images. The proposed system consists of the display and interaction modules. In the display module， 3D images are reconstructed through integral imaging 3D display screen and then floats into the air through the dihedral corner reflector array. In the interaction module， Leap Motion captures positions of hand joints， extracts the gesture information and judges the motive of the users. After checking interaction channels， positions of multiple 3D objects are changed individually， and the 3D film source is rendered， which forms a human-computer interaction closed loop and realizes the real-time floating 3D display interaction. The experimental results demonstrate that the system can reach 30 frame/s under the resolution of 3 840×2 160. The real-time individual interaction of multiple objects can be achieved in the proposed system， and the interaction freedom is increased.

关键词

Keywords

references

王琼华． 3D显示技术与器件［M］．北京：科学出版社， 2011 ． doi: 10.1002/j.2637-496x.2011.tb00390.x http://dx.doi.org/10.1002/j.2637-496x.2011.tb00390.x

WANG Q H . 3D Display Technology and Device ［M］. Beijing ： Science Press ， 2011 . （in Chinese） . doi: 10.1002/j.2637-496x.2011.tb00390.x http://dx.doi.org/10.1002/j.2637-496x.2011.tb00390.x

薛钊鸿，孙梓瀚，成泽锐，等．基于人眼检测优化的立体人机交互技术的研究［J］．液晶与显示， 2018 ， 33 （ 11 ）： 958 - 964 ． doi: 10.3788/yjyxs20183311.0958 http://dx.doi.org/10.3788/yjyxs20183311.0958

XUE Z H ， SUN Z H ， CHENG Z R ， et al . 3D human-machine interaction based on human eye detection ［J］. Chinese Journal of Liquid Crystals and Displays ， 2018 ， 33 （ 11 ）： 958 - 964 . （in Chinese） . doi: 10.3788/yjyxs20183311.0958 http://dx.doi.org/10.3788/yjyxs20183311.0958

CHEN C H ， TING C H ， HUANG Y P . 3D interactive system using integral photography by embedded optical sensors for portable devices ［J］. Journal of Display Technology ， 2016 ， 12 （ 11 ）： 1329 - 1334 . doi: 10.1109/jdt.2016.2599186 http://dx.doi.org/10.1109/jdt.2016.2599186

CHAVARRÍA I A S S ， NAKAMURA T ， YAMAGUCHI M . Interactive optical 3D-touch user interface using a holographic light-field display and color information ［J］. Optics Express ， 2020 ， 28 （ 24 ）： 36740 - 36755 . doi: 10.1364/oe.409126 http://dx.doi.org/10.1364/oe.409126

李炜，李海峰，徐良，等．基于体感检测器件的悬浮真三维显示力触觉交互［J］．光学学报， 2020 ， 40 （ 9 ）： 0915004 ． doi: 10.3788/aos202040.0915004 http://dx.doi.org/10.3788/aos202040.0915004

LI W ， LI H F ， XU L ， et al . Suspended three-dimensional display haptic interaction based on Leap Motion ［J］. Acta Optica Sinica ， 2020 ， 40 （ 9 ）： 0915004 . （in Chinese） . doi: 10.3788/aos202040.0915004 http://dx.doi.org/10.3788/aos202040.0915004

MAEKAWA S ， NITTA K ， MATOBA O . Transmissive optical imaging device with micromirror array ［C］// Proceedings of SPIE 6392， Three-Dimensional TV， Video， and Display Ⅴ . Boston ： SPIE ， 2006 ： 63920E . doi: 10.1117/12.690574 http://dx.doi.org/10.1117/12.690574

MAEKAWA S ， NITTA K ， MATOBA O . Advances in passive imaging elements with micromirror array ［C］// Proceedings of SPIE 6803， Stereoscopic Displays and Applications XIX . San Jose ： SPIE ， 2008 ： 68030B . doi: 10.1117/12.767198 http://dx.doi.org/10.1117/12.767198

MIYAZAKI D ， MAEDA Y ， MAEKAWA S . Floating three-dimensional image display using micro-mirror array imaging element ［C］// Proceedings of SPIE 9495， Three-Dimensional Imaging， Visualization， and Display 2015 . Baltimore ： SPIE ， 2015 ： 949508 . doi: 10.1117/12.2177534 http://dx.doi.org/10.1117/12.2177534

张汉乐．集成成像3D显示性能提升方法［D］．成都：四川大学， 2020 ．

ZHANG H L . Methods for improving performance of integral imaging 3D display technology ［D］. Chengdu ： Sichuan University ， 2020 . （in Chinese）

DUNN F ， PARBERRY I . 3 D Math Primer for Graphics and Game Development ［M］. 2nd ed . Boca Raton ： CRC Press ， 2011 . doi: 10.1201/b11152-10 http://dx.doi.org/10.1201/b11152-10

Views

338

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

CJLCD | Floating 3D display + gesture interaction = the future is within reach

Generation of elemental image arrays for different reconstruction depths

Technical analysis and research progress of near-eye display optical systems

Resolution-enhanced integral imaging 3D display using retro-reflector and reflective polarizer

Characterization and compensation of position errors of macro lens array in large-format integral imaging system

Related Author

XING Yan

LIN Xing-yu

ZHANG Han-le

WANG Qiong-Hua

DENG Huan

GUO Zhao-da

LI Jiao-jiao

YANG Cui-ni

Related Institution

College of Electronics and Information Engineering， Sichuan University

School of Electronic Engineering， Chengdu Technological University

Department of Information Communication， Army Academy of Armored Forces

China Shipbuilding Industry Corporation Jiangsu Automation Research Institute

Key Laboratory of Airborne Optical Imaging and Measurement, Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Science

AI问答

Address：No.3888 Dong Nanhu Road, Changchun, Jilin, China 130033 Postal code：130033
Tel：0431-86176059 Email：yjxs@ciomp.ac.cn
Technical support is provided by Beijing Founder electronics co., LTD 吉ICP备11002662号-17 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.

⁰