Integration of Live Surround Video for the First Immersive RockCommander VR Digital Twin Prototype

Project Member(s): Paul, G., Sutjipto, S.

Funding or Partner Organisation: Ausdrill Pty Ltd (Ausdrill)
Ausdrill Pty Ltd (Ausdrill)

Start year: 2023

Summary: The RockCommander is integral to Ausdrill’s extensive range of purpose-built drilling rigs. Currently, the operation of these vehicles is done manually and onsite in harsh environments. Thus, there is pressure to equip drill rigs with state-of-the-art remote-control features and improved levels of autonomy. Ausdrill has already combined existing Industry 4.0 systems to create a real-time virtual representation of the RockCommander (digital twin) capable of displaying the state of the vehicle for monitoring. Although this gives operators a method of understanding the internal state of the vehicle, it does not provide any information regarding external stimuli. This holistic approach would facilitate improved realism and immersion so that an operator could make informed control actions to operate the vehicle without being in the cockpit. Achieving improved remote control requires the strategic presentation of dense exteroceptive sensor data streams to the operator and a realistic RockCommander cockpit to create an immersive VR control experience. The VR environment that resembles the RockCommander cockpit would enable an operator to “see” the environment, like they would in a real cabin, and would provide a preliminary interface to be used for implementing basic drill rig controls from drilling to tramming.

Publications:

Chaisumdet, D, Tung Le, D, Khoa Nguyen, DD, Sutjipto, S, Rizvi, D & Paul, G 1970, 'Enhancing the Intuitiveness of Remote Mobile Industrial Robots with Haptic Devices', Australasian Conference on Robotics and Automation, ACRA, Australasian Conference on Robotics and Automation, ARRA, Sydney.

Tung Le, D, Sutjipto, S, Khoa Nguyen, DD, Long Vu, T, Thong Vu, PD, Munasinghe, N & Paul, G 1970, 'HALO: a Rock Scaling Mobile Manipulator with Interactive Virtual Reality Live Digital Twin', Australasian Conference on Robotics and Automation, ACRA, Sydney.

FOR Codes: Field robotics, Mining and extraction of energy resources not elsewhere classified

Acknowledgement of Country

UTS acknowledges the Gadigal people of the Eora Nation, the Boorooberongal people of the Dharug Nation, the Bidiagal people and the Gamaygal people, upon whose ancestral lands our university stands. We would also like to pay respect to the Elders both past and present, acknowledging them as the traditional custodians of knowledge for these lands.