A binocular vision-based platform has simulated autonomous aerial refuelling (AAR) between two unmanned aerial vehicles (UAVs) in real time, using a boom approach. The platform may be the first step toward a vision-based sensor and navigation system that could enable in-air refuelling of UAVs without pilot assistance.
In order to verify that a vision system could be designed for boom approach AAR, researchers at Beihang University built and tested an integrated vision-based platform consisting of a tanker UAV, a receiver UAV and a ground station. The octocoptor serving as the tanker was equipped with cameras and a visual measurement system.
When the UAV serving as the receiver came within the visual field of the cameras on the tanker (approximately five meters), the binocular vision system was able to capture pictures of the markers on the receiver UAV and extract and generate the pixel coordinates of the markers. An on-board next unit computing (NUC) device processed the pixel coordinates and estimated the position of the receiver.
The visual information obtained from the binocular vision system was transferred to the flight controller and boom controller, which used the pictures to control the flight of both UAVs and the movement of the boom toward the fuel receptacle. Micro air vehicle (MAV) link protocol was used to control communication between the ground station and the UAVs and between the UAV subcomponents.
The researchers based their binocular vision system on two primary procedures: feature extraction and pose estimation. The pixel coordinates of the markers on the receiver UAV, which were painted red, were generated after feature extraction. The generated pixel coordinates were utilized in pose estimation, and the pose estimation was used to calculate the relation between the binocular vision system and the marker coordination system.
Transformation from the red-green-blue (RGB) to the hue-saturation-value (HSV) colour model was implemented to process each captured image. In order to segment the regions of image containing red markers, the upper and lower bound of hue and saturation channels of HSV image were selected to conduct the threshold segmentation.
To confirm the effectiveness of the binocular vision algorithm, the researchers simulated a refuelling boom system. After the frame sequences of the markers were obtained and pose and position were estimated, an arm microcontroller used this information to determine the control level needed to control steering engines. The refuelling boom pointed at the receptacle in the resolved pose and position, and successfully implemented in-air connection.
The left frames captured on-board and the reprojection result on the corresponding right frame. (a) and (c) the feature extraction results on images captured with left camera; (b) and (d) relative projection result on images captured with right camera. Credit: ©Science China Press
A series of flight tests were conducted outdoors under various conditions, including in conditions of strong light and weak light. To ensure safety and ease of observation, the flight height of the UAVs was approximately 10 meters, or 33 feet. The results of the experiment showed that a binocular vision system for AAR of UAVs using a boom approach is feasible. The research team plans to focus its future efforts on improving the stability and accuracy of their binocular vision measurements in more complicated environments.
UAVs are used extensively in military and civilian initiatives. Since most unmanned systems are designed to execute long-running missions, it has been necessary for UAVs to return to base periodically for refuelling. As use of long-running UAVs grows, autonomous aerial refuelling will be of increasing importance.
The work of the researchers was partially supported by National Natural Science Foundation of China (Grant Nos. 61425008, 61333004, 61273054) and Aeronautical Foundation of China (Grant No. 2015ZA51013).
The research was published in Science China (doi: 10.1007/s11432-016-5553-5).
Sources: Photonics Media, Phys Org