|Member Login

Media Releases

More Stories

How we trained Harvey the robotic harvester not to ‘roll off the job’!

Are nine eyes better than one for a robotic fruit and vegetable harvester?

Yes and no… That’s the answer from a ground-breaking Australian Centre for Robotic Vision research project that has transformed Centre star, Harvey the robotic harvester, into a robo-Cyclops of sorts!

The project involves Harvey’s creator/Centre Associate Investigator Chris Lehnert teaming up with leading Defence industry avionics engineer Paul Zapotezny-Anderson to ensure an autonomous, vision-empowered robot like Harvey won’t ‘roll off the job’ when leaves get in the way of fruit and vegetable picking.

“In robotic harvesting, dealing with obstructing leaves and branches that stop a robot from getting a clear view of a crop is really tricky because the ‘occlusions’ are extremely difficult to model, or anticipate, in a technical way,” said Mr Zapotezny-Anderson, who is currently on sabbatical from Airbus to complete a Masters of Engineering (Electrical Engineering) at QUT.

“Basically, robots don’t cope well and often just give up!”

This is no longer the case for Harvey thanks to research led by Dr Lehnert at the Centre’s QUT-based Lab.

What Dr Lehnert initially developed in 2018 as a 3D-printed system with nine eyes (cameras) operating at different depths, enabling Harvey to look around obstructing leaves much like a human, has been revolutionised into a faster ‘one-eyed’ system that retains the scope of multiple eyes.

“The real beauty of the system is that one eye has the power of nine, without the performance limitations of a multi-camera system… not least being slower data processing time,” Dr Lehnert said.

The new monocular-vision system, aptly entitled Deep 3D Move To See, is the result of a novel ‘deep learning’ method involving one camera being trained off nine via a Deep Convolutional Neural Network.

Click here for more information contained in the paper, ‘Towards active robotic vision in agriculture: A deep learning approach to visual servoing in occluded and unstructured protected cropping environments’, to be presented at the 6th IFAC Conference on Sensing, Control and Automation Technologies for Agriculture (3-6 December).

“Basically, Harvey has been trained off all nine cameras to use monocular vision to guide its end effector (harvesting tool or gripper) on the fly around occluding leaves or branches to get an unblocked view of the crop, without prior knowledge of the environment it needs to navigate in,” Mr Zapotezny-Anderson said.

“Perhaps a good way of thinking about it is that Harvey can now harvest crops with one eye open and eight eyes closed!”

Watch Harvey in action

*NOTE: While the 3D-printed camera housing attached to Harvey’s gripper retains nine cameras (as seen in the video taken during rigorous trials at the Centre’s QUT-based Lab), Harvey now relies on only one camera, positioned in the centre of the array.

Looking ahead, Dr Lehnert said Harvey’s ability to seamlessly perform in unstructured greenhouse environments of the real world would be further strengthened as part of QUT’s involvement in the Future of Food Systems Cooperative Research Centre announced in March. Read more>>

In the CRC, Dr Lehnert will be working on developing robotics and smart technology for vertical and indoor protected cropping.

He said autonomous, vision-empowered robots like Harvey had the potential to assist fruit and vegetable farmers across Australia who often experienced a shortage in skilled labour, especially during optimal harvesting periods.

“The future potential of robotics in indoor protected cropping will be their ability to intelligently sense, think and act in order to reduce production costs and maximise output value in terms of crop yield and quality,” Dr Lehnert said.

“Robotics taking action, such as autonomous harvesting within indoor protected cropping will be a game changer for growers who are struggling to reduce their production costs.”

The Future of Food Systems CRC was initiated by NSW Famers Association on behalf of the national representative farm sector and as part of a broader industry-wide push to increase value-adding capability, product differentiation and responsiveness to consumer preferences.

Did you know? This week (14-18 October 2019) is Ag and Rural Week, marked by the National Farmers’ Federation (NFF) Leaders’ Summit in Canberra and release of its report card on progress towards the goal for agriculture to be a $100 billion industry by 2030. The week also featured World Food Day (16 October).

With the global population projected to reach 9.8 billion in 2050, the Australian Centre for Robotic Vision is focused on giving the next generation of robots the vision and understanding to help solve real-world challenges, including sustainable food production.

Shelley Thomas, Communications Specialist
Australian Centre for Robotic Vision
P: +61 7 3138 4265 | M: +61 416 377 444 | E: shelley.thomas@qut.edu.au

About The Australian Centre for Robotic Vision
The Australian Centre for Robotic Vision is an ARC Centre of Excellence, funded for $25.6 million over seven years to form the largest collaborative group of its kind generating internationally impactful science and new technologies that will transform important Australian industries and provide solutions to some of the hard challenges facing Australia and the globe. Formed in 2014, the Australian Centre for Robotic Vision is the world’s first research centre specialising in robotic vision. They are a group of researchers on a mission to develop new robotic vision technologies to expand the capabilities of robots. Their work will give robots the ability to see and understand for the sustainable well-being of people and the environments we live in. The Australian Centre for Robotic Vision has assembled an interdisciplinary research team from four leading Australian research universities: QUT, The University of Adelaide (UoA), The Australian National University (ANU), and Monash University as well as CSIRO’s Data61 and overseas universities and research organisations including the French national research institute for digital sciences (INRIA), Georgia Institute of Technology, Imperial College London, the Swiss Federal Institute of Technology Zurich (ETH Zurich), University of Toronto, and the University of Oxford.

PostedOctober 18, 2019

Australian Centre for Robotic Vision
2 George Street Brisbane, 4001
+61 7 3138 7549