Modern life as we know it is resource intensive, requiring reliable and low-cost supply of food, energy and minerals. Australia is an abundant producer, and exporter, of food, energy and minerals. environment in which we live is critical to our wellbeing. This environment comprises both the natural environment and the built environment.
Food production today is ultimately reliant on human labour. The least intensive is extensive crop agriculture and dairy production which are highly mechanised and beginning to become automated. Animal agriculture, extensive grazing and feedlots, is not labour intensive but the work is physically hard and in remote locations. Horiculture, the production of fruit, vegetables, nuts and flowers, varies from highly mechanised to labour intensive, and again the work is physically hard and in remote locations. It is increasingly difficult to source workers to perform remote physical work which leads to incomplete harvests, wasted produce and an inability to expand production to meet growing global demand. Robotic vision technology has the potential to identify and control weeds, pick fruit, pollinate flowers and even muster cattle.
Mineral production, in particular mining, is highly mechanised and increasingly automated. The business drivers are increased productivity and capital utilisation, reduced machine damage and removing workers from hazardous environments. Robotic vision technology has the potential to drive vehicles, guide excavation and truck loading, and perform drilling, blasting and survey work in surface, underground, undersea and space mining.
Thierry Peynot, Michael Milford
Researchers from the Queensland University of Technology (QUT) and the Australian Centre for Robotic Vision, in collaboration with Mining3, have been chosen to help Caterpillar take its mining equipment and automation technology to the next level. The Queensland Government awarded a team led by Robotic Vision Chief Investigator Michael Milford, and including Affiliate Researcher Thierry Peynot, $428-thousand in funding as part of its Advance Queensland Innovation Partnerships program. The funding, combined with other funding from QUT, Caterpillar and Mining3, will help Michael, Thierry, and their team develop technologies that could ultimately enable the automation of underground mining vehicles. Right now, lasers are being used to help with attempts to automate vehicles involved in underground mining operations. The aim of this project is to develop a camera-based positioning system on mining vehicles to help track them in these harsh, underground environments. That will make the work safer and more economic. “If you know where everything is in a mine site at all times you will be able to optimise how the mine site operates and keep your industry competitive,” Michael said. “We hope that we can develop some next-generation positioning technologies which can be deployed throughout their fleet of vehicles that are all around the world.” The Queensland Government funding was announced in September 2016 by State Development Minister Anthony Lynham and by State Minister for Innovation, Science and the Digital Economy Leeanne Encoch MP. “The researchers will look to solutions based on developing a cost-effective, reliable camera-based positioning system for locating and tracking underground mining vehicles within one metre of accuracy as well as a sophisticated, multi-sensor system that provides centimetre-accurate positioning,” Ms Enoch said. Milford says Caterpillar will give researchers access to some very good equipment. In collaboration with Mining3, they will also provide access to some exclusive mine sites. “Regularly interacting with and talking with key companies and players in this space including Caterpillar, Mining3 and other companies has been invaluable in further shaping our research agenda moving forward to maximize its relevance to what industry needs now and into the future,” Michael said. Michael says some of the systems his team will be working on are already automated. Their job will be to improve their reliability. Michael also thinks this type of research and technology could have wider implications outside of mining. “It will be interesting to see how technology development in mining will interact with and complement that being developed in other major fields such as self-driving cars,” Michael said. This was one of 15 projects to receive an Advance Queensland grant.
As part of the Queensland Department of Agriculture and Fisheries (DAF) three-year Strategic Investment in Farm Robotics (SIFR), capsicum (bell or sweet peppers) was identified as an important Queensland crop that could benefit from robotic harvesting. The major challenges for robotic harvesting were associated with image processing (green fruit on green background and heavy occlusion), and manipulation due to the unstructured environment of the crop. The SIFR team developed an algorithm to detect in-field capsicum that improves on state-of-the-art vision systems. By using more effective visual texture features than previously considered in the literature, our algorithm is able to detect approximately 70% of in-field capsicum. We tested our algorithms using both day and night imagery, and as expected best performance is obtained at night due to better control of illumination. We also assessed the performance of our algorithm against capsicum detection of humans and found that the results are comparable. A new agricultural robot prototype – nicknamed ‘Harvey’ – was devloped by the team and designed to harvest capsicums. In November 2015, the team conducted the first trials of the robot at a Queensland Government protected cropping facility in North Queensland. Tasked with identifying and picking red capsicums, Harvey performed significantly better than any capsicum-harvesting robot ever has. In late 2014, worldwide literature indicated a success rate of only 6% in testing scenarios similar to those used for Harvey, and up to 30% when the crop is modified and leaves are removed. Harvey achieved a harvesting success rate of 65% with unmodified crops (that is, with no leaves removed or fruit moved before harvesting). In 2016, the research team will fine-tune Harvey’s performance and conduct further trials, and they believe only minor modifications will be required to achieve an overall success rate exceeding 90%. How does it work? Harvey’s robotic arm has a camera and a unique cutting tool attached to it. Using data from the camera, the robot creates a 3D model of each fruit and its surroundings and plans and controls the robotic arm and cutting tool as they locate and detach the fruit. The combination of state-of-the-art robotic-vision software and novel crop-manipulation tools enables successful harvesting of the crop and promises significant benefits for horticulture growers, who export more than $2b in products every year. We are now seeking partners both in Australia and overseas to commercialise this technology. In the future, our researchers also plan to investigate how automated harvesting technologies can be used for other crops, such as mangoes, strawberries and avocados.
Australian Centre for Robotic Vision
2 George Street Brisbane, 4001
+61 7 3138 7549