Tree detection with low-cost 3D sensors for autonomous navigation in orchards
ABSTRACT
This paper deals with autonomous farming and with the autonomous navigation of an agricultural robot in orchards. These latter are typical semi-structured environments where the dense canopy prevents from using GPS signal and embedded sensors are often preferred to localize the vehicle. To move safely
in such environments, it is necessary to provide the robot the ability of detecting and localizing trees. This paper focuses on this problem. It presents a low cost but efficient vision-based system allowing to detect accurately, quickly and robustly the trees. It is made of four stereo cameras which provide a point cloud characterizing the environment. The key idea is to find the tree trunks by detecting their shadows which are materialized by concavities in the obtained point cloud. In this way, branches and leaves are not taken into account, improving the detection
robustness and therefore the navigation strategy. The method has been implemented using ROS and validated using data sequences taken in several different orchards. The obtained results definitely validate the approach and its performances show that the processing time (around 1ms) is sufficiently short for the data to be used at the control level. A comparison with other approaches from the literature is also provided.
EXISTING SYSTEM :
It is projected that agricultural production will need to double by 2050 to cover the needs of an increasing population for food, feed, fiber and biofuels, and do so in a sustainable manner, despite increasing shortages in available farm labor. Advanced automation of field operations has been identified as a key technology to achieve this goal . One of the main challenges in developing automated solutions for operations like mowing, spraying, pruning, thinning and harvesting in orchards is reliable and precise autonomous navigation. The most basic operation of an autonomous vehicle in an orchard is to drive from one end of a row to the other end (row traversal), and then navigate in the headland to turn and enter the next row.
This operation is repeated to cover the area of interest (e.g., an orchard block). Unfortunately, precision GPS-based guidance is often unreliable in orchards. Tall tree foliage can introduce large errors in GPS positions due to multipath effects, and also cause intermittent or complete GPS signal outage. Furthermore, commercial orchards constitute semi-structured outdoor environments: trees are planted in parallel rows, but individual tree positions and trunk-canopy shapes and sizes vary in space and time and tree lines may also not be perfectly parallel or at exactly the same distance from each other. Finally, precise georeferenced up-to-date orchard maps are typically unavailable and can be costly to generate. For these reasons, orchard navigation based only on GPS has not proved – so far – to be reliable and accurate enough to become commercially available. An alternative approach is to utilize on-board sensors that acquire data from the surrounding environment and develop navigation strategies that exploit the spatial structure of orchards. Since orchard structure is primarily defined by the tree rows, reliable detection and localization of trees is considered very important for navigation.
Such a capability would make it possible to navigate locally through rows and headlands using feedback controllers based on exteroceptive sensing, as well as to detect transitions between an orchard row and its neighboring headland. This navigation strategy appears to be very well adapted to orchards for several reasons. First, as a reactive control strategy is used, there is no need for building an orchard metric map or for localizing the robot within it. This is of great interest in our context. Indeed, embedded sensor-based localization can lead to inaccuracies with time and obtaining an up to date map of an orchard can be challenging because of its changing nature (branches, pruning, fruits, …). Next, the position of the trees appears as a relevant and robust feature to navigate safely within the rows and headlands. From them, it is possible to deduce the lines the robot has to move along to efficiently follow a row. Moreover, in previous works, it has been shown that the last trees of the rows can be used as references to navigate in the headlands, i.e., to perform U-turn maneuvers allowing the robot to transition from one row to the next one. Moreover, in commercial orchards, it appears more suitable to use tree trunks as characteristic features for navigation rather than to compute the free space in front of the robot and use it to determine a path, as it is done for example . One reason is that this computed free space can be severely distorted when branches are present in the field of view of the sensors or the non-uniform distribution of the branches around the tree. Such a distortion can cause path deformation and affect navigation robustness. Another reason is that free space in the headlands does not provide any valuable information to find the next row and reach it. Thus, the knowledge of the trees positions appears to be a key feature in order to perform orchard navigation. It has then be decided to provide the robot the ability of visually detecting trees.
PROPOSED SYSTEM :
This paper presents the design of a tree detection algorithm aiming at detecting the trunks of the surrounding trees and providing their position with respect to the current robot frame. The algorithm processes the range component of point cloud data acquired by low-cost stereo vision systems in real time. The adoption of low-cost sensors makes it possible to eventually use several of them to provide a large field of view. Detecting tree trunks using only range data makes the approach more robust to changes and discontinuities in illumination, which can be severe in orchards. The proposed method operates by detecting the empty spaces (”shadows”) in the point cloud created by the trees. Indeed, when processing data from a single view of a stereo vision system, it is not possible to compute 3D points behind the objects present in the scene. The empty spaces can then be seen as the shadows of the trees in the point cloud. Figure 1(c) presents an example of a single point cloud acquired in an orchard. The empty spaces are represented by the orange triangles, whereas the red circles represent the trunks of the trees. The proposed tree detection algorithm detects the tree shadows (orange rectangles) and their origins (red circles) in the point cloud, and can be briefly described as follows. First, the 3D points belonging to the ground are extracted from the point cloud. Next, these points are used to create 2D grid maps expressed in polar coordinates. As it can be seen in figure 1(b), the computed points form a concave shape, where the shadows of the trees correspond to the concavities1. The bottom point of a concavity corresponds to the trunk. In the second part of the tree detection process the concavities and their bottom points are detected, and concavity areas are estimated. Figure 1(b) suggests that it may be possible to represent the top boundary of the polar grid map by a smooth function. If that were so, finding the bottom of a concavity would be equivalent to determining its local minimum by using a gradient descent. However, as it can be seen in figure 1(d) where the green line represents the boundary, branches and irregular trunk shapes in the camera field of view create shapes in the grid map with boundaries that cannot be represented by single-valued functions. For this reason, this paper presents a novel concavity detection algorithm relying on 2D grid maps at different resolutions that takes advantage of the upward orientation of the concavities. Moreover, the algorithm does not filter the shape in the grid map in order to avoid shape distortion that would result in inaccuracies in the estimation of the trunk position. Instead, it processes the non-smooth contours of the shape which are the result of noise introduced by the low-cost sensors. Finally, the computed bottom points of the concavities, e.g., the trunks, are filtered using a density map to deal with the false detections. By combining the ground extraction process with the concavity detection algorithm, a robust and efficient method is obtained to estimate tree positions. Moreover, its implementation allows its use for realtime control, as the obtained computation times are smaller than 15 ms (see experimental section).
Thus, the proposed method allows to robustly and efficiently provide the features which are required to perform exteroceptive feedback based navigation in orchards, both inside rows and in headlands.
CONCLUSION
In this article, a robust, quick and accurate perception system dedicated to tree detection in orchards was developed. It proposes a vision-based system made of four low cost stereo cameras which provide a dense representation of the agricultural environment around the vehicle. In our method, the point cloud is processed to detect trunks instead of the whole trees to avoid considering leaves or branches which might reduce the result accuracy. To do so, the key idea is to find the tree trunks by detecting their shadows which are materialized by concavities in the obtained point cloud. The designed algorithm is made of two steps, respectively consisting in extracting the ground and detecting the trees. This latter objective is fulfilled thanks to an original algorithm whose execution time is around 1ms, allowing its use in a control context. The pipeline was implemented on our robot using ROS middleware and validated using various sequences of data taken in different orchards. The obtained results have been proven to be quite satisfying. A comparison with other approaches has also been proposed. Our future works will be mainly orientated towards the coupling of our perception system to our control strategies. Indeed, the positions of the trees are now available with a sufficient accuracy to compute the control laws. Then, it will be necessary to build a topological map of the environment. Finally, it will remain to sequence the row following and U-turn controllers proposed in depending on the robot position in the orchard to complete the navigation task.