Searching and Tracking People with Cooperative Mobile Robots

Alex Goldhoorn, Anaís Garrell, René Alquézar and Alberto Sanfeliu

Autonomous Robots

Abstract— Social robots should be able to search and track people in order to help them. In this paper we present two different techniques of coordinated multi-robots for searching and tracking people. A probability map (belief) of a target person location is maintained, and to initialize and update it, two methods were implemented and tested: one based on a reinforcement learning algorithm and the other based on a particle filter. The person is tracked if visible, otherwise an exploration is done by making a balance, for each candidate location, between the belief, the distance, and whether close locations are already being explored by other robots of the team. The validation of the approach was accomplished throughout an extensive set of simulations using up to five agents and a large amount of dynamic obstacles; furthermore, over three hours of real-life experiments with two robots searching and tracking were recorded and analysed.

Videos

On this page videos are shown which give an overview of the experiments done with the Multi-agent HB-PF Explorer to find and follow people. Like explained in the article, the robot has to search and follow the person. The person to follow is recognized using AR Markers. More information about the location of the experiments and the map can be found on the map page

Legend

In the video the experiment is shown in three sections:

1. Left: map and probability maps.
2. Right-top: video focusing on Tibi.
3. Right-bottom: video focusing on Dabo.

The left image shows a map and a probability map:

• Map (left): map as shown by ROS rviz; which shows the following:
  • Dabo: blue body, white head;
  • Tibi: orange body, white head;
  • Obstacles: black and dark gray;
  • Laser detections: blue/orange line/dots (of Dabo and Tibi respectively);
  • Path: blue/orange lines indicate the path already executed by the robots.
  • People detection:
    • Leg detection: shown by blue/orange dots;
    • Last used person location: red dot, this is a combination of leg detection and AR Marker detection.
    • Last detected person location: white dot, this the last detected location using the leg detection and AR Marker detection, which not necessarily is recent.
• Probability Map: shows the robot's probability of the person's location, right top Tibi's, and right bottom Dabo's:
  • Robot self: the large blue circle;
  • Other robot: the small blue circle;
  • Detected person's location: large red circle;
  • Detected peron's location by other robot: small red circle;
  • Obstacles: black squares;
  • Probability matrix: the probability of the person being on a certain location is shown with the colors white to red, the light blue color indicates a probability of 0, white is a low probability and red high.

---

Limitations

Like explained in the article the robots detect the persons in two phases: 1) by laser detection of the legs, and 2) by AR tags. Since the first can only be used to detect people, we added the second to recognize the person. The AR tag should be sufficient, but results in some false positives, therefore we only accept detected tags if there was a person detection by laser close enough. This however still results in some false positive if for example another person is close to the position of a falsely detected tag.
When a false positive detection occurs, the probability for the person being on that location increases, and therefore the robots go and explore that area, but since the false positive normally is detected only for a short time, the probability propagates to other places, and the probability map recovers to the correct area.

---

The videos

We did different type of experiments: exploration, search, and following with static and dynamic obstacles.
The following table shows the statistics of the different experiment types:

	Exploration	Search & Track	Tracking	Total
Distance per robot (km)	1.2	1.2	0.7	3.2
Measured distance person (km)	-	0.4	0.5	0.9
Total time (h)	1.1	1.2	0.9	3.2
Avg. visibility (%)	0	16.3	36.4	15.3
Avg. time connected (%)	95.0	79.5	85.8	86.6
Avg. distance person (m)	-	8.4 ± 6.4	8.4 ± 5.6	8.3 ± 5.9
Avg. number dynamic obstacles	2.0 ± 1.5	0.6 ± 1.3	3.9 ± 2.8	1.9 ± 2.2
Avg. time found (s)	-	106.8 ± 138.7	23.5 ± 42.5	72.9 ± 117.6
Avg. time recovered (s)	-	19.6 ± 39.0	12.0 ± 28.3	15.3 ± 33.6

• Exploration In these experiments we wanted to focus on the exploration behavior of the robots by letting them explore the area without the person being present.
  • Exploration 1: The robots explored the whole environment moving continuously. One robot stood still for a minute because of a problem on the robot, but it recovered soon after that. At the end of the experiment the robots took the same route since their goals were close to each other and the shortest path made them take the same route. Nevertheless they had sufficient distance between each other to explore the route thoroughly.
    
    
    
    
    
  • Exploration 2: The robots explored the whole area, and in this experiment Dabo was not able to navigate alone up the ramp because of the narrow passage and the lasers sometimes detected the floor as obstacle due to the inclined position. Therefore we teleoperated the robot up the ramp.
    
    
    
    
    
  • Exploration 3: The robots explored the whole area and Dabo needed some help to go up the ramp since there was not a lot of space to manoeuvre.
    
    
    
    
    
• Search and Track In these experiments the robots searched for the person and followed him during some time.
  • Search 1:: Dabo found the person and since Tibi also received his observations she also updated her probability map (belief) and went to the person's location.
    
    
    
    
    
  • Search 2: The robots searched for the person, but this time they took longer because they searched the location of the person last. They also crossed each other two times because it was the shortest path to their goal, and the exploration algorithm did not take into account the path. During two times they avoided the ramp because the robots found it to be too narrow, and therefore took a detour. Finally, when the person was found, he was also lost again quickly because the person was a bit quicker. Nevertheless the belief for both robots was propagated for Tibi mainly in one direction and for Dabo in both. Tibi then saw the person again, and in the end of the experiment there was no communication and therefore Dabo did not know where the person was.
    
    
    
    
    
  • Search 3: The robots started in opposite sites of the map and Dabo quickly found the person. However due to some problem Dabo did not move any more shortly after that but Tibi was still going to the person.
    
    
    
    
    
  • Search 4: Experiments with different start positions are shown, when the robot found the person it tried to follow it during some time.
    
    
    
    
    
• Follow and Group Following the person is not that obvious when there are obstacles, especially with dynamic obstacles such as other people.
  • Follow:: The person is followed by the two robots.
    
    
    
    
    
  • Search and follow: The robots start searching for the person, and then follow it for a long time. In the beginning only video with one robot is available. The experiment ended because of an issue with Dabo.
    
    
    
    
    
  • Follow 2: In this experiment the robots start seeing the person and follow the person during the whole experiment. Even though several times one of the robots stood still because of a problem, the other robot kept searching and following the person.
    
    
    
    
    
  • Follow and Search in group of people: Here the robots follow the person while three other people try to walk in front of the person, i.e. they are dynamic obstacles. Nevertheless the robots keep track of the location of the person.
    
    
    
    
    

If you cannot see the (complete) video, please right click it and choose the option Copy URL or Copy Location, and then try to view it with an external viewer (e.g. VLC) or another browser.