Discover our
resources

Students can teach the robot to follow their hand or a clearly identifiable object.

This can be taken a step further by letting the students' imaginations run wild, creating courses or choreographies in which the robot reacts to various visual cues, such as hand signals.

[Video content coming soon]

‍

‍

Introduction

Place the robot on a table with enough space to move around. You can use an arena if you want to prevent the robot from falling off the table. In this case, a small, unobstructed arena will suffice.
‍

‍

Setting

Connect the robot to the computer, then select the "navigation with camera" example configuration.

We'll use the camera sensor, and the following actions: move forward, rotate right, rotate left, and stop.

Be sure to adapt the speed to the size of the table.

‍

‍‍

Training and implementation (learning phase)

Training will be done in pairs, taking care to be well coordinated.
One of the students will wave to the robot, while the other manages the software.

The aim is to teach the robot to follow the student's hand (or a clearly identifiable object).

In the first phase, one of the students places his hand in the robot's camera field of view, while the other chooses the action that will bring the robot closest to the hand. Give the robot several examples for each hand position in the image.

In the second phase, you'll need to teach the robot to stop when it doesn't see a hand. Again, the robot needs several examples with different points of view: the student in charge of the software clicks on the "stop" action, while the other student places the robot in different locations (taking care not to put anything in front of the camera, of course!).

‍

‍

Learning tests and improvements

Now activate Stand-alone mode.

Check that the robot has learned correctly by observing whether it follows your hand correctly.
If this is not the case, deactivate autonomous mode again and re-train the robot.

‍

‍

Avenues for further development

• Reverse the roles. Does the robot manage to follow the other student's hand or a different object? If not: continue training until the robot is able to follow the hands of both students.
• Place the robot on a course (draw 2 parallel lines on the floor, for example), set it in autonomous mode and guide it along the course by hand. Allows you to evaluate both the robot's training and the student's guidance.
• Train one robot to follow another! The robot in front will be controlled, and the other will have to follow in autonomous mode. Tip: remember to choose a slightly slower speed for the following robot.
• Visualize training data and correct errors. This will improve the robot's behavior.

‍

‍

Review and feedback

The students' mission was to train an AI (supervised learning).

After learning, the robot is able to interact with a user by reacting to hand signals.

By showing the robot examples, it can correctly associate the right actions with the corresponding hand signals, thus "communicating" with it.

‍

‍

Key concepts:

• Training data determines robot behavior.
• Data quality (absence of errors) is very important.
• We also need to provide the robot with sufficiently diverse examples if we want it to be able to generalize the behavior it follows (e.g. following another object or a different hand).