Vision
Now that we have precise control over the movements of the robot, we want to find a way to have the robot automatically line up with the tower for a shot. To accomplish this, we are going to need a camera, and that camera is going to need a target that it can recognize. For our target, we are using a special kind of tape called retro-reflective tape. The nature of the reflective surface is that reflects light back in the direction of the source. This kind of reflective surface is actually quite common. Most road signs use it for the lettering so that at night your headlights will reflect off the surface and seem quite bright to you as the driver.
What we are going to do is to shine a bright green light onto the tape, and that green light will be reflected back as a bright image and we can look for that specific color. We can then find bounding boxes for the regions which are the correct color, and us those bounding boxes to determine how our robot is oriented with respect to the target.
The actual program which captures the images and process the color data is beyond the scope of this document and we will be treating it as a black box. What you need to know is that we will be running a program on the Raspberry Pi that will capture the images and give us the bounding box of the target. This program will act as a little Image Server to which we can connect via a network connection to get the data. This Image Server will also provide a way to transmit the actual images to the driver station PC so that the human driver can see the actual image that is being processed. On the PC that is running the Driver Station console, we will run a separate application called the Image Viewer which will connect to the Image Server running on the Pi and display the images that the camera sees.
Before we get into using the image data on our robot, let’s first explore the Image Viewer, and how we can use it to control what the camera sees. The Image Viewer program is a Java program that you can find in the piUtil folder. Since it is a .jar file, you must invoke java.exe with the proper parameters to get it to run and there is a shortcut in the piUtil folder which will do that for you.
Before we can run the Image Viewer we must start the Image Server on the Pi. Currently using the Pi Driver Station, we can launch the terminal application PuTTY which will automatically log us in and start the Remote Launcher. We need to change this so that is launches both the Remote Launcher and the Image Server. To do this, launch the Pi Driver Station and click on the Settings button. Then change the PuTTY Session field to MiniBot Runall and then click OK. Then click on the Launch PuTTY button to start both the Remote Launcher, and the Image Server on the Pi.
Now that you have these running, Double click on the ImageViewerMulti.cmd shortcut to launch the Image Viewer and you should see something like:
This is the view as the camera sees it and you can see that the retro-reflective tape is very bright. In fact, it is way overexposed (it should be much more green). The first thing we need to do is to change the settings so that we can only see light sources. Adjust the setting as shown in the next image:
We can now see our target as a nice bright green. The final step is to adjust the Hue, Saturation, and Value settings so the Image Server can recognize the green image. You can click anywhere withing the green and if you look down at the bottom it will show you the H,S,V values for that pixel. You can then adjust the min and max settings so that those values are included in the associated ranges like shown in the following image:
Once the settings are correct, the recognized green is replace with blue, a bounding box is drawn around the target and green lines are drawn showing the middle and top of the target. In addition red target lines are drawn. The position of these target lines can be adjusted by the panel on the right. Now click the Save button to save these parameters so that the next time we run the Image Server it will use the parameters we just set up.
Next we will find the optimum shooting position for the robot and then we will move the red lines so they coincide with the green lines. Then when we run our program we will know if the red and green lines match up, we can make the shot.
The next step is to find the best shooting position for the robot. Move the robot around in front of the tower until you find the spot where the shot is most reliable. For best results, when making the practice shots, be sure that you hold the spinup button long enough so that the spinner motors are fully up to speed (later we will add code to ensure the correct speed before firing).
Now look at the Image Viewer display:
What we want to do is use the Target Line and Center Line controls to move the red lines until they match up with the green lines (you can check the Small Adjustments box to make smaller adjustments). The final Image Viewer display should look something like:
Now click the Save button to save these parameters.
Since everything we do from this point will depend on the camera being set up exactly like it is now, we will want a way to reset it to this position in case it is accidentally moved. To enable us to do this, we want to record exactly how far the robot is from the tower, so we measure and record the distance from the front of the tower to some fixed point on the robot:
Now if the camera was accidentally moved, we could set it back into position by setting the robot the required distance from the tower and then adjusting the angle on the camera until the red and green horizontal lines once again align.