Index
Home
Next: Cameras
2 How the System Works
The heart of the system is the module that holds the two cameras. In order for stereo video to be possible, two cameras must be synchronized. The secret lies in the fact that many cheap board cameras can be synchronized together.
A stream of NTSC video is composed of frames, which in turn are made up of fields. One field will contain the odd-numbered lines of the display, and the other field will contain the even-numbered lines. The lines from the two fields interlace with each other to create a complete picture on a television set. The two fields that make up the single frame are generated separately. Two pictures are displayed on the picture tube in such a way that the lines of each field fills in the spaces between lines of the other field. This method of video display is called interlacing. All television sets use interlacing, and at one time computer monitors were also interlaced.
Computer monitors no longer use interlacing, and the entire picture is displayed in a single field. So a television displays 30 frames per second, and a VGA monitor displays 60 frames per second. For the rest of this booklet, the difference between frames and fields is only important when speaking of the video stream from the camera to circuit board that combines the two streams. After that, once the two streams have been combined, frames are the individual video pictures that make up the stream, alternating between left and right eyes.
2.1 Synchronizing video cameras
Two cameras can be considered synchronized when both are generating video using the same timing. This means the vertical sync pulse from each camera occurs at the exact same time. Synchronization is necessary so that when the field generated by one camera has been sent out, the other camera is starting the next field.
Figure 1 shows what happens as the two video streams are combined. The individual fields emerging from the left and right cameras can be labeled L1, L2, L3, ... and R1, R2, R3, and so on. To combine two video streams a circuit will switch every other field from each camera, into the outgoing stream, which then becomes a series of alternating fields, one from each camera. The result will be L1 from the left camera, R2 from the right camera, and so on. You should be able to see that it is not possible to combine both L1 and R1, because they represent fields that are being generated at the same time.
Figure 1a: Combining video streams
The single idea central to this system is that many cheap board cameras are available that provide a field sync output, as well as a field sync input. If you inject a positive TTL-level pulse on the sync input to the camera, you will reset the camera to the beginning of the frame. Therefore, if you connect the sync output of one camera to the sync input to the other camera, you synchronize the cameras.
Note that this form of synchronization does not put the two cameras into lockstep. It simply forces one camera to start generating video at a time dictated by the other camera. Timing differences that can still occur at the line or pixel level should be negligible.
Stereo ranging describes the process of using video from two or more cameras to help identity objects and their spatial relationships. It is used in medical instruments, in ocean floor mapping, and in vehicle navigation systems. NASA, for example, uses stereo ranging on the Mars Pathfinder rover to determine landing site topography and identify obstacles. The video camera pair (or set of cameras) and any controlling electronics is called a stereo sensor head or stereo ranging sensor.
The sensor head consists of the stereo camera module that has been specially configured to be used for stereo ranging. The video that is generated in this mode is not useable for viewing. While it was not tested on a computer monitor, when applied to a television it showed a picture distorted by the receiver's inability to synchronize on the input. This is because the input video consists, not of a standard NTSC frame, consisting of alternating odd and even fields, but of a series of odd fields, with no even field. Of course there's no guarantee all televisions would have that problem.
The output of the sensor head is normally connected to a frame grabber, which digitizes and stores the fields. Software (like the mounting base, is beyond the scope of this book) will then analyze the data, line by line, and provide information regarding objects and distances. A list of sources of software for stereo ranging is provided in the appendix.
The idea behind the stereo ranging sensor consists of two parts. First, each camera shows the same view of the scene. Because the cameras are separated by about two inches, the two captured images are rotated a few degrees away from each other. In fact, it is this disparity between the two images that is analyzed to provide information about depth.
Second, the video that is captured is the same field from both cameras -- the odd field, and that the two cameras are so well aligned that the same line in each left and right field holds exactly the same information, differing only by the angular difference between the two views.
Figures 1b and 1c show how the camera module's two video modes differ. Figure 1b shows the odd and even fields alternating to generate a frame that can be displayed. The timing of the right camera is controlled so that its odd field coincides with the left camera's odd field.
Figure 1b: Odd/Even Fields for Stereo Display
Figure 1c: Odd Fields for Stereo Ranging
Figure 1c shows fields are still selected from each camera alternatively, but the right camera has had its timing adjusted so that its odd field coincide with the left camera's even field. As a result, only odd fields are included in the video stream.
The camera mounting scheme that is described in section 5 is intended only for use with the stereo display mode. Those who wish to operate the module as a stereo sensor should find a better mount for the cameras.
There are four configurations based on display type: television or VGA monitor, and whether the camera module is local (direct cable link) or remote (wireless link). These four configurations are described next.
2.3.1 Local Camera with Television
Figure 2 shows the most basic configuration, which consists of the camera module, a television receiver, an RF modulator, a pair of LCD shutter glasses and a "TV dongle". The shutter glasses connect directly to the camera module, which provides the signal needed to control the glasses. The TV dongle is just a 25-pin female D-sub connector that imitates the parallel port on a PC. It applies the shutter control signal from the camera module to pin 4 of the connector. This is normally one of the bi-directional data bits on the parallel port. The shutter glass adapter (the one that comes with the glasses), then picks up the signal from that pin and uses it to control the glasses. The TV dongle is described further in section 6.
The cable that carries video from the camera module to the RF modulator should be short, just a few inches. The camera module uses an RCA connector for video out. The RF modulator may have an RCA jack, a coax connector, or just two points to solder to (like the one used in this project). A 75 Ohm coaxial cable should be used to carry the video signal from the RF modulator to the television's cable input connector.
Figure 2 Local Camera with Television
2.3.2 Remote Camera with Television
Figure 3 shows a camera module (and stereo microphone) connected directly to a radio transmitter. The video output of the receiver is sent to the RF modulator where it is modulated onto a channel 3 or channel 4 TV carrier.
The video is also sent to an adapter called the shutter dongle. Among other things, this adapter will generate the odd/even shutter control signal from the received video. The shutter dongle is described in section 6. (The stereo audio output function was not implemented.)
Figure 3 Remote Camera with Television
2.3.3 Local Camera with VGA Monitor
Figure 4 shows a camera module connected directly to a VGA converter (also called a line doubler), which converts the NTSC video to VGA format video.
From there it is sent to a VGA monitor. One of the signals generated by the converter is vertical sync. This signal is sent to VGA dongle, which uses it to create the shutter control signal. The circuit that creates the control signal is essentially the same shutter control circuit that is on the camera module.
The shutter control can also be taken from the camera module itself; in other words, instead of tapping off the vertical sync with the VGA adapter, just connect the TV adapter between the camera module and the shutter glasses.
The VGA Adapter is described in section 6.
Figure 4 Local Camera with VGA Monitor
2.3.4 Remote Camera with VGA Monitor
Figure 5 shows a camera module (and stereo microphone) connected directly to a radio transmitter. After the video is received, it is sent to the VGA converter, which generates the video for the monitor. The vertical sync output of the converter is used by the VGA dongle to generate the shutter control signal.
Other than the transmitter and receiver, the other difference between Figure 5 and Figure 4 is the microphone included with the camera module. The transmitter and receiver set can often carry stereo audio along with the video.
The transmitter and receiver are described in section 8.
Figure 5 Remote camera with VGA Monitor
Index
Home
Next: Cameras