What is "Optical Flow"?
The goal of this page is to illustrate optic flow and how it is useful, with the general reader as a target audience. Our goal is a simple, visual, and conceptual explanation.
Optic flow defined
The term "optic flow" refers to a visual phenomena that you experience every day. Essentially, optic flow is the apparent visual motion that you experience as you move through the world. Suppose you sitting in a car or a train, and are looking out the window. You see trees, the ground, buildings, etc., appear to move backwards. This motion is optic flow. This motion can also tell you how close you are to the different objects you see. Distant objects like clouds, and mountains move so slowly they appear still. The objects that are closer, such as buildings and trees, appear to move backwards, with the closer objects moving faster than the distant objects. Very close objects, such as grass or small signs by the road, move so fast they whiz right by you.
There are clear mathematical relationships between the magnitude of the optic flow and where the object is in relation to you. If you double the speed which you travel, the optic flow you see will also double. If an object is brought twice as close to you, the optic flow will again double. Also the optic flow will vary depending on the angle between your direction of travel and the direction of the object you are looking at. Suppose you are travelling forward. The optic flow is the fastest when the object is to your side by 90 degrees, or directly above or below you. If the object is brought closer to the forward or backward direction, the optic flow will be less. An object directly in front of you will have no optic flow, and appear to stand still. (However, because the edges of that forward object are not directly ahead of you, these edges will appear to move, and the object will appear to get larger. This is discussed more below.
Optic flow as seen from a bird, insect, or aircraft
The figure above shows what the optic flow might look like from an aircraft flying over a rocky desert (or over Mars!). The blue arrows show the optic flow that would be seen by a camera or a passenger on the aircraft. Looking downward, there is a strong optic flow pattern due to the ground and rocks on the ground. The optic flow is fastest directly below the aircraft. It is especially fast where the tall rock protrudes from the ground. A sensor on the aircraft that responds to optic flow would be able to see this optic flow pattern and recognize the presence of the tall rock. The meaning is clear: "Look out below!!!"
Looking forward, there is another optic flow pattern due to the upcoming rock and anything else the aircraft might be approaching. The blue circle directly at the center shows the "focus of expansion" or FOE. The FOE tells the aircraft the specific direction it is flying. (Remember above we said that if you are travelling in a straight line, the optic flow is zero in the directly forward direction.) The aircraft sees a large optic flow to the right of the FOE, which is due to the large rock on the left-hand side of this picture. The aircraft also sees smaller optic flow patterns in the downward-front direction, due to the ground. Towards it's upper left, it sees no optic flow because this region of the visual field only has the sky. The forward optic flow pattern reveals that the aircraft will fly close by the big rock, perhaps dangerously close. If the optic flow on the aircraft's right grows larger, then the aircraft should take that as a hint to turn away...
Here are some other examples of optic flow: The figure below shows an overhead view of a dragonfly, and the resulting optic flows in the sideways direction. (We are ignoring the vertical direction in this figure). The figure on the left shows the dragon fly travelling forward. The optic flow travels from the forward to backward direction, and is generally faster on the left and right than in the front or back. The figure on the left shows a dragon fly rotating to the right in one spot. Here the optic flow is to the left in all directions. If the dragonfly were flying a curved path, the optic flow patterns would be a combination of these two patterns. What we see here is that in addition to detecting obstacles, the optic flow can be used to measure or estimate one's own motion.
The figure below shows the same type of patters, this time from the point of view of the dragon fly, and with the 360 degree field of view flattened onto the screen. When the dragon fly is rotating, or yawing, to the right, the optic flow everywhere will be to the left. However in the directions along the axis of rotation, the optic flow will be zero. When travelling forward, the optic flow will diverge from the forward direction, flow backwards, and converge in the rear.
How can sensing optic flow be useful when flying through the world?
There is quite a bit of scientific research that has been performed on how animals and insects use optic flow to navigate through the world. There is certainly too much to summarize on one web page! However let's consider a few more examples...
Hovering: Suppose you want to hover in one place. The best way to do this is to try to keep the optic flow zero everywhere, as shown in the figure below. Note that this method will only work if the rest of the world is still. If the world is moving, for example if you are a small insect flying in between branches that are swaying in the wind, then you will end up swaying with the branches (not necessarily a bad thing).
Avoiding obstacles: The last thing any flying animal or aircraft wants to do is to crash into an obstacle. Below are some examples of a dragonfly detecting and avoiding a large obstacle by sensing the optic flow. On the left, the dragonfly is dangerously close to the rock. The large optic flow on the left warns the dragonfly, and it turns away to avoid a collision. On the right, the dragonfly senses it is flying towards a rock. The clue here is that the focus of expansion (FOE) is inside the rock, and the optic flow is expanding rapidly.
Many flying insects will actually make rapid turns when changing directions. These rapid turns are called "saccades", after the rapid eye motion that your eye makes as you shift gaze from one direction to another. Many insects, however, do not have eyes that can move independently of the rest of the body, or can do so with limited freedom. As a result, if the insect wants to look "facing" in a new direction, the whole body must be shifted. The resulting rapid turn executed by the insect is often referred to as a saccade.
Speed control through dense environments: In some cases, you may want to change your flight speed so that you fly slower when in a cluttered environment. A simple method of doing this is shown in the figure below. If the optic flow all around you is getting too fast, you simply slow down until the optic flow is at a more "comfortable" level. The dragonfly on the left is traveling between two rocks, and has a sufficient "space cushion" between it and the rocks. The dragon fly on the right is traveling in a tighter space, and therefore must slow down it's flight speed to keep the optic flow at a more comfortable level.
More examples: We can put up more pictures, or refer you to books or journal articles. However, a great way to learn is through hands-on experience. So here is my assignment to you: Next time you walk across the room, down the street, or ride in a car or train, take a moment to look at the optic flow patterns that you see. (Don't do this while you are driving!) What do you see when you turn? What do you see when you walk down stairs? What do you see as you pass under a tree? You will very quickly see a wide variety of patterns, and get a sense of just how much information you can get from optic flow or visual motion.