When fast moving air passes over slower moving air, a pressure gradient is formed. The slow moving higher pressure air layer pushes into the fast moving layer. This is the same principle that causes planes to fly; the air moves faster over the top of the wing, creating a zone of lower pressure. The slower moving air under the wing actually pushes the wing upward.
In a drafty house, the fast moving exterior airflow draws the static interior air upward into the stream. This results in interior drafts that have the ability to spread tainted air around the house.