What you missed here is the principle of uniform acceleration of the gravitational influence when you are in the presence of concentrated mass.
Huh? Everything falls together at a uniform rate based on the distance from one another as measured by a constant acceleration over time.
Double huh?
Think about it.
