We leave the physical body behind that is true. Yet we still have an astral body. That body is seen if we have ever left this body or we have had contact with ghosts or spirits. That body we retain and even after that we have a causal body. All these shells must disintegrate before the soul can be free to emerge once again into spirit. When our physical desires become strong enough again we encase our astral and casal bodies in a physical one to find expression for such desire and in the process we create more desires to imprison the astral and causal bodies.
I think this all may be logical conjecture, a strong belief in such specifics may well turn out that way, my bet is that it's a little more complex than this, and may not be as generic for everyone.