I thought of it as the peice of Voldemort's soul that is in his body not the peice that was on harry because the peice on Harry was killed so it wouldn't have gone in the limbo with him.