1. When returning Ether into mana (both in everyday use, but even for instance when an Eternity Sword or another entity of pure mana is destroyed), a portion of the mana is lost permanently from the world (Youtia explains this and even Yuuto is aware of it afterwards). Hence spirit warfare will eventually drain all the mana=life of the world. It's a long term plan yes, but automated, suitable for Eternals.
2. 'Rebirth' is acting as a giant sponge absorbing that lost mana.
3. Phantasmagoria is sealed by the Guardian Dragons, preventing Eternal entrance. Presumably the seal wasn't strong or specific enough to block all kind of interference, and they're able to ship in certain Eternity Swords - at least 'Rebirth' acting as an automated factory, soon followed by the Four Warriors (can't really say if their swords were handpicked because of their personality and shipped from the outside, or perhaps already existed in Phantasmagoria and summoned their wielders, there's not much info about this period).
4. Direct entrance by Eternals becomes possible presumably only after the possibly last(?) sane Guardian Dragon Sadgaraham is defeated (they're not called Gatekeepers for nothing) - certainly another long-term objective of introducing the strong fighters into the world (direct quote by Sadgaraham: "Very well. Do as you will. You defeated me. Perhaps this means that the gate has opened......?"). After that events speed along since Eternals upon arrival have the power to simply annihilate the resistance and return all to mana by pure force.
Where all the mana goes now... hopefully the world again, but not even Yuuto knows (he wonders about that himself too).
Story Qs - Origin & Restraints on Spirits [MAJOR spoilers]
Re: Story Qs - Origin & Restraints on Spirits [MAJOR spoiler
Napolitan.... Rush!!