-----
The Spirits seemed desperate, despite them never being able to voice their opinions, it seems that their patriarchs and matriarchs, the parents of the spirits and divine spirits were able to speak for them. The Spirit King, or the one named "the supreme deity of spirits" was a tyrannical elven lord that had been using the power of spirits to reach all his goals.
Using their power and abusing it with the many other children he had, who all could use the power of spirits, it ended over exhausting the spirits to the point that many began to fall and become ill, some even died. Over the eons since this race of spirit manipulators was born, the spirit quantity across the galaxy began to run lower and lower.
Check "The Protagnist's Sister Is Actually the Strongest"!