The spirit ship came from the main estate of the Veritas Clan. The Grand Elders directly left from there and picked up each party one by one, giving them adequate time to get acquainted with each other.
The Veritas Clan's Patriarch, Julian Veritas, only had two children. They were the Young Master and Young Lady mentioned before.
The rest of the geniuses of the clan came from collateral lines or were picked up from the outside, but the Veritas Clan didn't treat them any differently.
Their fairness principle applied everywhere. As long as one had talent and ability, one could do anything.
There wasn't even a real "Veritas Main Family." The family of the Clan Head was considered as such, but the Clan Head was always chosen fairly through competition whenever the time came for the previous head to step down.