As panic began to set in across the neutral territory after the Soul Ascension Realm city lord informed his friends and they in turn spread the news to their friends, Xuan Hao and Pei Yang travelled for a few weeks before arriving at the border to the small kingdom with the Earth Immortal Realm ancestor.
Although they could go around the kingdom and avoid possibly offending the ancestor of the small kingdom, doing so would add another few months to their journey, as they would be forced to pass through some dangerous regions that could even pose a threat to someone in the Earth Immortal Realm.
With this in mind, it was far more preferable to go through the small kingdom where they only had a small chance of running into the Earth Immortal Realm ancestor. Not to mention, even if they ended up running into the Earth Immortal Realm ancestor, they had no prior relationship with him and didn't have any reason to fight even if they ran into each other.