When the others heard the question of the Second Prince, their expressions changed slightly. If it's true, and Klaus and Reynolds were delaying them for Grey's arrival, then it spelled trouble for them. Klaus and Reynolds were already troublesome to deal with on their own, if Grey were to be added to the fray, then they would be in serious danger.
"We have to leave. I don't care if it's just me overthinking or whatnot, I can't take the chances of having to battle with him, especially when Elric isn't around." The Second Prince was not willing to be attacked by Grey.
Just like the Seventh Prince, the Second Prince has a phobia when it comes to Grey, he doesn't want anything to do with the freak of nature.
If Grey knew the Princes were this scared of him, he would be flattered, to say the least. It was quite satisfying to know that the mortal enemies of humans seem to have a deep fear for him. With this, he could walk with his head held up high.