Although he had a great deal of respect for Xin Yun, even admiration, Speedy Slash had no immediate plans to follow him. Xin Yun's performance was not yet convincing enough for him, after all...the last time Xin Yun instantaneously killed Ke Zhan, Speedy Slash was not present. Even though Xin Yun had once defeated him, if mere defeat was grounds to follow someone, then he would have already become a follower of Ke Zhan, and Xin Yun would not have had a chance.
While both Speedy Slash and Ke Zhan came from major families, they were not direct heirs with inheritance rights. They had money and power, though not in abundance, which could hardly satisfy anyone's desires. To make a name for themselves, they had to rely on their own efforts.
Every man is the same, each harboring ambitions to achieve greatness. No one is content with mediocrity; it's one thing if you lack the ability, but when you possess great power, the urge is even more pressing.