The conversation with Xue Wuye had clearly affected Zhang Ruochen. Since Xue Wuye could guess that he had an uncommon relationship with Chi Yao, then other people must be able to guess it too. It was just believed that Zhang Ruochen had died, so they didn't look too deeply into it.
But once news spread that Zhang Ruochen hadn't died, people would definitely start digging for the secrets. In the end, they would start investigating the Crown Prince of the Sacred Central Empire from 800 years ago.
Xue Wuye was smart. He wouldn't do things that would hurt himself. If he wanted to spread the news, he would've done so long ago instead of waiting until now. It was possible that he still wasn't sure if Zhang Ruochen was friend or foe.
Thus, Zhang Ruochen wasn't very worried. He would just go with the flow and there would be a result in the end.