Even though Tree Father Hoffenheim's soul was frozen in place, an aura of a mixture of shock and panic could be felt coming from it... Robin's avatar wasn't affected by the soul domain restriction?!
In theory, his plan was non-failable. The Soul Domain is considered a private kingdom for the soul of its owner, and with some methods and experience, the Soul Domain can suppress any intruding entity. This is why, since Robin entered his Soul Domain, he is a fish on a chopping board.
This is why he did not try to talk to him for a long time at first nor did bother to answer his questions: Why do you talk so much to a dead person?