"We were married by heart and soul! Nothing else matters." In fact, they felt it was best to not register their marriage. Being lawfully marry wasn't as important for them as long as their hearts were married to each other. And so, they jump to the gun and have children right away as a way to claim each other. But it was later that this was the best method as a way to keep Cao Huo's identity separate and lessened the danger for the children.
Cao Huo was not going to be home that often due to the nature of her background. Tying her to him legally could put their children in greater danger due to the fact that it would be easy for others to trace her to them or vice versa. It was a choice they both agreed upon, as long as their heart remain faithful to each other.