Figuring out the obsessions of the bad daughter and good daughter sides of Anna it did not take long for the Southern Ruler to piece one and two together to conclude that most of Anna's current condition was his brilliant daughter's doing, learning this he was heartbroken.
He knew that his daughter's inner demons would someday harm those she loved, he warned her and tried to stop her but ultimately gave up. Now seeing his granddaughter had paid for his inability and failure he was devastated.
What he had feared had become their reality, it was too late. However, he did not give up. He continued to further investigate Anna's origin card. He had failed his granddaughter once and did not want to fail her gain. It was no longer about finding a suitable heir for himself but right by his granddaughter.