Alexander really did not care about the slight Miss Linda threw regarding his family.
Because Miss Linda was unable to even grasp the concept of how Alexander's true family was.
They did not live in Adhania or Thesos, but in a whole other world, where they had access to technologies that made their quality of life even better than all the kings of this world.
People from Alexander's timeline would look at people like Miss Linda as little more than cavemen who fought with sticks and stones.
Thus Alexander found Miss Linda disparaging them laughable.
But although he did not care about the taunt on his family, he did care about the other point- about what he had done to the lady.
That was a fact no matter the reason might be behind it.
And if Alexander could go back, he would have stopped himself despite what the lady had said about his wife.