Charlton knew that if Annalys marries Geoffrey, the Cross ducal family which has always been a neutral party would be more inclined to side with the royal family. If that were to happen, then the monarchist would again hold the upper hand. Thus, even though he told himself that he would never use an innocent bystander to achieve his desired end…
It's not that he has no choice. However, at this point, there was no more turning back. Besides, he was no saint. All is fair in love and war. So, to come out victorious, he knew that he must use everything in his disposal. Though he doesn't see the Cross family as necessary in his plight, having them siding with Geoffrey was something he can not afford.
Either way, he would rather have them on his side despite the fact that he would be taking advantage of Annalys' affection. He felt guilty, but he knew that he was also doing her a favor. At the end of the day, as everything stood, no one knew how things would turn out.