While the girls were getting along with each other, Jackson had long sneaked out of the room because he felt like he was not needed there. Jackson had already heard the answer from Lady Varshir, therefore there was no need for him to stay there and listen to the girls talk.
As for Lady Varshir attacking the girls, Jackson was not worried because the girls can handle a level 6 cultivator with their spirit armor. Jackson also knows that Lady Varshir is less likely to attack them because when they were talking earlier Jackson noticed that even though Lady Varshir acted high and might be back on the battlefield.
Jackson saw many sides of her during that conversation, Sometimes she would be strong and straight forward but at other times she would be shy and embarrassed. In short Lady Varshir looked like a girl who had to carry a lot of burden on her shoulders even though she did not wish for it but had no choice.