It's a tragedy when a simple gesture of lowering one's gaze has two different interpretations. He was scared that she would ran away, if she caught that feelings in his eyes. She was scared of his intentions, fearing that he's only lusted for her body.