The villages had a population of about 200 people or so, out of which there were 80 men but...
With all these people having adapted to wildlife, even the women and children could defend themselves.
Though they fought wildly without any martial arts postures, their attacks had enough strength to take down even those black belt martial artists who had just trained under regulations and would likely be kicked and beaten to death if these people confronted them.
And despite the number of guards, being surrounded by the villagers posed a grave serious risk. Escaping was one thing, but doing so without hurting them several—now that was a problem. They might be able to escape but they can't hurt them badly.
The gears worn by Evan's men protected them from getting injured by the attack of the villagers, but if dozens of them were to get hold of one and tried to twist their bodies, it would cause a problem.