The young man was subdued by the five of them from the first attack. He couldn't even counterattack after the first time and was forced to only defend.
While the young man was being subdued by the others, Grey and Void had completely taken advantage of their numbers and the young man who was in the Fourth stage wasn't able to hold on against them.
Within the space of only a few seconds that Grey joined in on the battle, the young man was forced back and he couldn't defend himself properly. Although Grey was not in the same stage as the young man, Grey's attack power was terrifyingly strong. Those in the same stage wouldn't be able to take Grey's attack properly.
Void was also attacking, so the young man was in a difficult position.
Void knew not to use his space domain, so when the young man used his domain, Grey was the one who activated his domains, fighting back against the power of the young man's darkness domain.