Unlike Blake who was not bothered by the ranking, Castiel and Brian were in tougher positions.
Castiel was the 23rd in ranking and seeing that the top twenty were like unmovable obstacles, he was constantly being challenged. If not there was a limit to the number of times he could be challenged (AN: see the briefing chapter or the one before for more info.), he would have been swarmed with challenges.
Castiel had awakened the darkness element as his second element, but kept it a secret, he used his fire element and well-concealed attacks from the darkness element to defeat his opponents. Although he was strong enough to be in the top 20 if not the top 10, he decided not to because he wanted to get experience from handling those below.
As for Brian, his situation wasn't that great. Due to the fact he won his opponent by outsmarting him and not with much of his power, he was placed in the 40th position.