Cole's father was very drunk on the day that Adam was scouting Staten Island and upon seeing him, Adam began following him.
William was getting drunk because the next day his son would have the face and reveal himself to the Epstein Family, which was a big and stressful thing that he had been awaiting for 13 years since his wife was killed.
Adam followed him around and he did get into a brawl like he told Angela and Wayde, but although he was a skilled fighter and easily defeated them, he didn't seem to be a cultivator.
But the feeling that William gave Adam was that of a cultivator, so Adam gathered that William's cultivation must have been destroyed or ruined by something, but he still retained the skill and combat prowess that he acquired as a cultivator.
Adam had changed massively from when he was young and with the Disguise Illusion Stone activated, the drunk William didn't recognise him in the slightest and began babbling about everything to Adam.