The further back Daniel's memories reached, the less cohesive they would be.
When his thoughts reached his childhood during the first era of true humanity, Daniel could barely see what was a true part of the memory and what was the effect of his brain filling in the missing details.
The speed at which the visions changed from one to another accelerated as well, making it hard for Daniel to actually keep his awareness of what his true self was.
One moment he would be back to being the innocent teenager in the times of the greatest prosperity he ever witnessed. The next moment he would be struggling to keep himself together, unsure of what to expect once his memories would go back just a few years more.
But time didn't wait for anyone. And for someone who ran out of time in a way only an immortal like Daniel could, this argument held on even more.