Well, I've heard of a census worker who had to deal with a very hostile resident. The resident thought the census was some sort of government intrusion and started yelling and threatening the worker. The worker was really scared as they were just doing their job. Also, there are stories of census workers having to go to remote areas where their vehicle broke down on the way back. They were stranded in a desolate place with no immediate help available, which was a nightmare.
A census worker once shared that they went to a large, old mansion. The place looked really spooky from the outside. When they knocked on the door, an old woman opened it very slowly. She stared at the worker without saying a word for a long time. Then she started whispering strange things about the house being haunted and how the census was disturbing the spirits. The worker was so freaked out that they just quickly noted some basic information and left as soon as possible.