It's possible that his love for ghost stories was related to his own creativity. Hearing about the strange and the uncanny in these stories could have sparked new ideas in his mind. He could then translate some of that inspiration into his musical works, whether it was the mood, the pacing, or the sense of drama that he borrowed from the stories.
Maybe he was just curious about the supernatural. Ghost stories are full of the unknown and the otherworldly, which could have intrigued him.
He might have liked the way ghost stories were told. They often have a certain rhythm and build - up of suspense, much like a musical composition. The way the tension rises and falls in a ghost story could have appealed to his musical sensibilities. For example, just as in his symphonies where there are different movements and moods, a ghost story has different phases of mystery, fear, and revelation.