Qu Xing Xu knew that the Bai family thinks that he should be thankful for their efforts in the past for making him the successor as the next duke when the title could also have been passed to others. There had been too many people who were eyeing the title as a duke for many years and since he was just a young boy that time who had no real power, one would think he wouldn't be suitable to succeed.
That was also what the Qu elders thought at that time.
Although the elders weren't working with the Bai family and even hated that ridiculous family, they never considered Qu Xing Xu as the next heir to the title despite him being the son of the previous duke. After all, what could a little boy like him do? Will he even be able to grow up valiantly like his grandfather? Or maybe, he will be just like his father, a waste of a man and time and breath.