No....it's fine. Really it's an advantage for you if his QB can't score without you getting points also.
I dont consider it an advantage or a disadvantage. I just look at my team as a whole and put the best opportunity for points on the field and worry about the rest later.