Most likely it is because you have students who have already attempted the quiz.

You will need to delete all of the attempts by selecting them and choosing to delete them if you need to add/remove questions from a quiz.