A task system is not something that makes your decisions for you. It only supports you in the decision of what to do next. The difference between these two positions may seem small, but it is, in fact, quite vast.
Mistaking the system for the former will inevitably make it irritating and a constant reminder of everything you wish you could do. Viewing the system as the latter is much more freeing, as you recognize you are setting everything else aside to focus on something now.