With WatchOS2, developers can now create custom Complications that provide 1-click access directly from the watchface. I think you should create a Complication that takes you directly to your "Add task" functionality.
My desired behavior would be to add a "+" on my Watchface and when I think of something to do, I just click the plus. That takes me to the dictation screen where I can immediately speak what I need to add.
Our development team perpetually works to improve Todo across all the platforms we support. We greatly appreciate you taking the time out to provide us with your feedback.