The answer to this question depends on the user. If I was to be asked, answer would be no. Of course you don't have to play on Ultra, and by Ultra I mean the absolute maximum settings.
Personally I think, and asking some other players/friends confirms this, most cannot tell a difference between some setting set on Med/High and Ultra. If you can and want that, hey, more power to you. But a lot of the times, and I mean a lot, a particular Ultra setting tanks the framerate while offering no discernible visual quality upgrade.
In short, if I have to look for the visual difference, and not outright see it without seeing clips/stills of a scene side by side then as far as I'm concerned the setting does not matter.