I suspect you’ll get a whole load of comments telling you to just bloody wear it and not worry about scratches but I can sympathise. Where I have stretched the budget, I tend to baby the watch, which limits my enjoyment.
Ultimately, if you ding a £300 watch, that’s maybe £50 off the value, if you choose to sell it. If you ding a £20 000 watch, that might be a grand or two.
Where your comfort level lies depends on you, your attitudes and your circumstances.
If you can afford to keep it for special occasions and accept you might only wear it occasionally, keep it. If you think you’d rather have a few more affordable alternatives, each of which you can enjoy more freely, that might be the better route. There’s room for both approaches.
I have one watch which cost twice anything else in my collection which I wear very selectively. Others may scoff but that works for me and in a way, adds to the ceremony of a special watch.
YMMV.