No, it’s about your relationship to that person. Are you on friendly terms? Are they comfortable around you? Do you have some kind of established rapport?
Or are you a complete stranger making a weird comment about another strangers physical appearance out of nowhere?
I believe their reactions might be slightly different depending on the person, but if you simply asked them if it made the comment made them uncomfortable they’d say yes 100% of the time.