Well okay, it’s not fine. It is almost certainly not true, it is very rude, it coarsens our political discourse, it widens divisions, and I really really wish people wouldn’t do it.
But when pro-Brexit protesters call a Remain-supporting MP a ‘Nazi,’ that most certainly is political speech and should be covered by free speech protections.
By contrast, it is not okay to physically intimidate anyone, whether that person is is a man, a woman… or an MP.