'1 in 5 Muslims': How not to do a survey

The Sun have an alarming – some might say incendiary – headline on its front page today:

1 in 5 Brit Muslims’ sympathy for jihadis

There are two aspects to the report by political editor Tom Newton-Dunn that suggest the figure is unlikely to be accurate.
The first is that the report uses the words ‘poll’ and ‘survey’ entirely interchangeably.   In political polling circles this is a massive no-no.
A ‘poll’ refers to a series of questions asked of a group that is representative of the population.  A ‘survey’ is simply when a group of people are asked their views.  Often the people who answer are self-selecting.
(Updated because of this comment) Polls tend to be conducted about voting intentions, whereas surveys tend to be conducted about a specific subject among a specific audience – either a representative sample of the general public, or a sample of a specific population (for example, of a company’s customer base).
A poll is more difficult and expensive to carry out than a survey, because the polling company must take care to ask enough people to get a statistically significant result, and indeed ask enough people in the relevant demographics to ensure that the results can be weighted appropriately.
Polling companies such as YouGov or IPSOS-Mori make a big deal about differentiating between a poll and a survey.  Neither will allow you to describe a survey they have conducted as a ‘poll’.
This really matters because, as every schoolboy and political journalist knows, a less rigorous process can deliver inaccurate results.  In the present example, if only Muslims living in a particular area were sampled, or a particular branch of the faith, then the results would certainly not be representative of the wider Muslim population.  Indeed, any figure showing ‘Muslim’ support for ISIS is innumerate in itself, because presumably none of the circa 140,000 Shia Muslims living in the UK will have any sympathy for ISIS, which regards them as heretics.
Did The Sun interview any Shias?  How did they ensure a representative sample of the Muslim population?  Did they include people who attend a mosque weekly or anyone of Muslim heritage, cultural Muslims who have otherwise become as areligious and as ambivalent about their God as the rest of the UK?
What was The Sun’s methodology?  The article does not say.  Nor does is explain whether the survey was conducted by a respected polling company, or internally.
Precision and rigour in public polling are highly prized.  When it is done right and the results are significant or surprising, those publishing the poll always publish their methodology alongside, because it adds significant credibility to their findings.
The fact that Tom Newston-Dunn’s report is ambiguous over the nature of its questions suggests that the paper has conducted a survey and not a demographically balanced poll. The caginess over how the polls was conducted, and failure even to give the sample size, reveals that the figures have none of the mathematical or statistical rigour that such an important issue demands.  Tom Newton-Dunn and the editors of The Sun newspaper are either ignorant of this (highly improbable) or they simply don’t care about statistical accuracy. Either possibility is appalling.
But they have published on page 1 anyway.  At best, this is terrible journalism.  At worst, it is a conscious attempt to demonise a minority by presenting the results of a biased survey as representative of the entire population.
This article has been published in order to sow fear and division.
Do not believe these statistics. Do not be scared.

Update

Since I wrote this it has emerged that the survey was carried out by Survation and they do appear to have used a clear methodology.  Full details of the figures, weighting a and actual questions asked are available.
However, my point about The Sun’s caginess over the survey appear to be well founded.  In statement, Survation has distanced itself from the newspaper’s interpretation of the survey results.
It is worth quoting at length:

Our view remains that the most meaningful way to interpret the results of this polling is in the proper context alongside a comparable sample of non-muslims, as we did in March of this year using identical methodology and the same question wording.
This comparison shows that “sympathy with” (distinct from “support for”) those travelling to fight in Syria (among any group) exists as a limited, minority view among both muslims and non-muslims, particularly among young people of both groups.
…
This latest poll in fact shows a fall in sympathy with fighters travelling to Syria among Muslims since March, something which we would consider the most pertinent new finding of that particular question.

Emphasis added.  The company also alludes to their work being used to incite religious tensions.
Having read this, I would describe The Sun’s report as willfully misleading and inaccurate.  The overall message that the report a seeks to convey has been debunked.  However, I do wonder whether it will be censured by IPSO as a result. The survey does use the word ‘sympathy’ in its questions about people travelling to join ISIS and The Sun has reported that (albeit out of context).