Skip to main content

Interpretation guide

Please find guidance on how to interpret the following terms used in the tool:

  • Confidence intervals

    Confidence intervals show the natural variation that would be expected around a rate.  The wider the confidence interval, the more the rate would be expected to vary over time by chance alone.  If the confidence intervals are relatively wide, this makes it harder to draw conclusions about the difference in rate between area X and area Y, or the change in area X over time. 

    The size of the confidence interval is dependent on the number of events occurring and the size of the population from which the events came. Generally speaking, rates based on small numbers of events and small populations are likely to have wider confidence intervals, indicating greater uncertainty. Meanwhile, rates based on large populations are likely to have narrower confidence intervals, indicating greater precision. 

    In the Public Health Outcomes Framework reporting tool, we use 95 per cent confidence intervals. This represents a range of values that we can be 95 per cent confident contains the ‘true’ underlying rate within the population.  

  • Making comparisons: statistical significance

    Comparisons are often made between two or more estimates, for example between different areas or time periods (Figure 1).  In such cases, we need to test whether the differences are ‘statistically significant’, i.e. unlikely to be due to the play of chance alone. 

    There are different ways of testing for statistical significance: 

    1. Comparing the confidence intervals of the estimates to see if they overlap. Non-overlapping confidence intervals are considered to be statistically significantly different (Figure 1b). Whilst it is safe to assume that non-overlapping confidence intervals indicate a statistically significant difference, overlapping confidence intervals are inconclusive (Figure 1c) and would require a further statistical test as described in point 2 below.

    2. Calculating the ratio of the two estimates, or the difference between them, and constructing a test or confidence interval around that figure. Such methods are not covered in this technical guide, but can be found in a standard textbook. 

    3. Observing whether the confidence interval of the local area estimate (e.g. Carmarthenshire) touches the estimate for Wales (Figure 1a), which is treated as a ‘Target’ value.  This is the chosen method for establishing statistical significance within the ‘Comparison table’ section of the PHOF reporting tool, due to being relatively simple to calculate and to understand. 

Figure 1. Using confidence intervals for making comparisons a) & b) Non- overlapping confidence intervals are considered as statistically significant c) Overlapping confidence intervals do not always indicate a difference that is not statistically significant

 

  • Local and national deprivation fifths

    The Welsh Index of Multiple Deprivation (WIMD) is an official measure of relative deprivation for small areas in Wales and a National Statistic. It identifies areas with the highest concentrations of deprivation. Ranking these areas, and dividing them into five equally sized groups produces fifths of deprivation.

    For all national level analysis and most of the analysis at health board and local authority, fifths are calculated at the Wales level (national fifths). There are some indicators where local fifths are used, specifically the life expectancy and healthy life expectancy. Local fifths differ from the national fifths in that the five equal bands of deprivation are recalculated just for the small areas within each health board and local authority boundary, rather than inheriting the national fifths. This is useful for a more localised approach to producing health expectancies.

  • National Indicators (NI) represent the outcomes for Wales, demonstrating progress towards seven well-being goals. There are 50 NI's in total, ten are reported in the Public Health Outcomes Framework.