qqconf: An R Package for Putting Testing Bands on QQ and PP Plots

General Plot Features

Parameter estimation aside, we have now created a QQ plot that compares our sample to N(0, 1). While this plot is informative, there are a number of potential modifications we may want to make to this plot for ease of viewing. First, because the top and the bottom of the confidence bands are cut off in this plot, we may want to expand the y-axis limits. We do this by simply adding a ylim argument to the function call:

qq_conf_plot(
  obs = sample, 
  distribution = qnorm,
  dparams = list(mean = 0, sd = 1),
  ylim = c(-4, 4)
)

In fact, because qq_conf_plot takes ... as an argument, any other parameter normally passed to the plot function in base can be passed to qq_conf_plot. If, for instance, we wanted to modify the axis labels, we could write:

qq_conf_plot(
  obs = sample, 
  distribution = qnorm,
  dparams = list(mean = 0, sd = 1),
  ylim = c(-4, 4),
  xlab = "More Informative Title"
)

However, while the ... argument can be used for more general features of the plot like axis titles, because there are so many separate objects in the plot (points, lines, bands), we’ve added additional arguments so that the user can easily specify the visual features of any element of the plot.

Points

If we want to modify the appearance of the points on the plot, we can use the optional points_params argument. This argument is a list that takes any arguments that can be passed to the points function in base. For instance, if we wanted to make the points smaller, we could do the following.

qq_conf_plot(
  obs = sample, 
  distribution = qnorm,
  dparams = list(mean = 0, sd = 1),
  ylim = c(-4, 4),
  points_params = list(cex = .5) # makes points smaller
)

For more options, see the documentation of the points function.

Expectation Line

If we want to modify the line in the plot that indicates a perfect fit between the reference distribution and the sample, we can use the line_params argument. This list can take any parameters that can be passed to the lines function and will modify the plot accordingly. If, for instance, we wanted to make the line red, we would do the following:

qq_conf_plot(
  obs = sample, 
  distribution = qnorm,
  dparams = list(mean = 0, sd = 1),
  ylim = c(-4, 4),
  line_params = list(col="red") # makes expectation line red
)

Note, that if we don’t want this line to show up on the plot at all, we can pass in type = "n" to line_params.

qq_conf_plot(
  obs = sample, 
  distribution = qnorm,
  dparams = list(mean = 0, sd = 1),
  ylim = c(-4, 4),
  line_params = list(type="n") # removes expectation line
)

Again, for more information see the documentation for the lines function.

Testing Bands

By default, qq_conf_plot does not plot pointwise testing bands on the plot. Pointwise testing bands, in contrast to simultaneous testing bands, are anti-conservative, as they ignore the multiple testing problem inherent in QQ plots and simply conduct a test on each order statistic. To add these pointwise bands, one can simply set the plot_pointwise parameter to True in the qq_conf_plot function. To modify their appearance, one can use the pointwise_lines_params parameter. Because this parameter list is simply passed to the lines function, it works exactly the same way as the line_params parameter.

Finally, if we want to modify the appearance of the simultaneous testing bands, we can use the polygon_params parameter. As the name suggests, arguments in this parameter list are passed to the polygon function in base and modify the testing bands accordingly. Note that by default, border is set to NA and col is set to grey. If the user overrides the default option, all defaults of polygon are used unless otherwise specified, which does not set border to NA and col to grey. If, for instance, we wanted to change the color of the shading, we would do the following:

qq_conf_plot(
  obs = sample, 
  distribution = qnorm,
  dparams = list(mean = 0, sd = 1),
  ylim = c(-4, 4),
  polygon_params = list(col = 'powderblue', border = NA) # change shading and keep no border
)

Differencing

Normally in a PP plot, the expected probabilities are plotted on the x-axis and the theoretical probabilities are plotted on the y-axis.

pp_conf_plot(
  obs = sample, 
  distribution = punif,
  points_params = list(cex=.1)
)

However, in this plot it is very difficult to see anything because the individual points are so small and the magnitude of the deviation from the reference distribution is not very large.

Instead, if we plot the expected probabilities on the x-axis and the observed probabilities - the expected probabilities on the y-axis by setting the difference parameter to TRUE, things become much easier to see:

pp_conf_plot(
  obs = sample, 
  distribution = punif,
  points_params = list(cex=.1),
  difference = TRUE, # Make y-axis differenced
  ylim = c(-.0225, .0225)
)

log Scaling

Sometimes, we are only interested in deviations of the sample from one tail of the reference distribution. For instance, if we have a sample of n p-values that we would like to compare to the U(0, 1) distribution for the sake of global null hypothesis testing, we’re generally only interested in p-values that are small. In this case, especially with large n, it can be helpful to transform the probability values onto the −log10 scale.

To demonstrate this, we will generate 10,000 points from a mixture of a Beta(.25, 1) and a U(0, 1) and test this sample against a U(0, 1) reference distribution.

mix <- distr::UnivarMixingDistribution(
  distr::Beta(shape1 = .25, shape2 = 1), 
  distr::Unif(),
  mixCoeff=c(.01, .99)
)

sampler <- distr::r(mix)

sample <- sampler(10000)

Again, if we make a standard PP plot the relevant parts of the plot are very difficult to make out.

pp_conf_plot(
  obs = sample, 
  distribution = punif,
  points_params = list(cex=.1)
)

Even if we set difference to TRUE, it’s still difficult to see the left tail of the distribution.

pp_conf_plot(
  obs = sample, 
  distribution = punif,
  points_params = list(cex=.1),
  difference = TRUE
)

However, setting log10 to TRUE makes the plot much easier to see.

pp_conf_plot(
  obs = sample, 
  distribution = punif,
  points_params = list(cex=.1),
  log10 = TRUE
)

Note, that if we want to highlight the right tail of the distribution as opposed to the left tail, we can set log10 to TRUE and the right_tail parameter to TRUE. This results in a plot with the data transformed onto the log10 scale instead of the -log10 scale.