Stepwise regression

In statistics, stepwise regression is a method of fitting regression models in which the choice of predictive variables is carried out by an automatic procedure. In each step, a variable is considered for addition to or subtraction from the set of explanatory variables based on some prespecified criterion. Usually, this takes the form of a forward, backward, or combined sequence of F-tests or t-tests.

The frequent practice of fitting the final selected model followed by reporting estimates and confidence intervals without adjusting them to take the model building process into account has led to calls to stop using stepwise model building altogether or to at least make sure model uncertainty is correctly reflected by using prespecified, automatic criteria together with more complex standard error estimates that remain unbiased.

In this example from engineering, necessity and sufficiency are usually determined by F-tests. For additional consideration, when planning an experiment, computer simulation, or scientific survey to collect data for this model, one must keep in mind the number of parameters, P, to estimate and adjust the sample size accordingly. For K variables, P = 1_(Start) + K_(Stage I) + (K² − K)/2_(Stage II) + 3K_(Stage III) = 0.5K² + 3.5K + 1. For K < 17, an efficient design of experiments exists for this type of model, a Box–Behnken design, augmented with positive and negative axial points of length min(2, (int(1.5 + K/4))^1/2), plus point(s) at the origin. There are more efficient designs, requiring fewer runs, even for K > 16.