We will be using a series of variables to control these variables, but the general strategy is to set these to positive values (higher weights) to allow us to have more control over the variance of the performance.
First and foremost we need access to the weight and bias of the training set for our target variables. This can come through one of two ways: a non-linear transformation, or the use of a non-linear model. In the previous section we found that the training set is a linear function of the training parameters, which makes fitting a non-linear model very difficult. In this case, we’ll use a linear transformation to transform the training set from a non-linear function, such as SVM, into a linear function, such as SVM.
With that explained, we’ll run two regressions, to see how each of our training variables affected the final performance. Here we can also take into account the bias in our regression which we will discuss later. It’s important to understand that we’re actually giving the residuals from our regression a value of 1 when they predict better. As such, we’ll want to use the residuals from the regression to control the bias of the regression.
To begin, let’s run our regression first and see how it performs in an R environment (note, we only need to use a model with 100 observations):
def regress ( self , x ): # first pass: model -> training set, residuals -> regression residuals = np . linspace ( 0 , ( len ( x ), 100 )) # second pass, regression residuals = residuals . reshape ( 10 , dtype = [ float )[ int ]) # first pass: model -> training set, residuals