In my final exam for stat there is a question that asks what happens to F stat as independent variables increase

a. always increases

b. always decreaes

c. remains the same

what and why is the anwers pls help

can you post the whole question......... is that question from regression?

Yes, it's multiple choice, it is the whole question.

anybody

okay, it's been years i haven't done anything on regression...but if my memory serves me right, here's what i would probably do:

with the increase in independent variable, F stat should always decrease.

reason:
remember, F stat = (regression model Mean of Square)/(Error Mean of Square). for a given no. of samples, with an increase in the number of independent variables, the error mean of square increases and the regression model mean of square decreases thus leading to a decrease in F stat overall.

It has been a while since I looked into my econometric books but I thought it could go either way depending upon the marginal impact of the added variable. Addition of independent variables will cost degrees of freedom but will also have added explanatory power and thus, increasing the ESS and decreasing the RSS.

Try running a regression with one irrelevant independent variable and adding the relevant variables after that. What does the model say for F-stat? My take would be, F-stat will increase. I might be wrong as well.

By the way, here is a link to the material on introductory econometrics by a LSE Professor if you want to look:

http://econ.lse.ac.uk/courses/ec220/G/ieppt/series2/#chap1

Addition of independent variables will cost degrees of freedom but will also have added explanatory power and thus, increasing the ESS and decreasing the RSS.

increase in ESS leads to increase in EMS and decrease in RSS will lead to decrease in RMS as well. with the increase in the number of independent variables, there will be an increase in degree of freedom for RMS  and a decrease in degree of freedom for EMS. so this will further decrease F-stat...

i don't know why you say it could be increasing....

or am i missing sth? or are you just throwing the darts in the dark and trying to be safe stating it could both increase or decrease? hahahaha

Loote,
I don't understand your assumptions because you need to consider both effects, loss of degrees of freedom and gain in explanatory power before you can decide the impact of additional variables. I guess you are missing something...right now, am a little bit busy, will get back in a while.

yeah it can go either way. since MSres=  SSres /(n-p) where n is sample size and p is no. of predictors {i.e. no. of independent varaibles (k) + 1}. since SSres always decreases as p increases, MSres initially decreases, then stablizes, and eventually may increase. so increasing independent variables may decrease or increase the F statistic and thats the reason we choose the model with minimum MSres while selecting a model out of different models with varying number of predictors in multiple regression analysis based on MSres or Radj criteria. hope that helps.

Alright Loote,
I guess geico filled some holes in there but here I go.

You know that F-stat can be represented as R2/(k-1)/((1-R2)/(n-k-1)). As you increase independent variables, R2 will go up (even though Adj-R2 might go down if the variable is insignificant).

R2/(1-R2) will always go up with increase in independent variables. But, since we are taking the degrees of freedom into consideration as well, the net effect of an addition of a variable is an increment of F-stat only if the variable is significant.

Again, I am not the expert in this field.

hahahaha...
bibas and geico,
i feel that the question is not fully self-explanatory and after some thought to it, i am still adamant with my analysis. i feel it this way:

1) the F-stat should decrease if the independent variable is non-significant (from null-hypothesis) to the overall predictor...coz of the things i have already explained. bibas, the explanatory power doesn't come into play in this case.

2) if it is significant, then according to what geico put forth, it can lead to both increase or decrease to F-stat...but since we choose lesser MSreg model so again, we are looking at decreasing SSreg (which means decrease in MSreg and since MSerror is also decreasing), again the net effect to F-stat is decrement.


and bibas, where does your formula come from? hahaha i strongly suggest you to go back and check your stat book of UG years...i am sure you had taken the course. you have already mentioned in one of your earlier posts, that "Addition of independent variables will cost degrees of freedom but will also have added explanatory power and thus, increasing the ESS and decreasing the RSS."

you do  your own math now provided if you know the formula for F-stat

and also let it be known that no one (at least till now) is expert in here ... my study doesn't relate to stat at all...what i am explaining is based on what i studied during my sophomore years which is a good 6 years ago...and on another note, last i heard of you in sajha somewhere, you were an aspiring economist/analyst or sth?

Loote,
There are different ways to represent F-stat if you did not know about it. Check your books. If they are missing it, then the author of that book is missing it. I used the R2  to make it easy for people to understand because most people know that R2 increases with increase in the number of variables whether significant or not. Just try to play with the formula if you want a proof of it.

As I said before, the answer is not definitive. It can go either way depending on the explanatory power of the variable, whether it is significant or not.  As long as the explanatory power is able to offset the effects of loss in degrees of freedom, F-stat should go up. By the way, what do you mean there is nothing to do with explanatory power? What does F-stat measure?

Oh man....even though I am not an expert in this area, I work in this field. Come on!!!

The basic assumption behind increasing the independent variables or predictors is to improve F statistics and more predictors in the model lowers the bias of predictions but increases the variance. So there is always a trade-off and when we underfit the model (say ignore important predictors) we deposit the variation accounted for by the ignored predictors in the residual sum of squares and hence inflate the residual mean square. These biases reflect as bias in prediction of estimates of the coefficients of retained variables and the response.  The general behavior of MSres as predictors (P) increases is as shown in the graph. As I mentioned earlier SSres always decreases as P increases, MSres initially decreases, then stabilizes, and eventually may increase. The eventual increase in MSres occurs when the reduction in SSres from adding a regressor to the model is not sufficient to compensate for the loss of one degree of freedom in the denominator of equation MSres = SSres/(n-p) i.e. adding a regressor to a p regressors model will cause new MSres (p+1) to be greater than MSres (p) if the decrease in SSres is less than MSres (p). This unstable behavior of MSres with the increase in predictors also governs the value of F-statistics and we can’t say where F-statistic increases or decreases for sure with increase in predictors in a model.

PS:  i am also not an expert in statistics tara maile regression analysis course liyeko thiye 

Thanks for your input guys. If F - Stat continues to decrease can it become negative?