Dear all,
I have computed a likelihood-ratio test to test the external validity of the interaction-effects model vs. the "generic"/ main-effects model.
I thought that the degrees of freedom used to find the critical value for the chi-squared distribution would be equal to the number of parameters added (https://sawtoothsoftware.com/forum/13313/the-2-log-likelihood-test-for-interaction-effects?show=13313)
In my study, there are five attributes with 3 level each leading to 10 degrees of freedom accordning to the information above (see link).
I added 2 interaction effects, i.e. A x B and B x C leading to additional 18 parameters (!) in the model (w/o interaction effects, the model includes 16 parameters incl. the none-option). To find the critical value from the chi-squared distribution, which value do I have to use? DF = 18 (since 18 added parameters)?
Also, my analyses showed that the interaction-effects model fits the observed data significantly better (according to likelihood-ratio test, questions above). However, the goodness of fit criteria such as percent certainty (28.2 main, 28.83 interaction), rmse and mae just slightly increased, the hit rate even decreased. Also, the interaction effects are not interpretable very well across the four segments (I am doing an LC analysis). I could do so, but it just would be become a way much more complex, and the sample size (N= 265) is not that much.
Based on this information, is it fair to say that I go with the main effects model? Could there also be an issue with "overfitting"?
Best regards!