Dear Sawtooth team,
I just discovered the new "Utility Report" function for HB estimations of the Analysis Manager in Lighthouse. In the table for average utility values you now also report the upper and lower 95% confidence interval. I have some questions in this regard:
1) How do you calculate these intervals? Is it Standard error = [standard deviation / square root of n], and then taking the mean utility value +/- [1.96 (=critical value for 95%) times standard error]?
2) If so, is this even allowed for Bayesian approaches? If not, what would be an equivalent?
3) How would you interpret the provided confidence intervals (in combination with the mean utility value)?
4) In general: what would be a correct measure for judging whether a coefficient is "significant" (analogous to the t-ratio for point estimators)?
Thank you for clarification and your support!