I was recently asked, "How is an anchored MaxDiff better than simply asking the anchoring question for every item, for all respondents?"
For context, in addition to summarizing the utilities, I calculate the "percent reached" for each item, which is simply the percentage of respondents whose item utility is greater than the anchor utility. I also use this as the definition of reach when doing TURFs. I also always do the direct binary approach for either a subset or all items rather than the indirect approach, so the person is asking why not just skip the MaxDiff and use the "anchor" question.
For sake of discussion, let's assume we only care about determining reach for calculating the percent reached and for use in a TURF, and we don't care about summarizing the utilities or any of the other potential outputs of a MaxDiff. Is there a good reason to do an anchored MaxDiff with direct binary anchoring (all items or subset of items) rather than just asking that direct binary anchor question for all items? Of course, a MaxDiff provides much better differentiation among items, but if we only care about above/below a reach threshold, does it even matter?
My response was that an anchored MaxDiff can do a better job at handling inconsistent respondents by using a combination of their MaxDiff responses and the anchor question. But I could also see an argument where we should just use the true anchor question responses, even if they might seem inconsistent or illogical.
Thoughts?