A fair assessment of multiple similar items can only be made if all are measured by the same benchmarks beginning with a neutral or nonexistent predisposed opinion.
As a WordPress Theme reviewer I make every effort to stay within a near blinder-like focus of the themes I am looking at. I mostly start with the technical issues relating to theme development then I move on to basic functionality. The criteria of one or both of these are generally the reason a theme is not accepted.
The acceptance of the overall design of the theme will be decided by the end-users which leads to the last major criteria I review with: theme aesthetics.
This is also where I see a fudge factor can be introduced. If a theme is very well laid out, pleasing to look at, and easy to use I may consider overlooking minor technical issues with a theme if there are only a few.
If I accept a theme after finding minor technical issues that are outweighed by the theme’s design merits I make a point of advising the theme author those technical issues should be addressed within the next update of the theme … which is expected to be within a very short but reasonable time-frame.
