The problem is that writing a review criticizing the artistic aspects of tilesets, which are to be honest not of primary concern to a level maker, is a lot harder than simply going along a list of masking, palette, eyecandy, ease of use... I'm sure you have seen at least as many reviews which do this ("masking: 9/10. tile placement: 7/10. average: 8! download now!") as me. People just don't want to go through writing a really insightful, constructive review, since it takes a lot more time and effort. There are several reasons for this; it's the internet, J2O's audience is, though older than before, still relatively young, it's easier to use a template, and so on.

I'm not sure whether this was criticism towards J2O's reviewing system or a more general remark, but in any case I don't think this is something J2O can really do anything about. It's not the system that needs to change, it's the users of the system.

I do however think that the importance of "artistic criticism" is not as big as you picture it to be. At least not towards those who will in the end use the tileset - coincidentally also those that are most likely to review the it. Jazz2Online's download section is primarily intended as a repository of files people can download and then play - and it is to be expected that they are reviewed from that point of view too. Thus, while for you as a tileset creator feedback on more artistic aspects may be important, the average downloader will be more concerned about how easy it is to put platforms together, how versatile the set is, whether the masking is good, et cetera.

Your point about the barrier for reviewing being higher than for simply leaving a comment is correct, I think, but not because people feel such comments are out of place in a review. I assume people (or at least people like EvilMike, who actually know how to write decent reviews) realize that just such comments don't constitute a full review, and thus reply to the news post instead because they don't want to spend effort on a full-fledged review.