A while ago on #jj2, a few people were discussing ways to fix this problem. I didn't really participate in the discussion much, but I do remember some things:

People will not only rate levels, they will also rate reviews. The higher rating a reviewer has, the more their rating will count in the average for a level.

Example:

There are a total of 10 people registered on J2O (to keep things simple): Bob, Scott, Jeff, Red, Lisa, Cherry, Mike, Alex, Kat, and Jim.

After posting some levels and stuff, the review ratings for the people are as follows (they all voted on each other):

Bob = 4
Scott = 8.2
Jeff = 9
Red = 2
Lisa = 6
Cherry = 3
Mike = 3.2
Alex = 7.8
Kat = 9.8
Jim = 1

Say Mike and Cherry both rated a level, and they were the only raters (so far). Mike rated it 7, and Cherry rated it 6 (out of ten). To calculate how much their ratings count, it is done like this for every level:

The total of their review ratings is 6.2. Mike is 3.2/6.2, or 0.52. Cherry is 3/6.2, or 0.48. Multiply each of their review numbers by their review weighting, and you get:

Cherry = 0.48 * 6 = 2.88
Mike = 0.52 * 7 = 3.64

2.88 + 3.64 = 6.52, the average rating of the level.

Now say Scott came on the scene too and added his review (say he rated the level 9). The total of the reviewers ratings is now 14.4. After doing the math above, Mike's weight is now 0.22, Cherry's is 0.21, and Scotts is 0.57.

Cherry = 0.21 * 6 = 1.26
Mike = 0.22 * 7 = 1.54
Scott = 0.57 * 9 = 5.13

1.26 + 1.54 + 5.13 = 7.93, the new average of the level.

Now say that Jim, a really bad reviewer comes in and rates the level 2 (completely outrageous; it is better than that). There are now four ratings, with a total reviewer weighting of 15.4:

Cherry = 0.19 * 6 = 1.14
Mike = 0.21 * 7 = 1.47
Scott = 0.53 * 9 = 4.77
Jim = 0.06 * 2 = 0.12

1.14 + 1.47 + 4.77 + 0.12 = 7.50, the new average of the level. As you can see, Jim's rating did not count much at all. If even more people rated the level - say all the other good reviewers like Kat, Jeff, and Alex, Cherry's and Mike's rating would begin not to be worth much at all, and Jim's would be worth almost nothing.

I think that would work. A weighting system is really what we need to prevent bad reviewers from really hurting the level creators. I won't go into more detail (you should have a pretty good idea from what I've explained above), but if you have any questions you can just post them here.