Register FAQ Search Today's Posts Mark Forums Read
Go Back   JazzJackrabbit Community Forums » Open Forums » General Jazz Jackrabbit Talk

Degrading quality and fairness in reviews

Tik Tik's Avatar

JCF Member

Joined: Mar 2001

Posts: 4,982

Tik is doing well so far

Apr 27, 2004, 10:14 AM
Tik is offline
Reply With Quote
Degrading quality and fairness in reviews

I know everyone is more than sick of hearing about the reviews on j2o, whether they are being editted or whatever. This rant has nothing to do with admins editting reviews, it has to do with reviewers themselves.

Perhaps I'm just being a little melodramatic, but I'm having serious issues with the regular reviewers nowadays. It simply isn't fair, and I can no longer trust the rankings on j2o unless we get some dedicated and better reviewers. Back in the day, in fact as early as a year ago, getting above an 8 took some serious dedication and work on the creator's part. However any genuine incentive to produce quality has been degraded, in my view, in light of the barrage of unfair, and poor reviews.

The main download that caught my eye was this. Before I reviewed the download had an 8.2, I believe. If you take a look at the tileset, this is simply not good enough to recieve something like that. And yet, a string of people gave it 8+, mainly by what I guess are "influenced" reviews. That is, they base their reviews off of the previous reviews. So the standard is set by review 1 and confirmed as more and more people confirm something that simply isn't true. The only review that contains any semblance of good judgement is Snooz's.

Perhaps I am over-reacting. This could just be one case, but I think overrating is a real problem. The point of this post is to request that if you guys are going to write reviews, please try to be honost, fair, and don't be afraid to go against the consensus. You can point the finger at me and say I'm doing nothing, but I think I'm going to start reviewing much more often for this very purpose. It's for genuine concern for this slowly dwindling community that I make this request.

Thanks for reading. I want replies.
__________________
<table width="100%"><tr><td valign="top"><a href="http://www.jj2.info" title="waaaait" style="font-size: 14pt;font-family: Verdana;text-decoration: none;">penny on the train track</a>
<a href="/junk/tick/tickbot.html">readme</a> - <a href="/junk/tick/quote.html">quotes</a> - <a href="/junk/tick/rsg.html">formats</a> - are you brained? *\o/*</td><td width="1%" align="right"></td></tr></table>
MoonBlazE MoonBlazE's Avatar

JCF Member

Joined: Jul 2002

Posts: 2,543

MoonBlazE is an asset to this forum

Apr 27, 2004, 10:20 AM
MoonBlazE is offline
Reply With Quote
You cannot code idiots out of Jazz2online. No matter how much you try. Most of these reviewers are basing their opinion by comparing it to their own crap work, as in "Ohh, that's so great compared to my level, so I'll give it a 9!". Also these reviews are very short, only the form and major spacing is filling it up. So, I strongly agree.
__________________

Quote:
Originally Posted by Disguise View Post
*sings*
YOU AND ME BABY ARE NOTHING BUT MAMMALS LA LA LALAL LA LA ON THE DISCOVERY CHANNEL!!!
cooba cooba's Avatar

JCF Veteran

Joined: Jan 2004

Posts: 7,812

cooba is a glorious beacon of lightcooba is a glorious beacon of lightcooba is a glorious beacon of lightcooba is a glorious beacon of lightcooba is a glorious beacon of lightcooba is a glorious beacon of light

Apr 27, 2004, 10:21 AM
cooba is offline
Reply With Quote
For those who didn't get it:

Translation : DO NOT OVERRATE UPLOADS OR I WILL CAPS IN TALK UPS TYPO DO NOT BAN ME ^__^;;;;;;;;;;;;;;;;;;;
MaGoo

JCF Member

Joined: Jun 2001

Posts: 2,059

MaGoo is doing well so far

Apr 27, 2004, 10:23 AM
MaGoo is offline
Reply With Quote
You're absolutely right. Calling some of these people poor reviewers is an understatement. Some of the things I have been seeing is pure stupidity. This community needs to straighten up and fly right.
__________________
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
$n00z

Banned

Joined: Apr 2004

Posts: 101

$n00z is doing well so far

Apr 27, 2004, 10:34 AM
$n00z is offline
Reply With Quote
Yeah, i have to agree with all posts here. well. no more comments.
Link Link's Avatar

Untitled

Joined: Apr 2001

Posts: 2,099

Link is doing well so far

Apr 27, 2004, 10:37 AM
Link is offline
Reply With Quote
I've mentioned this before, but J2Ov2 will have a system to hopefully reduce problems like these and encourage people to review better. There will probably be some kind of ranking system that rewards good reviews rather than large amounts of reviews. For fairness, there will not be a specific group of people that determines the quality of reviews, and rankings will not affect rating weighting.
__________________
With our extreme gelatinous apology,
We beg to inform your Imperial Majesty,
Unto whom be dominion and power and glory,
There still remains that strange precipitate
Which has the quality to resist
Our oldest and most trusted catalyst.
It is a substance we cannot cremate
By temperatures known to our Laboratory.

~ E.J. Pratt
 
MoonBlazE MoonBlazE's Avatar

JCF Member

Joined: Jul 2002

Posts: 2,543

MoonBlazE is an asset to this forum

Apr 27, 2004, 10:38 AM
MoonBlazE is offline
Reply With Quote
I might even want to start review again if j2onlinev2 will be that good
__________________

Quote:
Originally Posted by Disguise View Post
*sings*
YOU AND ME BABY ARE NOTHING BUT MAMMALS LA LA LALAL LA LA ON THE DISCOVERY CHANNEL!!!
Fawriel Fawriel's Avatar

JCF Éminence Grise

Joined: May 2002

Posts: 11,990

Fawriel is doing well so far

Apr 27, 2004, 10:57 AM
Fawriel is offline
Reply With Quote
After seeing that, maybe the idea of simply having a few "professional reviewers" wouldn't be such a bad idea. I know, personal opinion and all, but 8+ for that tileset was more than ridiculous.
On the other hand, who'd want to review professionally anyways? =P
__________________
Violet CLM Violet CLM's Avatar

JCF Éminence Grise

Joined: Mar 2001

Posts: 10,993

Violet CLM has disabled reputation

Apr 27, 2004, 11:01 AM
Violet CLM is offline
Reply With Quote
J2Ov2 may fix this problem. However, I'm pretty sure that there is currently no way to restore ratings to normal other than BANNING EVERYONE.
...sorry.
Tik Tik's Avatar

JCF Member

Joined: Mar 2001

Posts: 4,982

Tik is doing well so far

Apr 27, 2004, 11:12 AM
Tik is offline
Reply With Quote
I'm well aware that j2ov2 will fix that problem, but that's not an excuse for ignoring it before it's released.
Violet CLM Violet CLM's Avatar

JCF Éminence Grise

Joined: Mar 2001

Posts: 10,993

Violet CLM has disabled reputation

Apr 27, 2004, 11:32 AM
Violet CLM is offline
Reply With Quote
What do you want us to do? Remove all overratings and get even more complaints?
Tik Tik's Avatar

JCF Member

Joined: Mar 2001

Posts: 4,982

Tik is doing well so far

Apr 27, 2004, 11:35 AM
Tik is offline
Reply With Quote
As I stated in my first post, I wasn't requesting anything of the admins, I was requesting better quality from the reviewers themselves.
Violet CLM Violet CLM's Avatar

JCF Éminence Grise

Joined: Mar 2001

Posts: 10,993

Violet CLM has disabled reputation

Apr 27, 2004, 11:36 AM
Violet CLM is offline
Reply With Quote
Ah. Well, that's harder. How do you tell someone, "stop rating things more than they deserve"?
MaGoo

JCF Member

Joined: Jun 2001

Posts: 2,059

MaGoo is doing well so far

Apr 27, 2004, 11:45 AM
MaGoo is offline
Reply With Quote
There are two problems with that:

1) Most of the reviewers who overrate levels think they're doing a good job, therefore, are not willing to change anything.

2) I have been noticing that many of these reviewers have a hard time with their English, so they would probably misunderstand the people who are trying to give them advice. (NOT a stereotype)

Frankly, I think we'll all just have to wait for J2Ov2 to come out before this mess can be resolved.
__________________
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Odin Odin's Avatar

Banned

Joined: Dec 2003

Posts: 3,776

Odin is doing well so far

Apr 28, 2004, 05:58 PM
Odin is offline
Reply With Quote
I am not an overrater, i'm nice. I gave that tileset an 8.2, or something like that, 'cause I thought, 'This guy is new, and this is better then all the other stuff i've seen by n00bs, so i'll rate nicely.' I did not mean to overrate.

The rating system is very hard to control because everyone has different opinions. For example, let's say Person A uploads a battle that uses the jj1 Tubelectric set. Person B might love the set as well as the battle level, so he'll rate high. Person C hates tubelectric more then anyother tileset, and in his opinion, the battle level is sux, so he'll rate low.

It is also hard to control because everyone has different rating systems. For instance, I always give a few points to be mixed and churned to boost the rating if the creator is new. Violet CLM doesn't. Danyjel rates using percentages. I rate using an averaging system.

Finally, it's impossible to control 'Popularity Impulses'. An example of a popularity impulse is when Disguise makes a mod file, he'll most likely get 8s, 9s, and 10s. But if a n00b hands in the SAME EXACT mod file, the n00b will get 5s, 6s, and 7s.

I est my case, as well as my face. So shut up the place.

-ThO (Hee hee!)
Risp_old Risp_old's Avatar

JCF Member

Joined: Mar 2003

Posts: 3,992

Risp_old is doing well so far

Apr 28, 2004, 06:05 PM
Risp_old is offline
Reply With Quote
Actually, if Disguise made a .mod file and got a good rating for it, and then a noob made something exactly the same it would be too high to give that .mod file 5-7, since he would be a plagiarizer. But, seriously, I see what you are saying. I do not neccessarily agree with your method, though.
__________________
<img src="http://i25.photobucket.com/albums/c100/Ashton_JX/the_web/stupid_prize.gif" border="0" alt="The rodent thingy wasn't worthy.">
I would not want anyone having sex with my cocktail. ~ Radium
Blackraptor Blackraptor's Avatar

JCF Member

Joined: Sep 2002

Posts: 3,702

Blackraptor is an asset to this forumBlackraptor is an asset to this forum

Apr 28, 2004, 06:57 PM
Blackraptor is offline
Reply With Quote
Imo I don't think giving 'bonus' points to noobs is a good idea, since if some oldie uploads a level that hits around 7, and a newbie uploads a level worth around 5.5, but just 'cause he's new he gets 1.5 extra points is fair (even though I think I used to do something like that a long long time ago). I just rate levels by how decent they are (except now I don't review at all unless its a special occasion).

And yeah, tic, I fully agree with your opinion. Also the reason I seldomly upload things nowadays.
__________________
Fear cuts deeper than swords
FireSworD

JCF Member

Joined: Aug 2001

Posts: 2,834

FireSworD is an asset to this forumFireSworD is an asset to this forum

Apr 28, 2004, 07:41 PM
FireSworD is offline
Reply With Quote
This is amusing, we complain about bad reviews and wether they should be removed or not because of the quality of them, yet not about bad levels.

I agree with the person who once said that if a level is overrated, and if you care, write your own review and fix it. Problem solved.
Tik Tik's Avatar

JCF Member

Joined: Mar 2001

Posts: 4,982

Tik is doing well so far

Apr 28, 2004, 09:01 PM
Tik is offline
Reply With Quote
Quote:
Originally Posted by Fire Sword!
I agree with the person who once said that if a level is overrated, and if you care, write your own review and fix it. Problem solved.
That's exactly what I did. And then I posted this topic to tell other people to help to do the same thing.

Don't encourage bad quality just because a person is a new. Give them an honost, fair rating, and tell them how they might improve. If they really care they will put in more effort and will thus become better at JCS.
__________________
&lt;table width=&quot;100%&quot;&gt;&lt;tr&gt;&lt;td valign=&quot;top&quot;&gt;&lt;a href=&quot;http://www.jj2.info&quot; title=&quot;waaaait&quot; style=&quot;font-size: 14pt;font-family: Verdana;text-decoration: none;&quot;&gt;penny on the train track&lt;/a&gt;
&lt;a href=&quot;/junk/tick/tickbot.html&quot;&gt;readme&lt;/a&gt; - &lt;a href=&quot;/junk/tick/quote.html&quot;&gt;quotes&lt;/a&gt; - &lt;a href=&quot;/junk/tick/rsg.html&quot;&gt;formats&lt;/a&gt; - are you brained? *\o/*&lt;/td&gt;&lt;td width=&quot;1%&quot; align=&quot;right&quot;&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;
MoonBlazE MoonBlazE's Avatar

JCF Member

Joined: Jul 2002

Posts: 2,543

MoonBlazE is an asset to this forum

Apr 30, 2004, 09:47 AM
MoonBlazE is offline
Reply With Quote
You should difinately not give someone bonus points because he or she is new. That way you make people think their doing is good and doesn't need improvement, thus they'll keep upload like that. The best way to learn is to recieve critic and be open to it. Besides, giving 'bonus points' are being unfair, you're being a bad reviewer by doing differences between members because of the date they joined.
__________________

Quote:
Originally Posted by Disguise View Post
*sings*
YOU AND ME BABY ARE NOTHING BUT MAMMALS LA LA LALAL LA LA ON THE DISCOVERY CHANNEL!!!
Trafton

JCF Member

Joined: Oct 2002

Posts: 3,589

Trafton is an asset to this forum

Apr 30, 2004, 10:04 AM
Trafton is offline
Reply With Quote
Not only should you not do it, but J2O pretty much disallows giving anyone points for who they are. If you want to be positive with a new user, do so in the review - people appreciate being told that they are good for being new. If they get a really awful rating the next time they upload something, they will be more discouraged than if they got one the first time but got positive comments. You don't rate the person, you rate the work.

~ Traft
Tubz Tubz's Avatar

JCF Veteran

Joined: Apr 2001

Posts: 5,851

Tubz is doing well so far

Apr 30, 2004, 03:35 PM
Tubz is offline
Reply With Quote
I think I'm gonna reinstall JJ2!
__________________

"Jewel is the Metallica of Yodelling." ~Edwin McCain
"Yes, it's a personal attack if you save my picture about internet safety" ~Labratkid
"You know what JBL? The only reason you were WWE Champion for a year was because Triple H didn't want to work Tuesdays!" ~Paul Heyman, addressing the public at the "ECW: One Night Stand" PPV on Sunday, June 12, 2005
Aiko

JCF Member

Joined: Mar 2001

Posts: 568

Aiko is doing well so far

May 6, 2004, 08:41 AM
Aiko is offline
Reply With Quote
I guess there is a lot to say here, because i'm still one of the guys with the most reviews here and most of them being pretty (what you would call) "harsh", but i don't have too much time right now. So it's a short statement:

I don't like the idea of "professional reviewers" at all. Our basic democratic conception of J2O, as i understand it, in terms of everyone, newbie or not, having the same rights, should not be changed into a system, where some people are "more equal than others". See George Orwell's "Animal Farm" for more info.

The easiest solution for "overrating" is: go ahead and write a more accurate review of the level. If enough other people will do the same, we don't have the problem anymore. That's democracy
Violet CLM Violet CLM's Avatar

JCF Éminence Grise

Joined: Mar 2001

Posts: 10,993

Violet CLM has disabled reputation

May 6, 2004, 11:00 AM
Violet CLM is offline
Reply With Quote
What you're saying is this: Because the rich people don't pay as many taxes as the poor people, the poor people should pay more taxes than they already do, to make up for the taxes not paid by the rich people.

...all right, so it's not perfect. I'll admit it seems strange to think of the newbies as the rich people. It's just kind of annoying to have to do extra work to fix someone else's mistakes.
Reply

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is On

Forum Jump

All times are GMT -8. The time now is 03:13 PM.