Lt. Commander
Join Date: Nov 2012
Posts: 126
# 1 A Reviewers Request
03-07-2013, 07:52 AM
It's been an opinion that's been voiced in a few other threads, but I figured I would put it out here on it's own.

First off, let me clarify that I think the Foundry qualifications should stay the way they are (even if they do seem somewhat amorphous and unclear) and the scaled dilithium rewards were a great idea to promote a variety of content. This post has nothing to do with complaints and/or changes to said mechanic we have currently, nor anything to do with Fleet Marks, EC, drops, or otherwise. Please do not make this thread about that as it's been discussed ad nauseam. I'd rather this thread be locked, dropped, and kicked while down before having that discussion again.

The issue is with the review system and it's effect on the very limited search function that we have. It's hard enough getting reviews for new missions, as playing qualified or spotlighted missions gets rewards and new unqualified missions don't. So the few reviews the new mission actually does get really could either make or break it completely, since bad reviews send it straight to the bottom of the abyss of new missions out there.

Let's face it. Critical reviews happen for every author. We make mistakes. Authors that actually want to contribute something to the community wants to fix those mistakes and give people something worth playing. For new missions, it's impossible to do that when you get a 5-star review, then get a 2 or 1 star review, with no input whatsoever as to why for either. Sometimes it might be a fleet member giving you a boost. Other times it could be someone who just blames foundry authors for their woes and decides to tap your mission for it. And then sometimes you have real players that want to give their opinion. Sometimes it could be the smallest thing or even a bug that can't be fixed. If something's wrong, please tell us why. We really do want to know.

In the end, there's no way of ever knowing, basically because ratings can remain anonymous. In my opinion, if you are given the privilege to rate someone else's efforts, especially if you are not paying them for it, then you should at least be required to give some feedback on your rating. It works that way in school. It works that way in the movies. It should at least work here as well. But then, some people probably don't feel they should be required to write anything. There's a fix for that too.

My idea is this: ALL ratings should appear in the comment box. Even if the box is left blank, the player's name should still appear in the list with their star rating, just like it does when a comment is left. That way, community authors will know if they just got that thumbs up from a buddy, and at the same time be able to tell if someone is trolling their missions to give them bad reviews.

As part of the community, users of the foundry have a right to the same fair treatment as anyone in general chat or on the forums. If someone is going to take an action or make an opinion that has an effect on someone else's game content, they should have their name stamped to it. Anyone who wants to give an honest, unbiased opinion on another user shouldn't mind identifying themselves anyway. In fact, I think real reviewers would like that even more.

Thoughts?


by @Skydawn - member of Starbase UGC

Last edited by pwebranflakes; 03-07-2013 at 08:28 AM. Reason: Fixed the title as petitions are not allowed.
Lieutenant
Join Date: Feb 2013
Posts: 44
# 2
03-07-2013, 08:25 AM
I see what you mean, right now someone can get a nice boost just by asking friends to 5 star all of their stuff. At the same time you can spend months crafting a beautiful work just to see it never see the light of day because one bored guy gave it a single star for reasons that do not reflect your work.

The one issue I see with making the reviews non-anonymous is people might be worried about comeback. Say I play someone's mission and think it really needs work or is just plain awful so I give them a single star and some feedback. They just see "hey, this person downvoted me", go onto one of my missions and downvote it just to spite me. Equally if I give someone 5 stars because their mission is excellent they may feel compelled to give me 5 stars back "you scratch my back.." type thing, even if my mission is rubbish. I saw this sort of thing happening in The Movies with people trading high review scores.

An alternative might be that if you are going to post a bad review you must give reasons, similar to Skype's call quality feedback. If you click 5 stars then the mission is fantastic to all is well. If you click 1 star then it prompts you to answer what was wrong with the mission. You could even extend it to also require that for all feedback, just a quick click as to the one thing you most liked and disliked about any mission. Just a drop-down list of things such as combat, story, layout, spelling, dialogue, visuals, time, "star trek feel". Instead of a simple 5 star system you then have a more diverse metrics on player experience without bothering the reviewer much more than a few more mouse clicks. Missions could then be sorted by those, so instead of just "highest rated" you say "I want to see the missions which people liked the story the most" or "I don't want any mission where people disliked the spelling."

People would still go on and downvote things, but the star ratings would no longer be all important. A game with a few 2 star ratings might still rank high in the list of games where people liked story.

Anyway, I'm just throwing ideas here. I agree the current system is open to abuse too strongly for new missions. When you get hundreds of reviews the bulk opinion will drown out the trolls and the fleet friends, but as you said, its the issue with getting to that point.


Click for more details
Lt. Commander
Join Date: Nov 2012
Posts: 126
# 3
03-07-2013, 08:32 AM
Quote:
Originally Posted by remairtamec View Post
I see what you mean, right now someone can get a nice boost just by asking friends to 5 star all of their stuff. At the same time you can spend months crafting a beautiful work just to see it never see the light of day because one bored guy gave it a single star for reasons that do not reflect your work.

The one issue I see with making the reviews non-anonymous is people might be worried about comeback. Say I play someone's mission and think it really needs work or is just plain awful so I give them a single star and some feedback. They just see "hey, this person downvoted me", go onto one of my missions and downvote it just to spite me. Equally if I give someone 5 stars because their mission is excellent they may feel compelled to give me 5 stars back "you scratch my back.." type thing, even if my mission is rubbish. I saw this sort of thing happening in The Movies with people trading high review scores.

An alternative might be that if you are going to post a bad review you must give reasons, similar to Skype's call quality feedback. If you click 5 stars then the mission is fantastic to all is well. If you click 1 star then it prompts you to answer what was wrong with the mission. You could even extend it to also require that for all feedback, just a quick click as to the one thing you most liked and disliked about any mission. Just a drop-down list of things such as combat, story, layout, spelling, dialogue, visuals, time, "star trek feel". Instead of a simple 5 star system you then have a more diverse metrics on player experience without bothering the reviewer much more than a few more mouse clicks. Missions could then be sorted by those, so instead of just "highest rated" you say "I want to see the missions which people liked the story the most" or "I don't want any mission where people disliked the spelling."

People would still go on and downvote things, but the star ratings would no longer be all important. A game with a few 2 star ratings might still rank high in the list of games where people liked story.

Anyway, I'm just throwing ideas here. I agree the current system is open to abuse too strongly for new missions. When you get hundreds of reviews the bulk opinion will drown out the trolls and the fleet friends, but as you said, its the issue with getting to that point.
I would love to see that too, but this might at least be a simple solution for now versus waiting for a complete UI overhaul. If someone is afraid of backlash just for giving an honest constructive opinion, then 1) they shouldn't be a reviewer, and 2) the author complaining about an honest review shouldn't be doing this in the first place.


by @Skydawn - member of Starbase UGC
Lieutenant
Join Date: Feb 2013
Posts: 44
# 4
03-07-2013, 08:54 AM
Yeah, I kind of realised my suggestion would end up being a long one. Maybe it could be brought in along with future updates on the find a mission UI.

As for the fact that authors shouldn't give any backlash against bad reviews, sure. But then folk shouldn't give bad reviews based on having a disagreement on the forums but as you say, it happens. All in all though I think your idea is a good stopgap solution. Maybe not ideal long-term but should do more good than harm. So for what its worth I'm behind this, at least until some other way is found to make a couple of bad reviews so destructive to new projects.


Click for more details
Ensign
Join Date: Sep 2012
Posts: 12
# 5
03-07-2013, 09:29 AM
Idk, I think reviewers should have the option of leaving their name or not. What I do support would be a revamping of the review process as such:

Current scoring system scrapped
New system put in place where reviewers score the mission on 5 criteria: Story, Combat, Layout, Spelling, and Canon Accuracy.
Each review is then given a final score based on the average of the above scores.
Overall mission rating would display the average final scores, as well as averages for each of the above criteria.

This would help in several ways. First, reviews without comments would still indicate what the player liked/didn't like about the mission. Second, troll reviews would be more easily identified and ignored as very few missions would truely deserve a 1 star rating in all 5 catagories. Finally, if someone is looking for say a mission heavy on combat, with or without story, he or she could search foundry missions based on combat scores only.
Survivor of Romulus
Join Date: Jun 2012
Posts: 2,428
# 6
03-07-2013, 09:32 AM
This is something we've talked a lot about over the years, particularly last year when we had a rash of 1-star bandits targeting missions and sinking them to the bottom of the list for nothing more than the lulz.

A lot of ideas were bandied about, but the one thing a lot of us get behind is definitely the "no anonymous ratings" thing. I think there are some folks who might think a bit harder about how they rate things if they knew their handle was attached to it. Not everyone, obviously, but some.

Still, I think a lot of people really don't care for the star system as a whole, including some of the devs. BranFlakes told us he doesn't even consider the star rating of potential spotlights. The best alternative I've heard was from Havraha, who never liked it from the start. He said it ought to just be a thumbs up/thumbs down thing like YouTube. I think this would be fairer (since every one of us decides those stars by different criteria) and easier to understand. I've seen a fair number of people recently who seem to genuinely think 1 star is the highest rating.
Join the Foundry Roundtable live Wednesdays at 7:30PM EST/4:30PM PST on twitch.tv/thefoundryroundtable

Looking for a Foundry mission for your Romulan character? Look no further!
Rihannsu
Join Date: Jun 2012
Posts: 10,713
# 7
03-07-2013, 09:41 AM
Quote:
Originally Posted by drogyn1701 View Post
This is something we've talked a lot about over the years, particularly last year when we had a rash of 1-star bandits targeting missions and sinking them to the bottom of the list for nothing more than the lulz.

A lot of ideas were bandied about, but the one thing a lot of us get behind is definitely the "no anonymous ratings" thing. I think there are some folks who might think a bit harder about how they rate things if they knew their handle was attached to it. Not everyone, obviously, but some.

Still, I think a lot of people really don't care for the star system as a whole, including some of the devs. BranFlakes told us he doesn't even consider the star rating of potential spotlights. The best alternative I've heard was from Havraha, who never liked it from the start. He said it ought to just be a thumbs up/thumbs down thing like YouTube. I think this would be fairer (since every one of us decides those stars by different criteria) and easier to understand. I've seen a fair number of people recently who seem to genuinely think 1 star is the highest rating.
That's a very good point. I'll 5 star anything that doesn't have a major problem. I've talked to some people who won't 5 star anything unless it makes them go OMG TEH AWESOME!.....
HAIL HYDRA!

-=-=-=-=-=-=-=-=-=-
I can haz joystick!
MMOs aren't charities. Corporations are supposed to make a profit. It's what they do.
Survivor of Romulus
Join Date: Jun 2012
Posts: 2,428
# 8
03-07-2013, 09:47 AM
Quote:
Originally Posted by markhawkman View Post
That's a very good point. I'll 5 star anything that doesn't have a major problem. I've talked to some people who won't 5 star anything unless it makes them go OMG TEH AWESOME!.....
Exactly. Plus like if it's a fleetmate or a friend's mission you might be inclined to rate it more generously than if it was a mission by a guy who's been a jerk to you on the forums.

It gets even more complicated when you include "grind" missions into the mix. I think everyone would agree that players rate those missions by a different set of criteria than they would story missions. As it stands, the rating system doesn't allow any distinction there. Granted a thumbs up/thumbs down wouldn't really help on this point either, but a better search UI would.
Join the Foundry Roundtable live Wednesdays at 7:30PM EST/4:30PM PST on twitch.tv/thefoundryroundtable

Looking for a Foundry mission for your Romulan character? Look no further!
Lieutenant
Join Date: Feb 2013
Posts: 44
# 9
03-07-2013, 10:03 AM
The thumbs up/down thing sounds like an all around better system. I don't think it will resolve the issue of review trolls but it seems much fairer than the star system. At the end of the day I don't really care what number out of 5 someone gave a mission, I want to know if they enjoyed it or regretted playing it. Maybe combining that with an optional, more detailed feedback of what people enjoyed/disliked would be the overall ideal.

Just another thought on the rating system could be some sort of internal tracking of average scores given by an account. If someone gives nothing but 1 stars to all reviews have the rating system downplay their vote when it comes to giving the overall rating. Equally if the system was moved to a simply up/down vote then someone who does nothing but downvote everything would equally be shifted to a "low priority" when calculating the overall rank of a mission. Same for people who tend to give 5 stars to everything, that 5 should be less impacting than someone who mostly gives 3-4 stars but then decides one map is truly wonderful.

Perhaps even requiring repeat downvoters to be required to give comments of set character limits to justify their opinions. Yes, this would lead to lots of "ajsfhsjdkkh" and general abuse but those could be flagged by the author and the troll's account blocked from any more voting, or harsher actions taken if abuse was given.

Yeah... again... my ideas are never simple.


Click for more details
Captain
Join Date: Jun 2012
Posts: 520
# 10
03-07-2013, 11:09 AM
I would prefer no rating system at all; only the ability to leave reviews. This would necessitate a much better search UI to be implemented first, of course. If we are going to have a rating system then the ?up/down? vote system would be best.
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


All times are GMT -7. The time now is 02:35 AM.