| Pages in topic: [1 2 3] > | Entries in the qualification round can be "ruled out", a few more adjustments are on the way Thread poster: Jared Tabor
|
|---|
Dear members,
In previous incarnations of ProZ.com translation contests, it was possible for entries in a pair in the qualification phase to be "ruled out." This means that entries which have no reasonable chance of winning are removed from contention.
Ruling out will be re-activated for this contest. Hopefully this will help to make it easier to rate entries, especially in pairs where large numbers of submissions were received, as entries get winnowed out.
... See more Dear members,
In previous incarnations of ProZ.com translation contests, it was possible for entries in a pair in the qualification phase to be "ruled out." This means that entries which have no reasonable chance of winning are removed from contention.
Ruling out will be re-activated for this contest. Hopefully this will help to make it easier to rate entries, especially in pairs where large numbers of submissions were received, as entries get winnowed out.
A few more adjustments designed to make rating entries more convenient are in the works; I will keep you updated on those here.
http://www.proz.com/translation-contests/43
Some notes on entry rating:
- You can rate as many or as few entries as you like. It is not necessary to rate all of the entries in a pair if you do not want to.
- Rating is anonymous. As opposed to tagging (likes and dislikes on an entry), raters are anonymous.
- Ratings are what help determine finalists. Tags ("likes" and "dislikes") can be helpful feedback on an entry, but the ratings on entries for "Quality of writing" and "Accuracy of translation" are what determine finalists.
- Participants with an entry can rate entries, too. Experience has shown that this does not negatively affect the outcome, and participants are also in a unique position to provide feedback on entries, since they have worked on the same text and are familiar with the challenges involved.
- Every rating helps. Even if you only rate a few entries, every little bit helps to determine finalists and is valuable feedback for those have submitted an entry.
▲ Collapse
| | | | Claudia Cherici 
Italy
Local time: 08:16
Member (2010)
English to Italian
+ ...
| Who's the out-ruler? | Sep 12, 2014 |
Hi Jared
I am not sure I understand: who's doing the 'ruling out' of the translations unlikely to win? And how do you determine who to rule out (i.e. who is unlikely to win) at this stage, since voting is still possible and will be for another full week? And when would this ruling out be done (at the moment, all the 64 original entries are still there in my language pair)?
Thank you!!
| | | | | How ruling out works | Sep 12, 2014 |
Hi Claudia,
Good questions. Raters are the ones who help determine which entries get ruled out. Staff review and do the actual removal of an entry from contention based on rating and feedback.
After a certain amount of rating has been done in a pair, entries may begin to be ruled out. A clear example of an entry which might be ruled out early on could be an entry which contains serious spelling or grammatical errors. Entries which are on the low end of a sorted list of ... See more Hi Claudia,
Good questions. Raters are the ones who help determine which entries get ruled out. Staff review and do the actual removal of an entry from contention based on rating and feedback.
After a certain amount of rating has been done in a pair, entries may begin to be ruled out. A clear example of an entry which might be ruled out early on could be an entry which contains serious spelling or grammatical errors. Entries which are on the low end of a sorted list of rated entries, in language pairs in which other entries have had relatively high ratings, may also be ruled out.
Ruling out will start soon in some language pairs; I wanted to get this announcement out before it begins, though.
Jared ▲ Collapse
| | | | | There is no zero option for the ratings | Sep 12, 2014 |
If a translation is garbage (for example, entirely machine translation), it does not deserve even 1 out of 5 stars.
But there is no option to actually say ZERO, just to leave it empty, which then means it was not rated.
I think it would help staff to out-rule the garbage entries if there was a way to actually indicate that.
[Edited at 2014-09-12 21:33 GMT]
| | |
|
|
|
Michael torhan (X)
United States
Local time: 23:16
Spanish to English
+ ...
| If the underlying problem is that we're not collecting enough votes... | Sep 13, 2014 |
...then how about forcing contestants to vote, so that only by voting will their own entry be eligible for the finals? In other words, if you're a contestant and don't vote, then you cannot win.
I participated as a contestant, but I still haven't voted and my reasons are both selfish and altruistic. As for altruism, that's easy - win or lose, I wanted to be able to say that I played no part in the decision.
The selfish part stems from how I believe scoring might ... See more ...then how about forcing contestants to vote, so that only by voting will their own entry be eligible for the finals? In other words, if you're a contestant and don't vote, then you cannot win.
I participated as a contestant, but I still haven't voted and my reasons are both selfish and altruistic. As for altruism, that's easy - win or lose, I wanted to be able to say that I played no part in the decision.
The selfish part stems from how I believe scoring might be done (just based on what I've read). My believe is that 1) if I vote and 2) nobody else does, then 3) I've just voted myself out of the contest. If one assumes this is how scoring is implemented, then the logical selfish best choice to refrain from voting.
And should enough people arrive at this (possibly false) conclusion, then it could explain poor voter turnout, if that's indeed what we're experiencing here.
My suggestion is two-fold:
1) Make contestants vote in order for their entries to be eligible to win.
2) Divide up the entries among the contestants; to ensure eligibility of their own entry, they must vote on the 3 to 5 entries assigned to them from a permutation that Proz formulates from the pool of entries. I'm not sure if I'd let contestants vote on any of the other entries -- just the 3-5 assigned by Proz. That way, if a contestant is the type that departs, say, from grammar, spelling, syntax, etc. and fixates instead on pointing out poor synonym choices, then at least their negative energy doesn't spread out of control.
I guess I'm prompted to write this because this idea of bringing back the "rule-out" option, just by its name alone, struck me as just, well, a negative thing to have in a contest where my expectations were positive.
[Edited at 2014-09-13 03:33 GMT]
[Edited at 2014-09-13 03:44 GMT] ▲ Collapse
| | | | | The plot thickens | Sep 13, 2014 |
I'm not sure I like this anonymity of ratings business. If I were intent on winning, there's nothing to stop me giving one star to all the other entries. I'm not going to do that, but I believe it's another flaw in the system.
| | | | | 1 should be considered the lowest possible rating | Sep 15, 2014 |
Hi Katalin,
Katalin Horváth McClure wrote:
If a translation is garbage (for example, entirely machine translation), it does not deserve even 1 out of 5 stars.
But there is no option to actually say ZERO, just to leave it empty, which then means it was not rated.
I think it would help staff to out-rule the garbage entries if there was a way to actually indicate that.
[Edited at 2014-09-12 21:33 GMT]
That's a possibility, though if the "1" rating is taken as the lowest possible rating, the same is accomplished.
| | | | | Thanks for the feedback, Michael | Sep 15, 2014 |
I don't think forcing people to rate or vote would be productive. There's no need to take the "rule out" feature negatively; it's just another aspect of the process of elimination to determine finalists. Strong entries rise to the top through the rating process, and eliminating entries which have no reasonable chance of winning helps focus voter/rater attention on the leading entries. Maybe it needs a different name?
| | |
|
|
|
| The plot is not as thick as you think! | Sep 15, 2014 |
Hi Elizabeth,
Elizabeth Tamblin wrote:
I'm not sure I like this anonymity of ratings business. If I were intent on winning, there's nothing to stop me giving one star to all the other entries. I'm not going to do that, but I believe it's another flaw in the system.
Staff monitor and review ratings, votes, and all other contest activity. If someone were found to be abusing the system in the way you mention, their ratings would be nullified. Abuse of the contest system is also grounds for disqualification and/or a ban from future participation.
| | | | | Should have been implemented right from the beginning | Sep 16, 2014 |
Hi Jared,
I didn't remember the rule-out feature but I think it should have been implemented from the very beginning of the contest. It would have saved (lots of) us from wasting precious time on junk material. There are at least 5 translations in my pair that are just plain garbage and no one got fooled. On the other end the good ones (and mine is not btw but I don't care...my ego is not that big) clearly stand out. So all in all, I guess the system just clears itself out and is qu... See more Hi Jared,
I didn't remember the rule-out feature but I think it should have been implemented from the very beginning of the contest. It would have saved (lots of) us from wasting precious time on junk material. There are at least 5 translations in my pair that are just plain garbage and no one got fooled. On the other end the good ones (and mine is not btw but I don't care...my ego is not that big) clearly stand out. So all in all, I guess the system just clears itself out and is quite fair for everybody. ▲ Collapse
| | | | | Yes, it is about respecting our time and effort | Sep 16, 2014 |
Christophe Delaunay wrote:
It would have saved (lots of) us from wasting precious time on junk material. There are at least 5 translations in my pair that are just plain garbage
Exactly.
When you count on the community to evaluate these translations, you need to at least respect that effort by removing the obvious obstacles.
When I pointed out that an entry was a 100% verbatim copy of Google Translate output, and suggested that the entry perhaps should be removed from the contest, I was told that using Machine Translation was not a reason for disqualification. When explained again that it was a rule violation, as the contest rules say very clearly the translation should be the contestant's own work, I received the same verbatim answer from support staff: using machine translation is not grounds for disqualification.
Let's not go into how that answer, given the circumstances, goes against common sense, but just focus on the time and effort issue. It seems that until a significant number of those sentences are tagged, staff will refuse to take notice.
I am sorry, but expecting that extra and mindless work (i.e. wasting precious time) from people who are volunteering their efforts is very disrespectful, IMHO.
If that is the attitude "from above", then don't be surprised that people are not keen on participating.
I am certainly done with contests.
| | | | | Count me out, too | Sep 16, 2014 |
Katalin Horváth McClure wrote:
Christophe Delaunay wrote:
It would have saved (lots of) us from wasting precious time on junk material. There are at least 5 translations in my pair that are just plain garbage
Exactly.
When you count on the community to evaluate these translations, you need to at least respect that effort by removing the obvious obstacles.
I am sorry, but expecting that extra and mindless work (i.e. wasting precious time) from people who are volunteering their efforts is very disrespectful, IMHO.
If that is the attitude "from above", then don't be surprised that people are not keen on participating.
I am certainly done with contests.
When I saw one entry that was clearly MT, I thought it must have been a joke. But then I realized there were more of them! This is supposed to be a site for professional translators and garbage like this just makes all of us look like a bunch of amateurs who have no idea as to what they are doing. I suddenly lost interest in all "Translation Contests". Sorting garbage is not my idea of fun. I agree with the previous posts: Please do something about this. Don't waste our time.
| | |
|
|
|
Christophe Delaunay wrote:
Hi Jared,
I didn't remember the rule-out feature but I think it should have been implemented from the very beginning of the contest. It would have saved (lots of) us from wasting precious time on junk material. There are at least 5 translations in my pair that are just plain garbage and no one got fooled. On the other end the good ones (and mine is not btw but I don't care...my ego is not that big) clearly stand out. So all in all, I guess the system just clears itself out and is quite fair for everybody.
I agree it could have/should have been implemented sooner. I for one admit to a miscalculation as to just how many entries were going to be made in some pairs! I figure now is better than later. The good news is that, thanks to those who have been rating, in many pairs the first round of "ruled-out" entries is being made clear (and so are the stronger entries), and in pairs I am reviewing now it seems the threshold has been reached which would allow for elimination of entries.
| | | |
Katalin Horváth McClure wrote:
I am sorry, but expecting that extra and mindless work (i.e. wasting precious time) from people who are volunteering their efforts is very disrespectful, IMHO.
If that is the attitude "from above", then don't be surprised that people are not keen on participating.
I am certainly done with contests.
I think when contests started, it was just for fun. I myself participated once in the first editions and didn't mind much about rules and so on. I just participated and gave my best. But since then, my criticism has grown.
I expressed already in other threads that for me it is not conceivable that contestants vote for other entries.
Similarly unconceivable is that we are supposed to waste our time rating machine translations. Any professional understands immediately what kind of translation he/she has in front, so why waste his/her valuable time with similar nonsense keeping in contest such a translation?
Once a machine translation has been recognized as such by a certain number of people, it should be ruled out immediately (and there should be the possibility to rule it out very clearly rating less than 1!!!) in order to spare time to other *volunteering people*, yes, absolutely! Never forget we spend our time "for free" and should not be bothered with such a blatant nonsense!
As I said, the contest has left its childshoes behind, and the rules should be changed appropriately!
| | | | | Nope this doesn't work | Sep 17, 2014 |
Katalin Horváth McClure wrote:
If a translation is garbage (for example, entirely machine translation), it does not deserve even 1 out of 5 stars.
But there is no option to actually say ZERO, just to leave it empty, which then means it was not rated.
I think it would help staff to out-rule the garbage entries if there was a way to actually indicate that.
[Edited at 2014-09-12 21:33 GMT]
Let's take two practical examples.
Brazilian Portuguese and European Portuguese are being treated like any translation outsourcer/client should: as two separate target languages. Legally they are one and the same language, but in practice they are quite different. This has caused trouble in previous Proz contests, so the situation now has been solved.
As a Brazilian, I won't touch scoring European Portuguese with a ten-foot pole. At least I shouldn't.
Now English is also one and the same language and, since this has caused no problem so far, it is being treated as one target language. However there are labels for the entries there, such as US, UK, Canadian, maybe Australian in other pairs, and "not specified".
I am not sure it would be fair for the speaker of one variant to judge another. Like Prof. Higgins said, "In America, they haven't spoken it for years", there is often some bias in this kind of judgment, so we'd better live without this issue.
Let's say that an overly ethical Brit, conscious of his/her prejudice, decided not to rate any entry stated as US, but only his/her countryfellows'. This should NOT automatically discard EN-US entries. The entire situation also applies vice-versa.
IMHO what COULD work is if a certain proportion of "1s" (i.e. judged and condemned) were reached, this would automatically bin that entry.
| | | | | Pages in topic: [1 2 3] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Entries in the qualification round can be "ruled out", a few more adjustments are on the way | Trados Business Manager Lite | Create customer quotes and invoices from within Trados Studio
Trados Business Manager Lite helps to simplify and speed up some of the daily tasks, such as invoicing and reporting, associated with running your freelance translation business.
More info » |
| | Pastey | Your smart companion app
Pastey is an innovative desktop application that bridges the gap between human expertise and artificial intelligence. With intuitive keyboard shortcuts, Pastey transforms your source text into AI-powered draft translations.
Find out more » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
