How far do mission reviewers go?

[Acecombat]

Being a mission reviewer myself , i'd like to ask OFPEC's mission reviewers as to how far they go when they review a mission by this i mean , when you get a crappy mission on hand.

I seem to be getting many these days , they are annoying/frustrating basically everythings in the ingredient that'd make you wanna delete them but obviously you cant without giving a proper review since it still works ...  

Now for example i got this little mission in for a review its about a cobra pilot(player) going to Nogovas airport to waste russian armour there , basically all you need to do according to objectives is go there clean it all up and its done. SO i play this mission i show those russians whose boss and burn every armour up with my fellow apache crewmen , however the objective doesnt change it means i havent met the objective criteria so i start busting up tanks which were only disabled and think maybe the author used a EAST not present trigger instead of naming every armour and checking if they are disabled like that canmove tank1 command or whatever it is (its evading my mind right now).

So i bust every tank , heli/plane whatever in sight still the objective doesnt tick and by now i've wasted about 25 mins on this mission which isnt very enjoyable. So what do i do now? Obviously theres a fault with the missions objectives , do i open up the mission inn editor and check it out or leave it at that and decide that the mission didnt meet submission requirement and delete it?

How do you guys at OFPEC do it , thats what i wanna know so i can judge better and see what the standards on this are.

Another such example is another mission i am playing (its basically a Battlefields [BIS mission] rip off) but its long and laggy and is showing no signs of ending whatsoever and ive wasted 30+ mins of gametime on it already. What to do? Reason i'm asking this is because this issue is dancing on the fineline of beta testing/mission review and its all grey country here.  You could either act the tough reviewer or go soft and comment on the fault and let the author know , basically making yourself a beta tester in the process.

Thanks for any replies  .

[macguba]

Minor cock-ups, such as an Objective not ticking, will affect the score slightly and get mentioned in the review providing they do not have serious effects.

Major cock-ups result in an IM to the author describing the problem and giving him two weeks to fix it.   If he does, well and good.   If not, the mission is deleted.   (We're not harsh though, if he replies and says can I have a bit longer that's fine.)    Occasionally, for one reason or another, such a mission gets through and achieves a score of zero.

If it's really bad you might suggest to an author that he put it on the beta board and spend some time testing before resubmission.   However, reviewers are not beta testers.    Although most reviewers do look at a mission in the Mission Editor (I certainly do, not that I've done many reviews) they are not obliged to go hunting for faults.

In the first example you quote I would send the author a message saying the mission is non-functional please fix it.   (The mission is obviously faulty so there is no need to unpbo at this stage.)   Say here are a few tips on what I think might be wrong, here's a few tips on where to get information on where to fix it, and maybe even here's a few tips on how to make the whole thing less crap more fun.    But from the sound of it he's staring down the barrel of a 2 or 3 anyway.     Mission making is hard though, and I'm a firm believer in giving noobs as much encouragement as possible.   Sometimes they come up trumps in the end.

The second example is trickier.    This is one I would open in the Mission Editor right now and see what's going to happen.    You can then take a view on how to proceed.    It sounds as if it's just crap rather that faulty, which just means a poor mark.  

Crappy missions like these are a real problem for the community.  They waste a lot of good people's time. and create disappointment, frustration and bad feelings all over the place.    As you probably know we're trying to encourage more people to put their missions on the beta testing board and then beta test other people's missions.    Most people who make crappy missions do so because they don't know any better:  if they are nudged into a journey of discovery it a) keeps them off our back and b) means that the next mission/version will be much better.

Like many people I've played a lot of missions, many of them as a beta tester, reviewer or judge.    And in general, having seen the download blurb, readme, overview, intro, briefing and first three minutes of the mission, you know roughly what standard it's going to be.   There are exceptions but they constitute well under 10%.   The trick with crappy ones is to cut your losses and finish the whole procedure as quickly as possible, but without being unfair to the author in case it is one of the few.   Unfortunately it's not an easy trick to carry off.   You have my sympathies.

To answer the general point of how far to go, as a rule it is not sufficient simply to play a mission once.    Apart from anything else you don't know what you've missed.    I tend to play it once and then look in the mission editor.    I'll play parts of it again, watch the other outro if there is one and discover how it occurs, run around bases and investigate interesting looking corners.    Alternatively I may play through it again in a different way with setCaptive true or with some other cheat.     I'm sure other reviewers have their own techniques but I know that they all do something.

For a very simple mission this may may mean one play through plus five minutes.  (Plus thinking about marks and writing the review of course.)    However, for a complex mission it may mean spending three or four times the normal mission time.     If it's complex-good that is merely a pleasure.  If it's complex-crap it can be really hard work ... but that is the cross the mission reviewer has to bear, and is partly why so few people have what it takes to be a mission reviewer.

[The-Architect]

For me, I find that a reviewer has to be a beta tester too. Just simply because some people don't use the beta testing forum at all. That means, as you guys know, we get missions that simply arn't ready for review. Rules are rules however and if the thing works we have to review it. Authors get bad scores because they are innocently oblivious, impatient or stupid. Maybe a big sign on the Pending page saying,

"Make sure your mission has been tested! Reviewers can be brutally honest!"

would help.
Also there's the fact that Ofpec Missions Pending, is a real easy place to upload a file. Some people are only submitting small files that fit on a forums page. They keep their file sizes down so that they can still throw the mission into a thread. Bigger missions can find their way onto the Pending page just simply because an author might have no-where else to host it. I know because I used to do it in the early days.  

As always in cases where people are behaving poorly, education is the answer. Unfortunatly I'm a bad teacher with little patience.  

[rado1265]

To answer the general point of how far to go, as a rule it is not sufficient simply to play a mission once.    Apart from anything else you don't know what you've missed.

I suggest you, Ace, to read this quote at least ten times, because this is a similar thing that was I, the arrogant sod, tell you, when you responded that you've played my mission only once before the reviewing on OPFLASH.

Just for you to know.

[Acecombat]

Thanks for the detailed reply Macguba , i agree with what you said.

Crappy missions like these are a real problem for the community.  They waste a lot of good people's time. and create disappointment, frustration and bad feelings all over the place.    As you probably know we're trying to encourage more people to put their missions on the beta testing board and then beta test other people's missions.    Most people who make crappy missions do so because they don't know any better:  if they are nudged into a journey of discovery it a) keeps them off our back and b) means that the next mission/version will be much better.

Exactly this is a major problem , of all the missions we get only 50% are worthy of being reviewed rest are just too crap and out of the remaining 50% only a few manage good scores and are enjoyable. For me as a reviewer its very tiresome and disheartening to write the same stuff again and again advising the author about his mistakes often too common and basic and to learn to rectify them and give some thought next time more on the mission design etc etc.... its a thorughly rehearsed rhetoric by me i have to launch in to 70% of the time  :-X.

If it's complex-good that is merely a pleasure.  If it's complex-crap it can be really hard work ... but that is the cross the mission reviewer has to bear, and is partly why so few people have what it takes to be a mission reviewer.

70% of the time its complex-crap and that can really burn you down as a reviewer especially if your the only SP reviewer around and got to review every mission sent in  . But ofcourse as i said before too its a curse reviewers have to suffer from , but to what degrees do they it in was all i wanted to know.

@Architect

Yes your right to some extent we are beta testers as well i can see the gist some of my reviews give and they almost seem like a beta test report more like a enjoyable review telling you the ups and downs of the missions , more then often it end up telling you of the downs and how to rectify them.

@karantan:

Damn your one persistent bugger , you dont give up do you? For your information i did replay your mission over again quite a lot (even though it was crap) i had a save point from which i had to constantly reload as i was trying to save my squad mates from the inevitable doom that used to befall them everytime i played it. Even if i did play it again and again from start i doubt my mind will change on the verdict , anyhow you got a review here aswell at OFPEC it was pretty much the same as mine so can you cut your crying already?

[rado1265]

Hey, you started to bugging me here, not all around!  But let us drop this once and for all, shall we?

Here is not the problem my mission (man, would you ever see this?), but you being unable to take any constructive criticism, or even advices, as a reviewer.  If you have the the power to criticise others work, then you should be able to take the constructive criticism yourself.  You can consider this as my humble advice to you, though I'm not a reviewer.

It's good for you that you seek some advices about the mission reviewing; maybe it isn't too late for you as a reviewer after all.

[THobson]

Calm down every body.

Now I have a supplemental to this question.  Not that I am a reviewer - I don't pass all the tests.  The point is how far does the mission builder go in communicating with the reviewer.  The standard answer I guess is through the readme file included with the mission.

I have a bit of a problem with this.  In my current mission there are a lot of things the player may, or may not notice - it is all part of creating an atmosphere, sometimes something will be noticed other times not, the intention being that the occasional player that catches sight of any of these will realise that there is a lot more going on than they might first think.  For example: they may visit a location that is not one of the objectives and get a cut scene; they may catch sight of a civilian watching them from a building in an abandoned town; they may find the site of a massacre, or witness a prison break with tragic consequences etc. etc. (yes there are more).  Or they may notice none of these.

Now if I were to explain all these in the readme file it would seriously impact on the experience I am trying to create for the player - but clearly if the mission is to get a balanced review the reviewer should be made aware of these features.

Any suggestions - this is not urgent by the way - my mission is months from completion.

[rado1265]

Well, in your case, THobson (I asume it's for your Abandoned Armies) the solution can be quite simple: ask macguba (he's also a reviewer, isn't he?) to review the mission; he's in the beta testing proces from the beginning, so he knows the mission quite well.

[The-Architect]

Now if I were to explain all these in the readme file it would seriously impact on the experience I am trying to create for the player - but clearly if the mission is to get a balanced review the reviewer should be made aware of these features.

Any suggestions - this is not urgent by the way - my mission is months from completion.

Write a section in the Readme called Spoilers. Put all your stuff in there. If people want to read it then they will, if they don't they won't. In case people accidentally read it, you may want to add another readme called Spolers and put in in with the original readme.  The Reviewer should read it if he's doing his job right.

@Karantan
You've been warned about airing dirty laundry in public. Watch you don't get banned.  

[dmakatra]

Karantan, I agree on constructive criticism. Everyone should be given a little kick in the arse sometimes. I've been smacked there a couple of times myself. However, you don't exactely do it in a pleasent way. If someone saws your mission, well, that's one thing. A mission is a couple of files, it's not really a person. Therefor you can be more harsh IMO. Criticising a person it's a good thing to take it a little easier.

:beat: *Gets Shot* :beat:

[macguba]

I am a reviewer, though I'm not convinced that somebody who has done a great deal of beta testing - as I have in this case - is best qualified to do the review.    A fresh eye may well be preferable.   Also, it would interrupt the cab rank system, whereby a reviewer always takes the first mission available whatever it is.      

I had a similar problem with Un-Impossible.    My solution was to create a [very] detailed readme and then just to trust the reviewer to do the right thing, which he did.    If that isn't going to work in a particular case, the solution is simply to send an IM to Artak, the Missions  Depot Admin, saying that there is a lot of stuff in the mission that might be missed, and could he bring this to the attention of the reviewer.    Once a reviewer has been allocated he can, if needs be, contact the author to discuss how best to review the mission.   Alternatively, if he examines it in the mission editor, he will discover that there is much going on.   The readme is for communicating with the player, not the reviewer.   You shouldn't need to communicate specially with the reviewer, but in this particular case all that needs to be done is make the reviewer aware that there is more to this mission than meets the eye.

Although Abandoned Armies is a case where it is reasonable for the reviewer to have just one proper play-through, since the mission time is typically 8-12 hours plus retries.

That's a very long winded way of saying it's a solvable problem, don't worry about it.  


Edit:  well done Armsty! :thumbsup:

[rado1265]

@dmakatra

Noted.    Obvously I just don't know how to kick an arse politely.

I'll work on it, me, the cave man, until I break my leg! ;D

[Acecombat]

Hey, you started to bugging me here, not all around!  But let us drop this once and for all, shall we?

Here is not the problem my mission (man, would you ever see this?), but you being unable to take any constructive criticism, or even advices, as a reviewer.  If you have the the power to criticise others work, then you should be able to take the constructive criticism yourself.  You can consider this as my humble advice to you, though I'm not a reviewer.

It's good for you that you seek some advices about the mission reviewing; maybe it isn't too late for you as a reviewer after all.

I started what? Who created this thread? Drop this for once and all? Oh yeah then why did you cry in the very first post you made? I dont think your ready to give it up any time sooner. The only thing you understand is boot , you got the boot by anmac at opflash.org and why was that , oh yeah you were 'offering' me constructive criticism right?  

I swear if i anmac hadnt deleted your mission and the review and the subsequent discussions that took place after it i'd have posted it here and let everyone else decide who needs to learn how to take in constructive criticism.

I am all for taking advices as a reviewer , why else would i open a topic here relating to it.

The irony here is killing me , first i go through a complex-crap/frustrating mission then face a author who wasnt able to take the review given to him tell me i should learn how to take it , wow it couldnt get any better then this.

If you wanna whine anymore do so via PM i dont want this topic getting down because of a 'complete waste of a time' argument with you.

[Noon416]

Okay AceCombat & Karantan, I don't want to hear another word from either of you over this matter. Period.

I don't care who started it or who continued it or anything else for that matter.
You've both aired your unchanging opinions many times on this problem and as you both can't come to a peaceful resolution over it, something has to be done.

As such, unless the next post I see from either of you is a promise to leave the matter lie and work on it privately (and constructively), I will give the offending party a 4 week holiday from OFPEC.

You have both been warned...

[ProudPotter2490]

Hey.
Noon416 is right Acecombat and karantan. It is getting rather petty now, don't you think :-\?
ProudPotter2490 :afro:

[Sui]

Oi... everybody drop it please. Right now.

[Morglor9]

The readme is for communicating with the player, not the reviewer.   You shouldn't need to communicate specially with the reviewer, but in this particular case all that needs to be done is make the reviewer aware that there is more to this mission than meets the eye.

Perhaps a separate .txt file entitled "For The Reviewer" alongside the Readme would solve the problem.

[THobson]

I think any file that is 'Top secret do not open - this is only for someone else to look at' would be like a candle to a moth for most people.  It certainly would be for me.

I am convinced there will be an easy solution (from an IM exchange with the reviewer to putting it all in a password protected Word document -   I might even write a little encription program, encript the file and only give the reviewer the key  I realise I could just use IM for the document, but as a closet programmer this would allow me another fix.)

I have started writing the document and there is a lot in it that I would be very happy to share with other mission makers, but only after they have played it.  Reading it through they really are spoilers.  Having said that any serious mission maker would de-pbo it and then unpick the logic - IMing me on anything they didn't understand.

I am relaxed about this.  There will be a solution.  It was a thought that was in the back of my mind when I saw the title to this thread so I just jumped in, with the added benefit that it has reduced the average temerature around here.

[Wolfsbane]

*Leaps in through the closed window*

More input from moi.

When I first came to OFPEC, way back in the days of the original concept5.com site, and it was Karls Mission Depot that was running things on a seperate site, it was a different world indeed.

Once we took on Karls work, and intergrated it, and I became a Mission bod, I tried to set down quite firm standards on how you should review a mission.
Here's what I used to do when I was reviewing:

Start off by checking briefings, images used, readmes, addons, etcetc, and noting all these, for points of clarity, detail and so on.
Once I got into the mission I used to have a £5 dictaphone sitting on the table beside me, and I'd dictate into it, points as I came across them, such as how the start was, the scripting, any intros, etcetc.
This meant I didn't have to keep alt-tabbing to write in Notepad or something.

When it came to playing the     , I'd play it the way the author designed it. Following the obvious choices, and sticking to the 'rules'.

Once I'd completed the Mission, I'd then write up the review, taking in all the points I'd noted.

I'd then try to break the Mission.

By that, I'd try to skip around markers, waypoints, and screw the Mission around to see if I could beat it.

Classic example I remember, was the highly ove -      Mission by Devilchaser, where you had to kill three super warriors. Right at the start, I ran them all over in a UAZ.

That's breaking the Mission.

I'd then re-write the review, taking into note these points as well, to give a balanced review.

I'd then phaff around a bit and see what fun and replayability the mission had.

To review one mission properly, could take anything up to and over thirty minutes to and hour, easy.

It's also about being harsh and being nasty. I've had missions I've scored zero, I've had missions I've sent back to the author with screaming abuse as a letter, because he'd put a cutscene in, which during the playing of, your character would be shot by AI soldiers still moving around!!! That reeks of not trying to beta test.

The scoring goes as Macguba said, from the titivation of a briefing affecting it very slightly, and the scripting being rubbish, having it flung back in your face.

Now if I were to explain all these in the readme file it would seriously impact on the experience I am trying to create for the player - but clearly if the mission is to get a balanced review the reviewer should be made aware of these features.

Any good reviewer will notice these without being told. If they don't, it raises the question of their reviewing ability. Put them in, but give the mission a catchphrase with something like: "With added Reality!" which will get people looking out for it.

I also noticed the bitching here, I don't have time to read it all, I'm a busy soldier, but guys, keep it clean, keep it happy, keep it sweet, and if that fails, go round his house and smash his head in.



Reviewing should be in depth, but people don't always put the time in, people don't always put the effort in, scores can be hideously off the mark, but at the end of the day, it's a personal opinion (Which should be a neutral opinion!) designed to make finding a mission for you to play, just a bit easier. Use the score as a bookmark to find the mission, then read the review and then look at the addons required.

That's how I find a mission I want to play.

[THobson]

Now if I were to explain all these in the readme file it would seriously impact on the experience I am trying to create for the player - but clearly if the mission is to get a balanced review the reviewer should be made aware of these features.
 

Any good reviewer will notice these without being told.

I beg to differ.  I am talking about a mission that will require anything from 8 to 20 hours of mission time (not playing time) to complete and which because of its design would need to be played several times before all the logical possibilites could have been exhausted and that is assuming the reviewer knows what all these possibilities all are.  I currently have a 12 page word document (and it will grow with the next beta version) of 'spoilers' -  things that I do not expect even the best review to notice more than a fraction of.  I would be very happy to share this with you privately to see if you still think I am wrong.

Just in case my tone of voice is not clear

[Mikero]

ok, in an obvious bit of self promotion for the beta board I hope everyone here will see the logic in what I quote below

of all the missions we get only 50% are worthy of being reviewed rest are just too crap and out of the remaining 50% only a few manage good scores and are enjoyable. For me as a reviewer its very tiresome and disheartening to write the same stuff again and again advising the author about his mistakes often too common and basic and to learn to rectify them and give some thought next time more on the mission design etc etc.... its a thorughly rehearsed rhetoric by me i have to launch in to 70% of the time  

70% of the time its complex-crap and that can really burn you down as a reviewer especially if your the only SP reviewer around and got to review every mission sent in

Because it's complex crap it should be passed thru for many hands make easy work.

I really could pick and quote every sentence, every phrase of the above quotes, but for goodness sake, gentlemen, the answers for me, glare obvious.

The mere fact that a mission gets passed through indicates the opposite, it has NOT passed. That alone alerts the mission maker that he has a problem. The reviewer need hardly make furthur comment to the maker if he doesn't want to.

If maker doesn't care, it's the right call in the 1st place (and saves the reviewer a lot of agony mail). If it's truly crap, it wont survive in there and will drop out of sight, real soon.

If there's all the usual problems then all the usual problems will be listed by one or many testers rather than the reviewer rehearsing the same old lines.

And, to answer Thobon's query, if the reviewer missed the 'spoilers', it will, utlimately, get passed back out of the beta board telling the reviewer he got it wrong in a far nicer way than one on one hate-mail.

I've said my piece on this, it's my 2 cents, and I apoligise for pushing my barrow.

[penguinman]

well, im not following this really but i heard when macguba said this in his first post on this topic

Major cock-ups result in an IM to the author describing the problem and giving him two weeks to fix it.  If he does, well and good.  If not, the mission is deleted.  (We're not harsh though, if he replies and says can I have a bit longer that's fine.)    

well i submited a mission alls good and everything no problems fully beta tested,  

So I submit it and its pending

a mission reviewer(chrisbonowhich or somthing) posted a coment on it and sent me a IM saying that I shouldnt be submiting it if i havent posted it on the beta testing board but I dont understand what he is talking about because I did beta test it. and I know 2 people who beta tested it several times in the ofpec beta testing board and will vouch for me.

so i put a coment on the mission saying that it was beta tested but he never responded.  I dont want my mission to get deleted.

what am I suposed to do.

here it is
http://www.ofpec.com/missions_depot/index.php?ID=1258

[The-Architect]

I am chrisbobnorwich.
The reason I said what I did was because of the way you worded your message, here's a quote.

"This is my first test so I dont know if somthings wrong or not but I have external addons listed and they dont show up "

Because I speak queens english I understood it to mean that it was a first test.
However, I later read that that was not what you meant. That's fair enough.
Nobody is going to delete your mission because of what I put into the comments box. That's not how it works.

Please don't think that I need to write a response. I understood after you clarified it. That's good enough for me. Anyone reading the comments should understand that too. No drama.

[penguinman]

lol, ok just wanted to make sure

but i just realised, you were one of the people who had beta tested it.