CPU load

[THobson]

There is a lot of stuff in this thread that I am sure someone could begin to make a turorial out of.  Though that person would need to do a lot of testing, for example, my experience is that an @ instruction can be smoother and less laggy than a  loop of the type:
#start
~0.5
?(condition):goto "start"

but not always.

[Terox]

an @ in reality is a loop, but more in the way of

#START
@ (condition): goto "SKIP"
~0.001
goto "START"

#SKIP

so i have no idea where you get the impression that a slower loop of say half a second or even slower can be more cpu intensive

[Mikero]

>encrypting files

ok, to save me a bit of time for the moment, have tracked down Amalfi's cpp2bin tool (version 1.2)

Gen Barron would be fairly familiar with it. It is, the so-called 'encryption' I'm on about. And I only use that word because other people have done so in the past, a sort of legacy thing.

For you guys to use it on files other than config.cpp (which it is hardwired to tableise) simply, and I mean, simply, rename your mission.sqm (eg) to XYZ.cpp and let this tool go to work. When done, rename it's output (XYZ.bin) back to whatever gave you a thrill in the first place.

I have tested it on various items like description.ext, mission.sqm and a few others. There are no ill effects I can detect, but I sure as hell could find them if I started *really* hunting.

It is quite logical that this tool from amalfi can 'encypt' most, almost any, 'script' file, regarldess of it's intent because the whole shebang of all scripted text begins and ends with classes that have a few and only a few, hard nosed variable types attached to them. Specifically, arrays, floats, integers and strings.

You will encounter various disparate script files, such as description.ext, such as mission.sqm, and often, config.cpp which have been encrypted in this fashion. I suspect binarize tool from bis has done the job on some of them but the intent is the same,

you can grab cpp2bin  here (200k)

Note that, as GenBarron pointed out on the digitalgrenade forum some time ago, this tool makes a muddle of #defines other than integer and is quite pernickity about 'correct' syntax. (use and location of comments)

For the most part tho, this would be a great start for you Thobson, because it must be _very_ close to the internal post scrunched text files the engine normally reads.

Just apply it to any one of your scripts to gain confidence that the bugger works. If as GenB and others have pointed out, that the engine re-reads any given script file,(this makes perfect sense now) you're on a winnner here.

Also, I have read elsewhere (haven't we all), that COMPRESSED pbo files are slower in operation that uncompressed. Your 'armies mission is riddled with compression.

Will bury my head deeper and try and improve on Amalfi's rather good tool.

[THobson]

Terox:

i have no idea where you get the impression that a slower loop of say half a second or even slower can be more cpu intensive

Not sure about more cpu intensive, but certainly generates lag.  I guess it is something to do with the scripted loop being interpreted on the fly and @ being some form of compiled instruction.  Clearly there must be a value of the wait interval where they are similar in performance terms.  My experience is that it depends on the complexity of the condition statement, but for a simple condition the @ beats the wait loop at intervals < 0.5

I put 5,000 BMPs on the map (I know this by heart now), each running an @ command to detect the damage of the vehicle and start the smoke script if necessary.   No lag whatsoever, providing I could not see the vehicles.  Change the @ to a wait loop and it was ugly anywhere on the island.  The wait loop I used was:

#loop
~0.5
if (getDammage _this < 0.98) then {goto"loop"}


The @ command I used was:

@(getDammage _this > 0.98)


Mikero:

Also, I have read elsewhere (haven't we all), that COMPRESSED pbo files are slower in operation that uncompressed. Your 'armies mission is riddled with compression

I've not read it.  How do you know the pbo has compression in it?  How do I create one without compression?

[Planck]

Well, Amalfi's PBODecrypter reports most files in your mission pbo are compressed.

Files not compressed are ogg's and jpg's, which are already a compressed format.

17 files were reported with checksum errors, but the program managed to recover them.

Your readme and trignotes were two of them and most of the rest were .sqs files, except for one which was description.ext.

I re-pboed your mission using StuffPBO which doesn't compress anything, the resulting PBO was 16,041 Kb as opposed to 15,289 Kb with the compressed version.



Planck

[Mikero]

A quick jump in in case Hobson has just been terrified. hmmm on the other hand that may be some nice revenge. Nah, I'll be nice.

>17 files reported as corrupt

this is an 'undocumented feature' of Amalfi's, they aren't. He mistakenly did a signed, not unsigned, checksum. Winpbo and my own depbo.dll confirms them as ok.

Apart from stuffpbo (which being old and no longer supported *might* have errors) you can use Amalfi's Makepbo version 1.5 Makepbo has the option of which types of file, if any, to compress, but make sure you grab 1.5, not, the more available 1.3 which has problems.

All of this stuff is here at ofpec in the editing depot.

>>have read elsewhere (haven't we all)

that was meant to be dripping sarcasm., more along the lines of yet another myth, but easy to check out whether true or not.

[Planck]

StuffPBO has never failed me yet, I use it to make any PBO I want.

Why would anyone want to make 'compressed' PBO's anyway, it just means the game has to uncompress them before use.  And it doesn't save that much on space used.



Planck

[UNN]

so i have no idea where you get the impression that a slower loop of say half a second or even slower can be more cpu intensive

~ is just @ with some extra parameters. Try running this:

Code: [Select]

Hint "Hello"

~

Hint "Goodbye"
You get the following error message:



So ~ is just a version of:

Code: [Select]

_Delay=_This Select 0

_WaitUntil=_Time+_Delay

@(_Time>_WaitUntil)
If you passed a parameter of 0.001 or used ~0.001, it would really pause for 0.002. Time increments in values of 0.001, BTW you can test this easy enough.

Code: [Select]

@(True)
So the above is faster, with less commands to process, because it pauses for a preset time period. But like any loop, dont try and do to much in one go.

Code: [Select]

@!(IsNull (NearestObject [_This Select 0,_This Select 1]))
Is probably just as bad as:

Code: [Select]


#L

~0.0009

If (IsNull (NearestObject [_This Select 0,_This Select 1])) Then {goto "L"}
And this is not much different to the above:

Code: [Select]


#L

_TTime=Time+0.0009

@(Time>_TTime)

If (IsNull (NearestObject [_This Select 0,_This Select 1])) Then {goto "L"}

[Mikero]

>why would anyone compress pbo's anyway?

On 'standard' missions and *especially* campaigns, the saved space isn't large, it's HUGE. PMCfury eg, (253+ missions)compresses to one tenth it's size (don't quote me, 10 meg to 1 meg from memory)

As you point out it has little effect on jpg's and sounds (in fact, jpegs EXPAND !!! For this reason you wont see it used on most / many? addons that are models (Bas stuff eg). Thinking about it, I'm guessing this is where the myth has been generated from because people started noticing many addons are uncompressed. They put 9 and 6 together and came up with halfpence. Still, all the same, it's worth checking out if compression really does slug the engine or not. Anyone playing Hobson's masterpiece would pick the difference, real quick.

Afaik, stuffpbo doesn't provide resistance signatures in it's pbo file. I'm not aware of anything that actually takes note of those signatures but it is a deficit.

[Planck]

StuffPBO is Pre-Resistance anyway.

I placed THobsons mission with the one I re-PBOed with StuffPBO in the missions folder, replacing the original.

It played the same as before, no discernable difference in my fps rate anyway.
It even still loaded my saves from the original.


Planck

[General Barron]

@Mikero

Could you open a new thread to discuss this "compression" that you are talking about? I'd like to ask some questions, but I don't want to use Thobson's thread. Plus I'm sure others would be interested in such a thread as well.

A dialogue is a good idea.  I avoided that because from the start of the intro I wanted the player to be completely immersed.  I could try and put the dialogue at the start of the intro though, or as a Plan B I could create a separate pbo file for lower spec machines.

Err... IIRC, your mission has no cutscene at the start of the real mission (after the intro). So why not put the dialog there? When the player first gets to climb that damn cliff I mean.

I may be completely wrong in this but the feeling I have about OFP is that there is some threshold of ‘load' beyond which performance deteriorates rapidly.

I'm not so sure about that. Like I said, an old mission of mine (Platoon Commander) has like 150 units in a relatively small area, but nearly no scripts. It lags my computer a noticable amount at the start of the fighting. Your mission was still playable on my computer, and it didn't seem to have an 'excessive' amount of lag to me, considering how un-great my system is, and how much stuff is happening in that mission at one time.

for some reason boats use a lot of cpu, probably because the system has to keep recalculating their height as the waves come in.

That might be an idea. "Sink" that boat at mission start, since I doubt it gets used anyway. Would add to the feeling that "there's no turning back".

[Fragorl]

Goto #label vs not

I get the feeling that using 'goto', or in fact looping in any way, is a large source of lag - simply the act of completing one loop and returning to a label causes a heck of a lot, for some reason

I ran into this problem during some drop scripts, where I was tring to produce a continuos glow in a single spot. One or two scripts worked fine, but after three or more were running consecutively   the lag demon reared its head. So, I replaced

#loop
drop ...
~0.01

with

#loop
drop ...
drop ...
drop ...
drop ...
drop ...
drop ...
drop ...
drop ...
~0.05

and completely counter to what you might expect, the framerate shot up. Unbelieveably, the second version still outperformed the first version with 10, 20, 30 and all the way up to about 50 drop commands in the space of one loop. By 'outperformed' I mean that the colour was more intense (drops were in the same spot), less drops were missed AND framerate was much higher/less lag. This is what led me to the conclusion in the first paragraph. I've no idea why it might be, but it is.

[THobson]

I have some experience that when a goto is executed the program pointer goes back to the top of the script and then searches forward until it finds the lable.  Directionally that would support your experience, though I am surprised that is was so significant.

[Mikero]

>program goes back to top of script

I believe this to be correct too and you are *definately* onto something here. The engine doesn't compile scripts only interprets line-at-a-time. It has no knowledge of where the #loop is until it encounters it.

This _might_ be a fundamental difference between sqs and functions. Maybe, perhaps, and well worth checking.

[General Barron]

I have some experience that when a goto is executed the program pointer goes back to the top of the script and then searches forward until it finds the lable.

Very interesting, and I'll definately take note of that. From now on, if I have any loops, they will go at the top of the script!

The beauty of this is, the function, which is basically a linear script is already in memory, it doesnt require the hard drive to be accessed or data written across to ram, its much more instant and therefore a lot less cpu intensive

This _might_ be a fundamental difference between sqs and functions. Maybe, perhaps, and well worth checking.

Err... yes and no. A "function" is just a STRING, which is executed as code via the "call" command.

Standard practice is to save these strings into a variable, with something like my_func = preprocessfile "blah.sqf" or my_func = loadfile "blah.sqf". The difference between "preprocess" and "load" is that all the comments are taken out of preprocessed files (I believe--although it would be simple to check).

Since this variable is saved, it would make sense that OFP could access it faster, since it doesn't have to load up the file each time it is run. However, I doubt that OFP actually RUNS the code in any different way than normal script code (except that functions can't have delays or 'goto's of course).


So Terox's statement is somewhat correct. The .sqf file itself is nothing special. It is the act of storing these functions into a variable that makes them different than scripts. I had to point this out because look at Terox's example code:

Code: [Select]

?!(local server):exit
~ 3 + random 1

#INIT
_a = 0
  #START
    ~1
    if(tx_StopSqs)then{goto "END"}
    _a = _a + 1
    if((_a == 3)&&(({alive _x}count tx_destroy)==0))then {[b]call loadfile "tx_sys\server\CheckB.sqf"[/b]}
    if(_a == 4)then {[b]call loadfile "tx_sys\server\CheckA.sqf"[/b]}  
    if ((_a == 5) && (count tx_damcon >0))then{{_x setdammage 0} foreach tx_damcon}    
    if(_a >=5)then{_a = 0}
    goto "START"      
       
#END
;; does some boring end of mission stuff then exits
exit
Notice that he LOADS the function every time he calls it. This is essentially the same as loading a script each time it is run, like OFP does. If he wanted "a linear script already in memory", then he should have loaded the file ONCE, and saved it into a variable (local or global).

But if he were to do that, then I think there would be some performance increase--at least in very fast looping scripts that repeatedly execute another script (or function).

Another great thing about functions is that line breaks are indicated by semicolons; instead of the 'return' key, so you can lay out long strings, if/then, or do/while conditions in a neat way.


EDIT

I think you guys already mentioned this, but I'm kinda dense . Anyways, if a "goto" causes the script to be read again from the start, then that might explain why an @condition usually runs faster than an very short "goto" loop.

The @condition is prolly checked every clock cycle or so--while the "goto" statement causes the script to be searched thru until the label as mentioned. Unless the label is near the top of the script, then there might be quite a bit of code for OFP to search thru--every 0.5 seconds or so, depending on the delay.

So which is faster? Searching thru X lines of code to find a label, then checking a condition, every 0.01+ seconds? Or checking a condition every 0.001 seconds? I imagine the answer is going to be a combination between how much code is above the label, and how complex the condition being checked is. With that in mind, it seems likely that an @might be faster than a ~0.5 delay, if it is down under a bit of code.

Good stuff to keep in mind, instead of just blindly making loops everywhere, in fear of using an @condition.


Oh yeah, and I have more evidence to support this (labels being searched for from the top): if you ask a script to "goto" a non-existant label, the script will exit instead of giving an error or continuing on--indicating that it just kept searching until the end of the file, as opposed to 'preprocessing' the labels on first execution.