Improve compilation speeds on 8.7

DSUser2000 · Post by **DSUser2000** » Tue Feb 19, 2013 3:51 am

We are migrating from Datastage 7.5 (AIX 5.3, XL C 8.0) to 8.7 (AIX 7.1, XL C 11.1) and are experiencing quite long compile times. As far as I can see, it's not client related but the time seems to be rather spent by the XL C compiler process on AIX. Is there anthing known on how to improve compilation speed under 8.7? All suggestions are welcome.

pandeesh · Post by **pandeesh** » Tue Feb 19, 2013 5:12 am

Are you compiling individual jobs or using bulk compile option?
i prefer using bulk compile for compiling large number of jobs.

Thanks,
pandeesh

jwiles · Post by **jwiles** » Tue Feb 19, 2013 11:11 am

1) Reduce the number of transformers in the jobs--each one is a separate compilation process
2) Turn down compiler optimization options so the compiler spends less time trying to optimize transformer executable code
3) Get a bigger/faster machine to compile on

I don't recall the options to set for #2 (maybe something related to -O)...you will have to look into the compiler documentation, but perhaps someone else here has done similar recently. You set the options in the APT_COMPILEOPT environment variable in DataStage Administrator.

Regards,

DSUser2000 · Post by **DSUser2000** » Tue Feb 19, 2013 11:30 am

Compile options are pretty much at standard for 8.7, almost the same as suggested here.

Code: Select all

APT_COMPILEOPT -O -q64 -c
APT_LINKOPT -G -q64

Under 7.5 we have:

Code: Select all

APT_COMPILEOPT -O -c -qspill=32704
APT_LINKOPT -G

So difference is the qspill (but conforming to docs that should be only set when code doesn't compile at all) and 64 Bit but I'm not sure if it is a good idea to deactivate this.

Compilation is done for a single job in designer but it also counts up similarly for our batch compilation where we call datastage from command line. The tested job is quite simple and takes twice the time (20s instead of 10s) and with lots of jobs, this sums up to quite some problem...

FranklinE · Post by **FranklinE** » Tue Feb 19, 2013 1:12 pm

Check with IBM for possible known issues and resulting patches. During our install and migration to 8.7, we had several general performance issues with some of them related to the Oracle backbone. Multi-job compiling was not used for a period until they announced that it was okay. We had a bad time with imports, too.

I regret I can't give you internal details on that. I would be surprised if IBM didn't have something useful for you from their end.