Improve compilation speeds on 8.7
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 42
- Joined: Tue Oct 20, 2009 8:36 am
Improve compilation speeds on 8.7
We are migrating from Datastage 7.5 (AIX 5.3, XL C 8.0) to 8.7 (AIX 7.1, XL C 11.1) and are experiencing quite long compile times. As far as I can see, it's not client related but the time seems to be rather spent by the XL C compiler process on AIX. Is there anthing known on how to improve compilation speed under 8.7? All suggestions are welcome.
1) Reduce the number of transformers in the jobs--each one is a separate compilation process
2) Turn down compiler optimization options so the compiler spends less time trying to optimize transformer executable code
3) Get a bigger/faster machine to compile on
I don't recall the options to set for #2 (maybe something related to -O)...you will have to look into the compiler documentation, but perhaps someone else here has done similar recently. You set the options in the APT_COMPILEOPT environment variable in DataStage Administrator.
Regards,
2) Turn down compiler optimization options so the compiler spends less time trying to optimize transformer executable code
3) Get a bigger/faster machine to compile on
I don't recall the options to set for #2 (maybe something related to -O)...you will have to look into the compiler documentation, but perhaps someone else here has done similar recently. You set the options in the APT_COMPILEOPT environment variable in DataStage Administrator.
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.
-
- Participant
- Posts: 42
- Joined: Tue Oct 20, 2009 8:36 am
Compile options are pretty much at standard for 8.7, almost the same as suggested here.
Under 7.5 we have:
So difference is the qspill (but conforming to docs that should be only set when code doesn't compile at all) and 64 Bit but I'm not sure if it is a good idea to deactivate this.
Compilation is done for a single job in designer but it also counts up similarly for our batch compilation where we call datastage from command line. The tested job is quite simple and takes twice the time (20s instead of 10s) and with lots of jobs, this sums up to quite some problem...
Code: Select all
APT_COMPILEOPT -O -q64 -c
APT_LINKOPT -G -q64
Code: Select all
APT_COMPILEOPT -O -c -qspill=32704
APT_LINKOPT -G
Compilation is done for a single job in designer but it also counts up similarly for our batch compilation where we call datastage from command line. The tested job is quite simple and takes twice the time (20s instead of 10s) and with lots of jobs, this sums up to quite some problem...
Check with IBM for possible known issues and resulting patches. During our install and migration to 8.7, we had several general performance issues with some of them related to the Oracle backbone. Multi-job compiling was not used for a period until they announced that it was okay. We had a bad time with imports, too.
I regret I can't give you internal details on that. I would be surprised if IBM didn't have something useful for you from their end.
I regret I can't give you internal details on that. I would be surprised if IBM didn't have something useful for you from their end.
Franklin Evans
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872
"Shared pain is lessened, shared joy increased. Thus do we refute entropy." -- Spider Robinson
Using mainframe data FAQ: viewtopic.php?t=143596 Using CFF FAQ: viewtopic.php?t=157872