Help Needed - Job Compilation is extremely slow

[newsaur]

The compilation of my job is taking almost 4 mins to complete and this is becoming intolerable.

A typical job has around 30 stages in total including those within containers. It has 1 transformer stage and others are simply Lookup's DB2 Enterprise Stages and etc.

I notice that while the job is compiling, my workstation's CPU is 100% occupied (memory is quite free). Is this normal? I thought the compilation is done at the server end.

The pre-compilation already takes around half the time and the job compilation takes another half.

Apart from this it is slow even if I simply press the "Run" button. The Job Run Options window pop-up only after 15 secs. It happens to all workstations at our site.

I am using a Intel Pentium 4 1.8GHz CPU with 1GB memory.

Any suggestion would be of great help.

Thanks.

[ArndW]

I had a similar issue a while ago with 7.1 and I could never figure out the problem - the same sequence of keystrokes with my userid on another person's PC would work just fine, but on mine it could take 10 minutes before the screen refreshed. I finally got rid of the issue by doing a client reinstall and I would suggest you give that a go.

[ray.wurlod]

A lot of the pre-compilation work for parallel jobs is done in the client, which explains the CPU consumption you observed. Slowness at compile time (or, indeed, any task that needs to refresh the client) may be attributable to a number of things, even to there being very many jobs in the project, or a very busy LAN.

[vsurap]

Your Client configuration looks fine. We had a similar issue long back. The issue was in the connectivity (between Client and Server).
Do FTP a big Flat file from your client to server and see if it is a Network connectivity/Bandwidth issue.

[ArndW]

Isn't the c++ compiler on UNIX limited to a given number of concurrent compiles, so if that limit is reached the compiler waits until it can compile? That might explain some slowdowns in compilation times.

[roy]

When I had the compiler licenses issue, Arnd referes to, the compile time would be very long, since the compilation was actually waiting longer then it took the previous compilation to finish and was waiting even up to 15 minutes, while most of the time the machine was idle, in case I started compiling when another user was already compiling a job.
So it's esay to notice if this is the case.

Just to be on the safe side find out if and how many users your compiler license supports.

In case of network, check to see if the slow client has a slow connection configuration compared with other clients.

When you notice a slow compile try to monitor the server and network to find out if annomalies occur (use your system administrator/s), such as heavy CPU ussage or memory swappings.

Most important keep us posted of what solution worked for you

IHTH,

IHTH,

[jking123]

We have the same issue. It compiles a little faster if we compile from a machine closer to the server as opposed to over a vpn or a longer connection. But it is still sloooooooooow in both cases.
anyway am looking at the various environment parameters. Will psot if I find anything.

[newsaur]

It seems to me the performance is much more affected by the number of fields being passed in links (in our case > 40) as opposed to the number of stages.

We had some jobs with many stages but only processing a few number of fields (<8) and they seem to compile reasonably fast.

Having said that we still don't have a solution to this - apart from adapting ourself to cope with this