DSXchange

Hello Forum,

What are the different factors that affect startup time. Trivial jobs that I create that have one input and one output file and one transform usually have a startup time of less than 5 seconds. However, trivial read of a table often results in startup times between 15-30 seconds.

Is this abnormal behavoir?

Thanks,

Greg

The obvious one is getting the section leader processes started and talking to the conductor processes. There are many potential variables here especially on a cluster of machines.

Connecting to database servers from multiple player processes also can take a lot of time. It's not just connection, it's also authentication, preparing SQL, and so on. These costs are worse in parallel execution simply because they have to occur N times.