dsjob -report vs DataStage C++ API
Posted: Tue Jul 08, 2014 2:21 pm
Hi,
We have DataStage 8.5 (Does it make sense if I say "we would better have individual version in the Release drop down as there is much difference within 8.x versions" ?)
There are 10000 jobs in a project to collect the run time statistics and currently we are using dsjob -report (without logon clause) to get that information by making it loop over 10000 jobs.
Its a Single tier topology so Engine/Metadata/Services all on single Linux box, The question here is "Does the dsjob call have to go through authentication each time in a loop" ?
At this moment it is taking 40 mins to get data for all 10000 jobs and if it has to go through authentication and each time attach to the same project just wondering if redoing this using C++ API would save some time ?
It would take considerable time to code it using C++ API and want to take experts comments to avoid the embarrassing situation of having spent lot of time on getting it to work and finding out no improvement in time consumed.
Thanks.
We have DataStage 8.5 (Does it make sense if I say "we would better have individual version in the Release drop down as there is much difference within 8.x versions" ?)
There are 10000 jobs in a project to collect the run time statistics and currently we are using dsjob -report (without logon clause) to get that information by making it loop over 10000 jobs.
Its a Single tier topology so Engine/Metadata/Services all on single Linux box, The question here is "Does the dsjob call have to go through authentication each time in a loop" ?
At this moment it is taking 40 mins to get data for all 10000 jobs and if it has to go through authentication and each time attach to the same project just wondering if redoing this using C++ API would save some time ?
It would take considerable time to code it using C++ API and want to take experts comments to avoid the embarrassing situation of having spent lot of time on getting it to work and finding out no improvement in time consumed.
Thanks.