Hi Ray,
Thanks for all of your comments!
I am using Before Routine to Concatenate the Files but the problem is its giving me an error message as follows
***********
UDBBulkLoadJob..BeforeJob (ExecSH): Error when executing command: Cat Stage_Party_Varchar.txt Stage_Party_Integer.txt Stage_Party_Date.txt > Stage_data.txt
*** Output from command was: ***
SH: Cat: not found
************
Please send me your comments!!!!!
What is the performance issue, When I directly load data into the UDB database and when I use a Seuential Stage to get the outputed data and then use the UDB Bulk load Stage to load the data?
How to improve the performance of an ETL Job? or what are the criterias which are needed to keep in mind when Designing,Developing and Testing an ETL Job using Data Stage v6?What are the performance issues which can be applied to help the performance of an ETL job?
What are the possible testing tools which can be used when using Data Stage?
What are Containers?(Just a short note)
Please let me know these details.
ray.wurlod wrote:SMKraj wrote:Hi,
I have the following questions:-
1. Currently I am doing a Performance Testing in a Job which inserts data into the UDB DB2 Stage serially as well as parallelly.The Question is When I load data serially the performance is faster but when I use the parallel laoding the performance slow compared to the serial load!
I have increased the Array Size and the Transaction Limit in the UDB DB2 Stage.
Please advise!!!!!!
2. Looking at the above scenario now i working on loading the data in to the Sequential file and then using the UDB Bulk Load Stage I would load the data into the table using the REPLACE option.
The question is I need to concatenate 6 different sequential files and the load into the Bulk Stage.
I can either use the CAT cmd in UNIX or use the Before Routine! But I am not sure how to do so???.
Please send me ur invaluable comments for working towards my goal. Also please let me know which of my approach is better suited or if there is any other possible approach!
Awaiting an early reply from ur end.
In the before-job subroutine (which you find in the Job Properties window) chooses ExecSH as the name of the routine. This will execute the command that is in the Input Value field.
In the Input Value field put a command to cat the six files, directing the output into a seventh file that DataStage can use. For example:
Code: Select all
cat file1 file2 file3 file4 file5 file6 > file7
Another way, if you're using DS 6.x or later, would be to do a similar thing using the filter capability of a Sequential File stage; first run the command
Code: Select all
cat file1 file2 file3 file4 file5 file6
as the filter command; the Sequential File stage will read stdout from this command, which will obviate the need for file7.
As to the performance question, are you measuring DataStage's loading of the data file(s) for the DB2 bulk loader, or the performance of the DB2 bulk loader itself? This was not clear in your question.