Hi,
I have 150 files (text) as source. I need to load 150 files data into 150 Oracle tables. Without any changes for columns order and data types.
Every file has different number of columns. Delimiter is same (^) for all the files.
I need to load 150 files data into 150 tables every day and simultaneously
File1 --------- Table1
File2 ------- Table2
File3 ------- Table4
My approach is I have to develop 150 Datastage jobs for loading 150 tables.
Instead of developing 150 jobs, is there any solution in datastage that can load 150 tables simultaneously (From 150 files).
If there is no solution in datastage , is there any other way to solve this issue?
If you have any solution, can you please explain me.
Thanks
Srini
Loading files into Oracle tables
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 38
- Joined: Sat Dec 29, 2007 9:58 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Use one job, with a schema file (whose pathname is passed as a job parameter) to describe the structure and Runtime Column Propagation enabled. Pass the file name and the Oracle table name as job parameters also.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Of course make sure it's a multi-instance job if you're going to run 150 loads with it at the same time. AND make sure your job log retention is set to a number of days (1 or more) rather than number of runs. It's not pretty when the log of a running job instance is deleted automatically by the limit, as happened at one of my clients.
Regards,
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.
-
- Participant
- Posts: 133
- Joined: Wed Mar 05, 2003 4:19 pm
- Location: Lima - Peru. Sudamerica
- Contact:
Re: Loading files into Oracle tables
Hi Srini
You can make 150 jobs to load each table to Oracle, another idea you can group similary table (records) to load in a single job, in this case, you build fewer jobs.
Another alternative would be create a single job and use parameters and enabled RCP.
Your job must define this parameters
1) Schema file name
2) Table name
TEXT FILE ---> TRANSFORMER --> ORACLE STAGE
In the TEXT FILE add the roperty Schema file = #SCHEMA_FILE#
In the transformer don't defined fields, because dont appear fields
In the Oracle stage, define the Table Name = #TABLE_NAME#
Mark the job as Multiple Instance, this is a check in job properties
Then when you run the job asign an Id and pass the parameters, in this case, the name of the schema and table.
You can make 150 jobs to load each table to Oracle, another idea you can group similary table (records) to load in a single job, in this case, you build fewer jobs.
Another alternative would be create a single job and use parameters and enabled RCP.
Your job must define this parameters
1) Schema file name
2) Table name
TEXT FILE ---> TRANSFORMER --> ORACLE STAGE
In the TEXT FILE add the roperty Schema file = #SCHEMA_FILE#
In the transformer don't defined fields, because dont appear fields
In the Oracle stage, define the Table Name = #TABLE_NAME#
Mark the job as Multiple Instance, this is a check in job properties
Then when you run the job asign an Id and pass the parameters, in this case, the name of the schema and table.
Saludos,
Miguel Seclén
Lima - Peru
Miguel Seclén
Lima - Peru