Hello,
I have 50 Cobol copy books(EBCDIC Format) that need to be populated into Teradata with minimal Transformation. The Job design would be
CFF ----> Modify/Transformer--->TD
One option is to create the 50 Datastage Jobs(read and load).
But I am looking for some generic option Like create Multiinstance jobs. But I believe CFF does't support schema files at run time.
Incase anyone is aware about any generic solution, please let me know.
Regards
neeraj
Multiple Cobol Copy Books
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 50
- Joined: Tue Jan 19, 2010 4:14 am
-
- Participant
- Posts: 3593
- Joined: Thu Jan 23, 2003 5:25 pm
- Location: Australia, Melbourne
- Contact:
This is very hard to do as the Complex Flat File stage does not support column propagation and dynamic ETL. Cobol mainframe files each have a copybook that describes them and this tends to be unique for each file. You will need at least 50 DataStage jobs - one per unique copybook. You may be populating more than 50 target tables - those copybooks that have multiple arrays and different record types can have many types of outputs.
You are going from complex data to flattened data - the flattening process may mean you have many more than 50 target tables.
Minimum transformation can also be tricky when it comes to handling blanks or nulls or empty fields, sometimes the CFF stage needs to convert these to a default value which is a type of transformation.
I have found that if the copybooks are accurate you can load a copybook to Teradata in under an hour but if there are formatting and consistency and data type problems in those copybooks it could take several days to debug a single file.
You are going from complex data to flattened data - the flattening process may mean you have many more than 50 target tables.
Minimum transformation can also be tricky when it comes to handling blanks or nulls or empty fields, sometimes the CFF stage needs to convert these to a default value which is a type of transformation.
I have found that if the copybooks are accurate you can load a copybook to Teradata in under an hour but if there are formatting and consistency and data type problems in those copybooks it could take several days to debug a single file.
Certus Solutions
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn
Blog: Tooling Around in the InfoSphere
Twitter: @vmcburney
LinkedIn:Vincent McBurney LinkedIn