DataStage PX and Optimised ABAP extract and IDOC extract
Posted: Wed Dec 06, 2006 8:38 am
hello
Recently i was commissioned to be involved with a large SAP ledger implementation; This required the abap extract and idoc extract. The volume is approximtely 1TB per night; I have architected a two ETL server solution ( AIX ) machines, that will act as catalyss to move data only. There is an intermdiate machine ( Z/OS ) that will be doing the transformations.
So,
SAP ( AIX ) --> ETL1 ( SAP plugin ) --> Z/OS --> ETL2 --> DW ( datawarehouse) .
For scalability I have installed DB2 DPF on the DW as other groups and departments of the client will be coming on board in the future.
Question:
I want to be able to propose the most efficient solution for both ETL1 and ETL2 (DS EE) however, the same solution doesnt apply for the "PULL" portion of architecture?!?
rationale;
ETL2 install 8 to 10 way machine with DB2 DPF toincrease performance of the large accounting transaction datafiles coming from Z.OS
ETL1 however, the intent is to extract and directly insert into Z.OS;
limitation:
DB2 EE stage does not work with MVS DB2.
Only DB2 API stage is supported
I am not sure if DB2 API supports partition DB2 updates/inserts? and I would like this confirmed?
Question:
If SAP R/3 was originally written for DS Server, is it optimised ( parallelism enabled) in DS EE? Is it perhaps in an optimised state in DS 8.0?
IF anyone can answer these with rationales i would be appreciative.
thanks in advance,
wmaass@verizon.net
Recently i was commissioned to be involved with a large SAP ledger implementation; This required the abap extract and idoc extract. The volume is approximtely 1TB per night; I have architected a two ETL server solution ( AIX ) machines, that will act as catalyss to move data only. There is an intermdiate machine ( Z/OS ) that will be doing the transformations.
So,
SAP ( AIX ) --> ETL1 ( SAP plugin ) --> Z/OS --> ETL2 --> DW ( datawarehouse) .
For scalability I have installed DB2 DPF on the DW as other groups and departments of the client will be coming on board in the future.
Question:
I want to be able to propose the most efficient solution for both ETL1 and ETL2 (DS EE) however, the same solution doesnt apply for the "PULL" portion of architecture?!?
rationale;
ETL2 install 8 to 10 way machine with DB2 DPF toincrease performance of the large accounting transaction datafiles coming from Z.OS
ETL1 however, the intent is to extract and directly insert into Z.OS;
limitation:
DB2 EE stage does not work with MVS DB2.
Only DB2 API stage is supported
I am not sure if DB2 API supports partition DB2 updates/inserts? and I would like this confirmed?
Question:
If SAP R/3 was originally written for DS Server, is it optimised ( parallelism enabled) in DS EE? Is it perhaps in an optimised state in DS 8.0?
IF anyone can answer these with rationales i would be appreciative.
thanks in advance,
wmaass@verizon.net