Search found 56 matches

by debrujr
Tue Aug 10, 2010 9:07 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: DB2 stage Write method
Replies: 5
Views: 2835

I'm not following your question. The DB2 stage has a selection for which "Write Mode" you want to use. Insert, Update, Delete, Insert then Update, etc... Are you concerned that it will error due to table constraints? If so then use an Insert then Update method OR if the table isn't too lar...
by debrujr
Mon Aug 09, 2010 8:56 am
Forum: General
Topic: Datastage Challenge
Replies: 7
Views: 2286

Personally I would do a mix of ksh with embedded sql. I know this is a DS forum but in my head it works out better going that route. In essence I would group my key columns by table name in one select and then pass the table and column params to another query that will plug in the variables in their...
by debrujr
Fri Jul 16, 2010 6:35 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Upsert Inserting Duplicates - DB2 - Enterprise
Replies: 10
Views: 3513

I assume that I am seeing all of the Rollbacks because it is trying to update but rolling back then to insert (My own theory so probably wrong) since the row doesnt exist. Basicly the upsert updates else inserts. But the other issue is that it inserts everything and updates nothing when my primary k...
by debrujr
Fri Jul 16, 2010 6:17 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Upsert Inserting Duplicates - DB2 - Enterprise
Replies: 10
Views: 3513

Am I incorrect in thinking that the Upsert option will update all records that match the predicates given and those that do not will be inserted?
by debrujr
Fri Jul 16, 2010 6:12 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: DB2 Connector Performance question
Replies: 10
Views: 8435

How is the table setup? Are you hitting indexes, contention of insertion, building keys, ri, etc..? Is it set at row level locking? How many records are in the table? Runstats? I think chulett is on the right track, try doing it outside of DS. I had people pointing the finger at DS when we had some ...
by debrujr
Thu Jul 15, 2010 2:26 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: DB2 Connector Performance question
Replies: 10
Views: 8435

Have you tried it using the DB2 Enterprise Stage? I dinked around with the DB2 stage and seemed to do better with the Enterprise. That does seem absurdly slow for your environment. What is the width of your row?
by debrujr
Thu Jul 15, 2010 1:43 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Upsert Inserting Duplicates - DB2 - Enterprise
Replies: 10
Views: 3513

But that is why I am confused.... Shouldn't the update statement update the rows that already exist based on the key constraint and then the insert add those rows that do not exist currently? The staging records are distinct on the key so I am unclear as to what would be causing a duplicate. The for...
by debrujr
Thu Jul 15, 2010 11:26 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Upsert Inserting Duplicates - DB2 - Enterprise
Replies: 10
Views: 3513

Also I am seeing a lot of "Rollback to savepoint" messages on the db2top sesion. Indicator of something wrong?
by debrujr
Thu Jul 15, 2010 10:57 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Upsert Inserting Duplicates - DB2 - Enterprise
Replies: 10
Views: 3513

I seem to have it working now. I re-built my test table with the primary key being the two columns that I am using to define an update record(predicate). It worked perfectly after that BUT for me to utilize an upsert in this manner on PROD would encur a large amount of work and redesign considering ...
by debrujr
Wed Jul 14, 2010 11:07 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Upsert Inserting Duplicates - DB2 - Enterprise
Replies: 10
Views: 3513

I just changed my ArraySize to 1 thinking that may solve it but I still have the same result.
by debrujr
Wed Jul 14, 2010 8:55 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Upsert Inserting Duplicates - DB2 - Enterprise
Replies: 10
Views: 3513

Upsert Inserting Duplicates - DB2 - Enterprise

I am trying to perform an update/insert on one of my warehouse tables from staging and it seems to be inserting duplicates. I am using the DB2 enterprise stage. The incoming records are distinct on the match keys as well as the data to be updated. If I start with an empty table it will insert all of...
by debrujr
Fri Apr 23, 2010 9:26 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Type of Slowly Changing Dimension used in DW
Replies: 4
Views: 2629

Are you asking about the differences between using a SCD in DS (Whether it be a 1 or 2) as compared to performing it outsude DS? Basicly the performance difference between DS and non DS? Another option if you wanted, you could "stage" the data for a delta table to be used in an external SC...
by debrujr
Fri Apr 23, 2010 9:06 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: I/O Tuning
Replies: 7
Views: 2326

Well I have finally gotten back to researching this and it appears that it is an issue with how it is pulling from the DB itself. My test DB is running with 4 partitions so DS executes 4 statements to pull over the data from each partition in parallel. This all seems to be working correctly. The pro...
by debrujr
Fri Mar 05, 2010 11:05 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Designer Indicative of Throughput While Running?
Replies: 9
Views: 5241

When I started this headache I was using the DB2 operator (it pulled sequentially with auto partitioning just fine but would not run in parallel nor run at all with DB2 partitioning set) and recently I have been working with the DB2 Enterprise operator and I can't get it to connect even with the par...
by debrujr
Fri Mar 05, 2010 10:22 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Designer Indicative of Throughput While Running?
Replies: 9
Views: 5241

ray.wurlod wrote:Take a look at the Score.

In particular, are there inserted tsort operators on the inputs to the Join stage? Inserted buffer operators? Either of these will "block" the pipeline. ...
Ray,

Thank you for your reply, but it's not set to sort and the buffer is set to it's default setting.