Missing records need help

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
wojtask
Participant
Posts: 10
Joined: Tue Nov 02, 2004 4:04 am

Missing records need help

Post by wojtask »

When I run a job it writes records to 2 Oracle tables A and B (one Upsert (update and Insert) and one Load) exactly same data. In table A I have 3000 rec written and in B 6000 (all from input) only 9 rows are written to reject table, no errors in log appear.
Both A and B are empty in the beginning
Any idea what can cause this??

Thanks in advance

BTW: Where should I set NLS_LANG to avoid this:
main_program: An NLS map <windows-1250> is specified, but NLS_LANG is not set. The NLS map must be set to an ICU codepage which is equivalent to the character set specified by NLS_LANG
-On the server?
-on the environment?
-in DS?
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Why do you think there is a problem?

Upsert - update and insert - will check the key column you defined whereas the load will be append only. So check the logic with the keys mentioned in the Plugin.

NLS is set when you install the product. You may be able to upgrade to include the NLS.
wojtask
Participant
Posts: 10
Joined: Tue Nov 02, 2004 4:04 am

Post by wojtask »

Maybe I'm wrong but I think that upsert (with insert) works that way:
if there is a record with a particular key value update it
if there is not insert a record
So if I have 6000 input records after upsert I should have 6000 in a table, right??
Sainath.Srinivasan
Participant
Posts: 3337
Joined: Mon Jan 17, 2005 4:49 am
Location: United Kingdom

Post by Sainath.Srinivasan »

Check whether there are any duplicates.
wojtask
Participant
Posts: 10
Joined: Tue Nov 02, 2004 4:04 am

Post by wojtask »

Nop, unfortunately there are no duplicates so it's not that case.
There was no PK in the B table (Load) I added on the Client_id which is unique and it was still the same
T42
Participant
Posts: 499
Joined: Thu Nov 11, 2004 6:45 pm

Post by T42 »

Have you ensured that the log is not saying anything negative (such as dropping records)?
wojtask
Participant
Posts: 10
Joined: Tue Nov 02, 2004 4:04 am

Post by wojtask »

I discovered that on DS server there was 2 nodes (CPU's?) and total number of rows was smaller two times if I changed the apt file to use 4 CPU's numer of rows was 4 times smaller. I used Key Generator instead and everything worked fine. Thanks.
T42
Participant
Posts: 499
Joined: Thu Nov 11, 2004 6:45 pm

Post by T42 »

nodes != CPU.

Nodes = Processing flow.

Please read up on Parallel Extender Configuration File (Chapter 11) on your DataStage Manager guide (man_gde.pdf). By reading this, you will gain an important insight on how Parallel Extender (EE) works.
Post Reply