Issue with Parallel loads with logical nodes

[vskr72]

We have a simple job that uses a direct table to table dump with a Copy stage in between. This is what we did:
- We created 2 logical nodes in the config file and used that config file in the current job
- Added the 2 nodes in the node map constraint field
- Rest of the options are defaults.

When I run this job, the number of records that load in the target is twice of what we have in the source table. If I increase it to 4, then the target has 4 X.

I am new to DS. Can any one tell me what do i need to configure in this case?

Satish

[mavrick21]

"Added the 2 nodes in the node map constraint field " ->Do you have any particular reason for adding this?

Can you remove it from node map constraint field and check?

[vskr72]

Even if I remove the nodes in the Nodemap constraint, it still is duplicating based on the number of nodes.

[chulett]

Then you've got a partitioning issue. Can you describe your job design - stages, partitioning settings per stage, etc?

[vskr72]

This is my job design:

Oracle Ent Stage --> Copy --> Seq.File

Oracle Stage:
- have a SQL override with conditions rownum < 50000
- Buffering is default
- Combinability - Auto
- Preserve partition default

Copy Stage:
- Preserve partition propagate
- Auto Partitioning

Seq File:
Default settings

Thanks

[vskr72]

Also have this in the job properties for $APT_ORACLE_LOAD_OPTIONS:

OPTIONS(DIRECT=FALSE,PARALLEL=FALSE)

[chulett]

Why have 'load options' when there is no loading going on?

If your Oracle stage's Execution Mode is parallel, try setting it to sequential.

[udayk_2007]

Remove the copy stage if it is not required and try running the job. Check if you get the same result

[Sainath.Srinivasan]

View the run statistics from the director with the rows per node mode enabled. That will give you an idea of which stage is increasing the volume.