Simultaneous Read and Write to a DB2 Table

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
mouni
Charter Member
Charter Member
Posts: 49
Joined: Tue Jul 11, 2006 11:30 pm

Simultaneous Read and Write to a DB2 Table

Post by mouni »

I have a requirement to do simultaneous read and write to a DB2 table.

I need to lookup a DB2 table, if a value is not found in the lookup, compute a new value and update the lookup table so that all subsequent input rows with same key will use the same value from lookup table.

The steps I am doing is:

Read Input
For every input record
Lookup DB2 Table for the value
If Value is found
Write to Output
Else
Compute the random value
Write to Db2 Lookup Table
Write to Output
end If
end For


Probably in Server job we can do this using Hash files, but in Parallel job I am not able to implement this using a DB2 Stage. Please Help.
prabu
Participant
Posts: 146
Joined: Fri Oct 22, 2004 9:12 am

Re: Simultaneous Read and Write to a DB2 Table

Post by prabu »

mouni wrote: Read Input
For every input record
Lookup DB2 Table for the value
If Value is found
Write to Output
Else
Compute the random value
Write to Db2 Lookup Table
Write to Output
end If
end For

Probably in Server job we can do this using Hash files, but in Parallel job I am not able to implement this using a DB2 Stage. Please Help.
DB2 stage is not an equivalence to Hash files :roll:

I believe you are missing a step here:-

1)
If your reference[dimesnion] and the target[fact] , are not the same
1)You should load the input data into your reference table first, then load your target(dimenion load).

i beleieve you are trying to load your dimesnion and the fact table(s) in the same job.

2) Also i undersand , your input key can appear multiple times.
say like

Product_id price
P001 100
P001 120

In this scenario you should want either the first record or the last record, use remove duplicates to keep either the first or the last.

3)Avoid using table lookups as much as possible , use a lookup fileset instead

regards,
Prabu
kumar_s
Charter Member
Charter Member
Posts: 5245
Joined: Thu Jun 16, 2005 11:00 pm

Post by kumar_s »

I am not able to implement this using a DB2 Stage
What is the problem you faced while implementing it?
Is it you have row level commit?
Impossible doesn't mean 'it is not possible' actually means... 'NOBODY HAS DONE IT SO FAR'
Post Reply