Hi,
I have a requirement to use a generic job to read source file and load data into table.
Each time I have a new source file, the corresponding target may have different number of Checksum/Surrogate Key columns.
Example (Scenario 1):
---------------------------
Source --> File 1 (Col1, Col2, Col3, Col4)
Target --> Table1 (Col1, Col2, Col3, Col4, Checksum(Col1,Col2), Checksum(Col3,Col4))
Example (Scenario 2):
---------------------------
Source --> File 1 (Col1, Col2, Col3, Col4, Col5)
Target --> Table1 (Col1, Col2, Col3, Col4, Checksum(Col1,Col2), Checksum(Col3,Col4), Checksum(Col5))
So each time I receive a source, I have to generate different number of checksum column.
Is there any way I can achieve this using generic job like
(Source --> Generate different number of Checksums --> Target)
Thanks,
Rohit
Generate multiple Checksum/SK in a generic job
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 41
- Joined: Wed Oct 08, 2008 9:19 am
there are a couple of ways... the way we have been doing it is to set up for a good maximum # of columns you might need, for example say you need 3 or 4, then you might set up for 6 or 8.
If a particular input is blank, then the related checksum column on the output would be blank (and no work done, beyond carrying the empty columns around for a short time). Your using job can drop the unused columns.
so if you needed 2, the first 2 columns to your shared code have the data that will be run thru the checksum, the others empty. If you need 3, the first 3, etc. Its a little clunky, but its flexible and worked well for us.
If a particular input is blank, then the related checksum column on the output would be blank (and no work done, beyond carrying the empty columns around for a short time). Your using job can drop the unused columns.
so if you needed 2, the first 2 columns to your shared code have the data that will be run thru the checksum, the others empty. If you need 3, the first 3, etc. Its a little clunky, but its flexible and worked well for us.
-
- Participant
- Posts: 41
- Joined: Wed Oct 08, 2008 9:19 am
Thanks for the replies.
Even we are following similar approach. Putting a maximum number of Checksum.
For each checksum we provide column/s using parameter. This job is RCP.
At the end we Drop unnecessary checksum columns (again controlled by parameters for particular instance/value file).
Thanks for all help.
Even we are following similar approach. Putting a maximum number of Checksum.
For each checksum we provide column/s using parameter. This job is RCP.
At the end we Drop unnecessary checksum columns (again controlled by parameters for particular instance/value file).
Thanks for all help.
Rohit