Easy null handling

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Offshored2002
Participant
Posts: 17
Joined: Wed Apr 16, 2008 6:39 am
Location: Arlington, VA

Easy null handling

Post by Offshored2002 »

DS 7.5.2: I have an XML file with some empty fields, a small example without header info follows:
<FIELD1></FIELD1>
<FIELD2>1234.56</FIELD2>

Using the XMLInput stage, I can carry the fields across with no problems by checking the "Replace empty values with NULLS" in the stage properties. All of the FIELDs listed above will eventually be loaded to DB2 mainframe (Version 8.1) whose attributes are DECIMAL(15,2) NULLS=YES.

Immediately following the XML stage is a transform stage to set some additional fields such as timestamps. The next stage is the DB2 load. All the computed fields from above are automatched and the attribues on output are the same as above. However DS is dropping records that contain the null values in them (warnings coming from director log) unless I test/set them, i.e.:
if IsNotNull(link.FIELD1) then link.FIELD1 else SetNull()

Some of these tables have 300-400 fields in them. I would prefer not to have to manually change every derivation across numerous tables. I can't find any way to tell the transform stage to just pass the null values thru.
OddJob
Participant
Posts: 163
Joined: Tue Feb 28, 2006 5:00 am
Location: Sheffield, UK

Post by OddJob »

If the transformer's input and output is defined as Null=Yes, and you're not doing anything but setting the output derivation to the input data then you shouldn't need to handle the null, it should just pass through as you are hoping for.

DS will drop records when you try to pass a null value to one of the transform functions. Also, using null values in stage variables gives arbitrary results and should not be done.

You could try using RCP on the transformer stage and just reference the fields you actually need to do any transforming on (although using RCP is quite contentious on this forum, I'll brace myself for the flaming!).
Offshored2002
Participant
Posts: 17
Joined: Wed Apr 16, 2008 6:39 am
Location: Arlington, VA

Post by Offshored2002 »

I went into admin and turned RCP (new to me) on for our project and recompiled and it worked. I am not going to get into the debate about the pros and cons of RCP.
Post Reply