Sequential File - NULL Handling (Best Approach Please)

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
nmacolin
Participant
Posts: 19
Joined: Mon Jun 16, 2008 6:01 pm

Sequential File - NULL Handling (Best Approach Please)

Post by nmacolin »

Hi all,

I'm just trying to figure our how to best handle null in the Sequential File stage.

I have the following Jobs in this scenario.

Server Job
Unidata --> Transformer (Oconv(dates)) --> Sequential file

Parallel Job
Sequential File --> Transformer (various) --> Db2 table.

In the serverjob I have set the delimiter to Comma and enclosed "", I do not have anything for the null string option.
Ihave defined the meta data for the Sequential file for the last four columns as
Integer
Date
Integer
Date

In the Parallel job.
I initially started with the sequential job having the same meta data as the server jon sequential file.
But since my file looks like this.

"REFERENCE","DESCRIPTION",AMOUNT,"BUDGET_REFERENCE",TRANSACTION_DATE_RW,"TRANSACTION_DATE",POSTING_DATE_RW,"POSTING_DATE","REPORT_NUMBER_NO_LONGER_USED","USER_BATCH_NUM_BATCH_INDEX_ID","BUDGET_ALLOCATION_EG_MT",JOB_COSTING_DATE_RW,"JOB_COSTING_DATE","CLAIM_NO",POSTING_DAT_PRCSSD_INTO_CLM_RW,"POSTING_DAT_PRCSSD_INTO_CLM",JOB_CSTNG_DT_OF_CST_PRGRSS_CLM_RW,"JOB_CSTNG_DT_OF_CST_PRGRSS_CLM"
"50-01","OBAL",2000000,"50-01",14124,"01-09-2006",14197,"13-11-2006","","","MT",14124,"01-09-2006","",,"",,""

The last four columns are empty. So I get the following error when trying to use the Parallel Sequential File.

##E IIS-DSEE-TFIG-00187 05:51:49(000) <SEFBudgetJournals,0> Field "POSTING_DAT_PRCSSD_INTO_CLM_RW" has import error and no default value; data: <empty>, at offset: 102

I've tried using the null string in the Format Tab, but that looks like it need to be applied to all non VARCHAR values, which is not ideal.

Where should I put the null handling ?

In the output to the ServerJob Sequential file ?
In the Transformer of the ServerJob ?

In the input Sequential file in the Parallel Job?

There seems be a lot places to intercept a null.

In short I want to load an empty field if that is what is in the source.

Thanks in advance for your help guys. Can you give an example in your reply.

Regards
Nick
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

:? Nick, you just repeated everything from your existing post on this problem into a brand new thread. There's absolutely no need for shenanigans like that which just leads to confusion with disjointed conversations going on in two different places.

viewtopic.php?p=305800
-craig

"You can never have too many knives" -- Logan Nine Fingers
nmacolin
Participant
Posts: 19
Joined: Mon Jun 16, 2008 6:01 pm

Post by nmacolin »

[quote="chulett"]:? Nick, you just repeated [b]everything[/b] from your existing post on this problem into a brand new thread. There's absolutely no need for shenanigans like that which just leads to confusion with d ...[/quote]

Sorry Craig new to this site, comment on the other post was that it should be moved to the parallel forum. Hence the repeat.

Anyway here is the answer

In the Server Job which generates the file they have set the
Format Tab :: Default Null String "

In the Paralle Job which read the file the have set
Format Tab : Field Defaults : Null Field Value "

Yep may be obvious now .....

Regards
Nick
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Actually, the comment was probably a request for the Moderator to move the post. We can't move our own posts but there's no need to duplicate everything as eventually Roy will find some time to put things where they belong. :wink:
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply