DSXchange

It's still a requirement that the reference data set can fit in memory. Does your 10 million+ row reference data set fit in memory? Can you push the join back into the database, with a BETWEEN condition in the join specification? Well, the 10 million+ row reference data set is a stream input and I ...

Range lookups compile as transform operators. Therefore you really are looking at a "Transformer" stage. Now see if you can figure out the "field not found". This is interesting. So, If Range look ups compile as transform operators. How is the performance going to differ for a r...

Looks like we can't go to 8.5 yet. I am leaning towards splitting up the file with the help of a shell script. But, I just heard about XSLT. If I use the style sheet, do you think DataStage will still read the entire XML file into memory. In the Job have seen it looks like we are providing the XSLT ...

lstsaur,
Can you please let me know how I can use this work around? Which technology are these StAX parser and DOM subtrees built in?

Thanks,
Nagin.

Yes Ray, Please let us know the names of the institutes. It would be great to learn from the Master himself.

Hi, I am trying to load an XML file into a Oracle table column. And, the XML files can be really big, they can go upto 10Gb. This oracle table is more of a control table. It will have a row for each xml file received with Received timestamp and a unique id and other things populated Any ideas how th...

I found a work around, I had to add a filter stage after the sequential file stage and specify the where condition is not null for each of the not nullable columns. Ofcourse, this works as I know the not nullable column names. But for a truely dynamic metadata scenario I don't know how this would wo...

Gregg,

greggknight wrote: BEFORE_COMPANY:nullable string {null_field=''};

BEFORE_FACDIVN:nullable string {null_field=''};

Looks like this works for strings that are nullable. How can I reject strings coming in as null but are not nullable

No, I am okay with String field accepting numbers. Are you saying we can use Num() in the schema file ?

Still the main purpose is I need to reject null strings at source schema file level

If I remove nullable property for Col_One, that row is getting rejected if it has null. Again, Col_one is integer. So, If I pass in a null or a string in Col_one it is rejecting successfully. I am just getting a warning in the log saying that import error and no default value. Samething does not wor...

Thank you Nil Kamal for null_filed. It works without any syntax problems.

But still, the source sequential file is rejecting if it gets a null in place of an integer but not a string. Is the string null handling different?

Any ideas all you experts out there??

By default columns are not nullable in a schema file. So, I tried it, but it is not getting rejected. If I specify Column:not nullable the job throws an error and says Unknown schema type. Here is the content of my schema file record{final_delim=none, delim='|', quote=none, charset="ISO8859-1&q...

Ray,
I have a similar requirement. I need to reject if the incoming column is null with schema file at the source sequential file stage itself.

Can you please share if you have an example of how to specify null field value properties in the format string or column definitions?

Thank You.

The Generator property folder for a date data type includes Use current date as an available property to add. Thanks Ray. But, I don't need to populate date, I actually need to populate a next value from oracle sequence and the others some strings, I will have to get them from job parameters I think.

[Moderator: Split from this post, similar <> same] Thanks! Solved it. How did you solve it? I have the same scenario. The source file is schema file driven, and I need these few extra columns added. I need to populate one of the columns with Oracle sequence call and the others hard coded. I took th...