reading a delimited file

ady · Post by **ady** » Fri Mar 09, 2007 8:16 am

Hi ,

I Have a seq file seperated by the column delimiter "|"

abc|123|xyz|000|

When I read the file in parallel, i am missing data in 2 of the columns,

like

abc|3|z|000|

Why does it show like that , I have the metadata for all the columns ? .... i am just missing data in 2 columns.

Can anyone help me with this please...

ray.wurlod · Post by **ray.wurlod** » Fri Mar 09, 2007 8:19 am

Do you have the Field Width property set on these truncated columns? What data type have you assigned to them?

DSguru2B · Post by **DSguru2B** » Fri Mar 09, 2007 8:24 am

You sure they are missing or you dont have the 'Display' equal or more than the Length

kumar_s · Post by **kumar_s** » Fri Mar 09, 2007 8:24 am

When I read the file in parallel

Are you referring this to multiple readers per node? If so its not recommended for variable length delimited file. Its for fixed width files.

ady · Post by **ady** » Fri Mar 09, 2007 8:45 am

The text file is been generated from a sql script .... The type is varchar from the source. It not a fixed width file.

but when I change the datatype to decimal .... I am able to view the data.

The "display" property is not present in the metadata, Cant figure out where I can add that

kumar_s · Post by **kumar_s** » Fri Mar 09, 2007 8:49 am

Display is for Server jobs to control the length of the field while suing view data. As I asked you earliar, could you let me know whether its Multiple reader per node.

kumar_s · Post by **kumar_s** » Fri Mar 09, 2007 8:51 am

Or probably the Display option mentioned was in the column select option available in PX.

DSguru2B · Post by **DSguru2B** » Fri Mar 09, 2007 8:52 am

Never mind the display property if you are able to view the complete data when you change datatypes. Display is next to the length in the column's tab. As the name suggest, it limits how much data you can view. But if you drag the column end, you can extend it.
But as I said, that does'nt seem to be your issue.

ady · Post by **ady** » Fri Mar 09, 2007 9:02 am

@Kumar_s

I have no idea about "Multiple reader per node" , how can i find out ?

kumar_s · Post by **kumar_s** » Fri Mar 09, 2007 9:07 am

May I know, what you mean by "When I read the file in parallel"?
And let know that, where are you looking for all these data?
Through view data or in server?

ady · Post by **ady** » Fri Mar 09, 2007 10:01 am

View data , ....

kumar_s · Post by **kumar_s** » Fri Mar 09, 2007 10:05 am

Try to write into a file and check it. "Multiple reader per node" is an opition available in PX sequential file to read a single file parallely to improve the performance.

ray.wurlod · Post by **ray.wurlod** » Fri Mar 09, 2007 10:33 am

kumar_s wrote:
When I read the file in parallel
Are you referring this to multiple readers per node? If so its not recommended for variable length delimited file. Its for fixed width files.

I think that all that was meant was "in a parallel job".

kumar_s · Post by **kumar_s** » Fri Mar 09, 2007 10:36 am

ray.wurlod wrote:I think that all that was meant was "in a parallel job".

Oh... simply may be that.

ady · Post by **ady** » Fri Mar 09, 2007 2:14 pm

Yes it is a parallel job , .....

I am going to pass the values as decimal and then later in the job convert to char, ....