problem in reading variable number of columns from file

elias.shaik · Post by **elias.shaik** » Tue Sep 29, 2009 3:59 am

Hi all,

Would you please let me know if there is a way to read variable number of columns from a sequential file
.

Sample data:

Code: Select all

"abc","yui",123,"rty"
"ois","ght",45,

ArndW · Post by **ArndW** » Tue Sep 29, 2009 4:05 am

Your example has 4 columns in both rows.

elias.shaik · Post by **elias.shaik** » Tue Sep 29, 2009 4:21 am

sorry typo mistake.Here is the correct sample of records

Code: Select all

"abc","yui",123,"rty","mnb"
"ois","ght",45

ArndW · Post by **ArndW** » Tue Sep 29, 2009 6:49 am

This is tougher to do in PX than in server - I would use a server job, where missing columns at the end of a line can be handled automatically - a functionality not present in PX jobs. The Server job could read & reformat the file and then a PX job could do the real processing.

ray.wurlod · Post by **ray.wurlod** » Tue Sep 29, 2009 4:22 pm

About the only way I can think of (apart from using cascading reject links on Column Import stages) is to read the entire line as a single VarChar field and parse it using Field() functions in a Transformer stage. Missing fields will return "" from the Field() function; you can do with these what ever you wish. You can also count the delimiters (in a stage variable) to check whether the field is truly missing or validly empty.

Sreenivasulu · Post by **Sreenivasulu** » Wed Sep 30, 2009 1:12 am

Use schema files.
With this input column are mapped to their respective database column irrespective of the position.
Hope this helps

Regards
Sreeni

ArndW · Post by **ArndW** » Wed Sep 30, 2009 2:41 am

Except in this case the source is a sequential file.

keshav0307 · Post by **keshav0307** » Wed Sep 30, 2009 6:22 pm

Read as a single column and in a transformer use the FIELD function to split