Handling New Line within a column while reading

jerome_rajan · Post by **jerome_rajan** » Thu May 03, 2012 10:33 pm

Hi All,

My source is a .csv file in the following format

"col1","col2","col3","col4"
"1","2","3\n2","4"
"2","2","3\n2","4"

As you can see, col3 has a new line withiin it. When I try reading this file using sequential file stage, it isn't able to read the entire line since it finds a new line before the actual end of the record.

Can someone please help me with this issue?

Thanks in advance

ray.wurlod · Post by **ray.wurlod** » Thu May 03, 2012 11:14 pm

Use a server Sequential File stage, either in a server job or in a server Shared Container in your parallel job.

ray.wurlod · Post by **ray.wurlod** » Thu May 03, 2012 11:15 pm

Moderator: please move to Enterprise Edition forum

(moved)

jerome_rajan · Post by **jerome_rajan** » Thu May 03, 2012 11:31 pm

Thank you Ray. But looks like I cannot select the server sequential stage in my parallel job. Either it has been disabled or I am missing something. Your advice would be much appreciated.

SURA · Post by **SURA** » Thu May 03, 2012 11:41 pm

Read carefully.

You are right. You can't use directly in PX, but you need to use the Server pallet and place the sequential file stage and then use it.

jerome_rajan · Post by **jerome_rajan** » Thu May 03, 2012 11:46 pm

My bad! Thank you.

Besides this, are there no other alternatives solely using parallel stages?

ray.wurlod · Post by **ray.wurlod** » Fri May 04, 2012 12:51 am

None of which I'm aware. Unless you want to write your own, of course.

ray.wurlod · Post by **ray.wurlod** » Fri May 04, 2012 12:53 am

jerome_rajan wrote:But looks like I cannot select the server sequential stage in my parallel job. Either it has been disabled or I am missing something. Your advice would be much appreciated.

Read what I wrote again, carefully.

pandeesh · Post by **pandeesh** » Fri May 04, 2012 4:17 am

you can try some other way using filter command in seq file stage ..
such as:

Code: Select all

tr -d "\n"

But make sure that, your \n at the end should not be removed.

Thanks

ray.wurlod · Post by **ray.wurlod** » Sat May 05, 2012 1:33 am

The newlines are part of the client's data. You can't arbitrarily delete them!