Not able to load sequential file in parallel jobs
Posted: Tue Mar 25, 2008 5:48 am
Hi,
I have a .dat file which i need to load into a table. The file is fixed-width with no delimiters for fields. The file has 10 different columns and each of the columns i read them as char(n). The total length of the fixed data is 89 chars.
I am not sure if UNIX newline is the record delimiter? Though i tried with and without unix newline delimiter.
Issues that i face while trying to do View Data:
a. I am able to view the data perfectly well for the same table definition in a server job but not in a parallel job. After quite a bit of tug of war, i was able to view the data with 3 chars less in parallel job.
b. I tried a workaround for reading the record as a single entity and then do a substring. This gives a strange output: After the transformer, i inserted a Peek stage and the peek outputs the values as required ie, 89 chars displayed as per table definition. But when it writes to the output sequential file, in the last column last 2 chars are truncated though as per the definition it should have 9 chars, it displays 7 chars only.
If someone has faced an issue similar to these and have an idea of what could be the workarounds or solutions, please help me out.
Thanks in advance.
Regards,
Ram
I have a .dat file which i need to load into a table. The file is fixed-width with no delimiters for fields. The file has 10 different columns and each of the columns i read them as char(n). The total length of the fixed data is 89 chars.
I am not sure if UNIX newline is the record delimiter? Though i tried with and without unix newline delimiter.
Issues that i face while trying to do View Data:
a. I am able to view the data perfectly well for the same table definition in a server job but not in a parallel job. After quite a bit of tug of war, i was able to view the data with 3 chars less in parallel job.
b. I tried a workaround for reading the record as a single entity and then do a substring. This gives a strange output: After the transformer, i inserted a Peek stage and the peek outputs the values as required ie, 89 chars displayed as per table definition. But when it writes to the output sequential file, in the last column last 2 chars are truncated though as per the definition it should have 9 chars, it displays 7 chars only.
If someone has faced an issue similar to these and have an idea of what could be the workarounds or solutions, please help me out.
Thanks in advance.
Regards,
Ram