Fixed width Seq File from Server to Parallel

ady · Post by **ady** » Thu Dec 28, 2006 12:57 pm

I am using a seq file created in server with "fixed width". Now I need to use the file in parallel but in parallel when I select fixed width I need to specify the width value.

Where can I see the fixed width value which was set when the file is created?? I have the job which created the original file.

DSguru2B · Post by **DSguru2B** » Thu Dec 28, 2006 1:01 pm

The length of each and every column, specified in the length section of the columns tab in sequential file properties.

ady · Post by **ady** » Thu Dec 28, 2006 1:05 pm

Where do I specify each and every column length in the parallel seq file stage?

ady · Post by **ady** » Thu Dec 28, 2006 1:07 pm

In parallel seq file I can specify only 1 value under Type defaults>General>Field width. Is theer any other option to do that ?

ady · Post by **ady** » Thu Dec 28, 2006 1:30 pm

I had "smallInt" and "decimal" datatypes in the table definition, I have changed them to "CHAR" now but i get this error.

##I TOSH 000002 14:29:02(001) <main_program> orchgeneral: loaded
##I TOSH 000002 14:29:02(002) <main_program> orchsort: loaded
##I TOSH 000002 14:29:02(003) <main_program> orchstats: loaded
##I TFSC 000001 14:29:02(004) <main_program> APT configuration file: /ascential/datastage/70/UND/Ascential/DataStage/Configurations/default.apt
##W TCOS 000049 14:29:03(000) <main_program> Parameter specified but not used in flow: _SeqFilePath
##W TCOS 000049 14:29:03(001) <main_program> Parameter specified but not used in flow: DSProjectMapName
>##E TOIX 000158 14:29:04(000) <Dev,0> Error reading on import:
>##E TOIX 000050 14:29:04(001) <Dev,0> Bad record delimiter after fixed-length record: expected "\n", got "0"
>##E TOIX 000179 14:29:04(002) <Dev,0> Import error at record 0.
>##E TOIX 000089 14:29:04(003) <Dev,0> Operator's runLocally() failed
##I TOIX 000094 14:29:04(004) <Dev,0> Output 0 produced 0 records
>##E TFCI 000089 14:29:04(005) <APT_CombinedOperatorController,0> Operator's runLocally() failed
>##E TFPM 000040 14:29:04(006) <APT_CombinedOperatorController,0> Operator terminated abnormally: runLocally did not return APT_StatusOk
>##E TFSR 000011 14:29:10(000) <main_program> Step execution finished with status = FAILED.
##I TCOS 000026 14:29:10(001) <main_program> Startup Time 0:06 Production Run Time 0:01

BAD RECORD DELIMITER ?

DSguru2B · Post by **DSguru2B** » Thu Dec 28, 2006 1:30 pm

All you need to specify is
Under format tab
- Record Level --> Record length = fixed
Record Delimiter = UNIX newline

- Field defaults --> Delimiter = none.

Then go to the Columns tab and specify all the columns as 'char' type and specify the correct length. That is it.

ady · Post by **ady** » Thu Dec 28, 2006 1:39 pm

I got the above error when I did everything you just said.

I think the error is due to "Spaces between columns" value, its set to "0" in the server job when the file was created.

I tried giving "0" as the "Fill Char" value in parallen seq stage, it dosent work

DSguru2B · Post by **DSguru2B** » Thu Dec 28, 2006 1:42 pm

Can i see a sample record, maybe a few records and the meta data that you defined. I suspect your lengths are not correct.

ady · Post by **ady** » Thu Dec 28, 2006 1:52 pm

SAMPLE DATA:

Code: Select all

0501025414638979P+4 2005-02-01      0      -126.75       507.00 0.25-
0603057365607336P 4 2006-07-01      0      -124.43      1244.33 0.10-
0501025673495227P+4 2005-03-01      0      -282.25      1129.00 0.25-
0501025673495227R+2 2005-01-01      0      -395.15      1129.00 0.35-
0501025673138242P+4 2005-03-01      0        -7.25        29.00 0.25-
0501025674501638R+2 2005-01-01      0      -120.75       345.00 0.35-
0501025674501636R+2 2005-01-01      0       -92.05       263.00 0.35-
0501025674501637R+2 2005-01-01      0      -108.03       308.67 0.35-
0603060356383213P+4 2006-07-01      0       -80.08       320.33 0.25-
0603060356405056P+4 2006-07-01      0      -104.75       419.00 0.25-
0603060356400574P+4 2006-07-01      0       -60.17       240.67 0.25-
0603060356493300P+4 2006-07-01      0       -61.08       244.33 0.25-
0603060356535813P+4 2006-07-01      0       -63.75       255.00 0.25-
0603060356535813P+4 2006-08-01      0       -63.75       255.00 0.25-
0603060356535813P+4 2006-09-01      0       -63.75       255.00 0.25-
0603060356405056P+4 2006-08-01      0      -104.75       419.00 0.25-
0603060356405056P+4 2006-09-01      0      -104.75       419.00 0.25-
0603060361146877P 4 2006-07-01      0       -34.80       348.00 0.10-
0603060342792445P+4 2006-07-01      0       -19.83        79.33 0.25-
0603060342694837P+4 2006-09-01      0        -0.58         2.33 0.25-
0603060342797881P+4 2006-09-01      0        -1.75         7.00 0.25-

Metadata:

<a href="http://server6.theimagehosting.com/imag ... 02.1cd.jpg" target="_blank"><img src="http://images6.theimagehosting.com/clip ... 1cd.th.jpg" border="0" title="The Image Hosting" /></a>

Click on the image when it open in a new window

DSguru2B · Post by **DSguru2B** » Thu Dec 28, 2006 1:58 pm

The first record of your sample data is of length 54 (including all spaces). If you add up the length's in your meta data, its 61. Your meta data is wrong, or your file is wrong. At a glance, a person can tell that, for a fixed width file, its not formatted correct. Go back to your post and enclose the data with

Code: Select all

 tags so that it maintains the correct format. Then go to your data and copy past it again, as it is.

us1aslam1us · Post by **us1aslam1us** » Thu Dec 28, 2006 2:02 pm

Your MetaData doesn't seem to be matching the data. Could you post the sample date in codes and also change the SQL type to that of original file.

Sam

ady · Post by **ady** » Thu Dec 28, 2006 2:14 pm

I made the change

us1aslam1us · Post by **us1aslam1us** » Thu Dec 28, 2006 2:22 pm

There seems to be spaces between the fields.

Sam

DSguru2B · Post by **DSguru2B** » Thu Dec 28, 2006 2:29 pm

Your schema is messed up. Analyze it with me, if you will.
Lets take your first record and break it up to your schema. The length of the column is in brackets.

Code: Select all

0501025414 (10)
638979 (6)
P+ (2)
4 (1)
  (1)
2005-02-01 (10)
  (1)
     (4)
 0      -12 (11)
6.75        (11)
507 (3)
. (1)
00 0.25- (This is the extra part)

As usaslam said, you either have spaces inside that you dont need, or your missing a column or two in the schema.

us1aslam1us · Post by **us1aslam1us** » Thu Dec 28, 2006 2:38 pm

Code: Select all

0501025414(10) 
638979(6) 
P+(2) 
4(1) 
 (1) 
2005-02-01 (10) 

Till this part it is fine. After this column you are having a space between each field.

space
 (1) 
space
0   (4)
space 
     -126.75 (11)
space
       507.00(11)
space
0.25(3) 
-(1)

Check the schema again.

Sam