Fixed width Seq File from Server to Parallel
Moderators: chulett, rschirm, roy
Fixed width Seq File from Server to Parallel
I am using a seq file created in server with "fixed width". Now I need to use the file in parallel but in parallel when I select fixed width I need to specify the width value.
Where can I see the fixed width value which was set when the file is created?? I have the job which created the original file.
Where can I see the fixed width value which was set when the file is created?? I have the job which created the original file.
I had "smallInt" and "decimal" datatypes in the table definition, I have changed them to "CHAR" now but i get this error.
##I TOSH 000002 14:29:02(001) <main_program> orchgeneral: loaded
##I TOSH 000002 14:29:02(002) <main_program> orchsort: loaded
##I TOSH 000002 14:29:02(003) <main_program> orchstats: loaded
##I TFSC 000001 14:29:02(004) <main_program> APT configuration file: /ascential/datastage/70/UND/Ascential/DataStage/Configurations/default.apt
##W TCOS 000049 14:29:03(000) <main_program> Parameter specified but not used in flow: _SeqFilePath
##W TCOS 000049 14:29:03(001) <main_program> Parameter specified but not used in flow: DSProjectMapName
>##E TOIX 000158 14:29:04(000) <Dev,0> Error reading on import:
>##E TOIX 000050 14:29:04(001) <Dev,0> Bad record delimiter after fixed-length record: expected "\n", got "0"
>##E TOIX 000179 14:29:04(002) <Dev,0> Import error at record 0.
>##E TOIX 000089 14:29:04(003) <Dev,0> Operator's runLocally() failed
##I TOIX 000094 14:29:04(004) <Dev,0> Output 0 produced 0 records
>##E TFCI 000089 14:29:04(005) <APT_CombinedOperatorController,0> Operator's runLocally() failed
>##E TFPM 000040 14:29:04(006) <APT_CombinedOperatorController,0> Operator terminated abnormally: runLocally did not return APT_StatusOk
>##E TFSR 000011 14:29:10(000) <main_program> Step execution finished with status = FAILED.
##I TCOS 000026 14:29:10(001) <main_program> Startup Time 0:06 Production Run Time 0:01
BAD RECORD DELIMITER ?
##I TOSH 000002 14:29:02(001) <main_program> orchgeneral: loaded
##I TOSH 000002 14:29:02(002) <main_program> orchsort: loaded
##I TOSH 000002 14:29:02(003) <main_program> orchstats: loaded
##I TFSC 000001 14:29:02(004) <main_program> APT configuration file: /ascential/datastage/70/UND/Ascential/DataStage/Configurations/default.apt
##W TCOS 000049 14:29:03(000) <main_program> Parameter specified but not used in flow: _SeqFilePath
##W TCOS 000049 14:29:03(001) <main_program> Parameter specified but not used in flow: DSProjectMapName
>##E TOIX 000158 14:29:04(000) <Dev,0> Error reading on import:
>##E TOIX 000050 14:29:04(001) <Dev,0> Bad record delimiter after fixed-length record: expected "\n", got "0"
>##E TOIX 000179 14:29:04(002) <Dev,0> Import error at record 0.
>##E TOIX 000089 14:29:04(003) <Dev,0> Operator's runLocally() failed
##I TOIX 000094 14:29:04(004) <Dev,0> Output 0 produced 0 records
>##E TFCI 000089 14:29:04(005) <APT_CombinedOperatorController,0> Operator's runLocally() failed
>##E TFPM 000040 14:29:04(006) <APT_CombinedOperatorController,0> Operator terminated abnormally: runLocally did not return APT_StatusOk
>##E TFSR 000011 14:29:10(000) <main_program> Step execution finished with status = FAILED.
##I TCOS 000026 14:29:10(001) <main_program> Startup Time 0:06 Production Run Time 0:01
BAD RECORD DELIMITER ?
All you need to specify is
Under format tab
- Record Level --> Record length = fixed
Record Delimiter = UNIX newline
- Field defaults --> Delimiter = none.
Then go to the Columns tab and specify all the columns as 'char' type and specify the correct length. That is it.
Under format tab
- Record Level --> Record length = fixed
Record Delimiter = UNIX newline
- Field defaults --> Delimiter = none.
Then go to the Columns tab and specify all the columns as 'char' type and specify the correct length. That is it.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
SAMPLE DATA:
Metadata:
<a href="http://server6.theimagehosting.com/imag ... 02.1cd.jpg" target="_blank"><img src="http://images6.theimagehosting.com/clip ... 1cd.th.jpg" border="0" title="The Image Hosting" /></a>
Click on the image when it open in a new window
Code: Select all
0501025414638979P+4 2005-02-01 0 -126.75 507.00 0.25-
0603057365607336P 4 2006-07-01 0 -124.43 1244.33 0.10-
0501025673495227P+4 2005-03-01 0 -282.25 1129.00 0.25-
0501025673495227R+2 2005-01-01 0 -395.15 1129.00 0.35-
0501025673138242P+4 2005-03-01 0 -7.25 29.00 0.25-
0501025674501638R+2 2005-01-01 0 -120.75 345.00 0.35-
0501025674501636R+2 2005-01-01 0 -92.05 263.00 0.35-
0501025674501637R+2 2005-01-01 0 -108.03 308.67 0.35-
0603060356383213P+4 2006-07-01 0 -80.08 320.33 0.25-
0603060356405056P+4 2006-07-01 0 -104.75 419.00 0.25-
0603060356400574P+4 2006-07-01 0 -60.17 240.67 0.25-
0603060356493300P+4 2006-07-01 0 -61.08 244.33 0.25-
0603060356535813P+4 2006-07-01 0 -63.75 255.00 0.25-
0603060356535813P+4 2006-08-01 0 -63.75 255.00 0.25-
0603060356535813P+4 2006-09-01 0 -63.75 255.00 0.25-
0603060356405056P+4 2006-08-01 0 -104.75 419.00 0.25-
0603060356405056P+4 2006-09-01 0 -104.75 419.00 0.25-
0603060361146877P 4 2006-07-01 0 -34.80 348.00 0.10-
0603060342792445P+4 2006-07-01 0 -19.83 79.33 0.25-
0603060342694837P+4 2006-09-01 0 -0.58 2.33 0.25-
0603060342797881P+4 2006-09-01 0 -1.75 7.00 0.25-
Metadata:
<a href="http://server6.theimagehosting.com/imag ... 02.1cd.jpg" target="_blank"><img src="http://images6.theimagehosting.com/clip ... 1cd.th.jpg" border="0" title="The Image Hosting" /></a>
Click on the image when it open in a new window
Last edited by ady on Thu Dec 28, 2006 2:12 pm, edited 1 time in total.
The first record of your sample data is of length 54 (including all spaces). If you add up the length's in your meta data, its 61. Your meta data is wrong, or your file is wrong. At a glance, a person can tell that, for a fixed width file, its not formatted correct. Go back to your post and enclose the data with
Code: Select all
tags so that it maintains the correct format. Then go to your data and copy past it again, as it is.
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
-
- Charter Member
- Posts: 822
- Joined: Sat Sep 17, 2005 5:25 pm
- Location: USA
-
- Charter Member
- Posts: 822
- Joined: Sat Sep 17, 2005 5:25 pm
- Location: USA
Your schema is messed up. Analyze it with me, if you will.
Lets take your first record and break it up to your schema. The length of the column is in brackets.
As usaslam said, you either have spaces inside that you dont need, or your missing a column or two in the schema.
Lets take your first record and break it up to your schema. The length of the column is in brackets.
Code: Select all
0501025414 (10)
638979 (6)
P+ (2)
4 (1)
(1)
2005-02-01 (10)
(1)
(4)
0 -12 (11)
6.75 (11)
507 (3)
. (1)
00 0.25- (This is the extra part)
Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep.
-
- Charter Member
- Posts: 822
- Joined: Sat Sep 17, 2005 5:25 pm
- Location: USA
Code: Select all
0501025414(10)
638979(6)
P+(2)
4(1)
(1)
2005-02-01 (10)
Till this part it is fine. After this column you are having a space between each field.
space
(1)
space
0 (4)
space
-126.75 (11)
space
507.00(11)
space
0.25(3)
-(1)
Sam