Sequential file load problem
Moderators: chulett, rschirm, roy
Sequential file load problem
The error I get is:
##I TFCN 000001 11:47:47(000) <main_program>
Ascential DataStage(tm) Enterprise Edition 7.5.1A
Copyright (c) 2004, 1997-2004 Ascential Software Corporation.
All Rights Reserved
##I TUTL 000031 11:47:47(001) <main_program> The open files limit is 1024; raising to 65536.
##I TOSH 000002 11:47:47(002) <main_program> orchgeneral: loaded
##I TOSH 000002 11:47:47(003) <main_program> orchsort: loaded
##I TOSH 000002 11:47:47(004) <main_program> orchstats: loaded
##I TFSC 000001 11:47:47(005) <main_program> APT configuration file: /opt/Ascential/DataStage/Configurations/default.apt
##W TCOS 000049 11:47:47(006) <main_program> Parameter specified but not used in flow: Prm1
##W TCOS 000049 11:47:47(007) <main_program> Parameter specified but not used in flow: Prm2
##W TCOS 000049 11:47:47(008) <main_program> Parameter specified but not used in flow: Prm3
##W TCOS 000049 11:47:47(009) <main_program> Parameter specified but not used in flow: Prm4
##W TCOS 000049 11:47:47(010) <main_program> Parameter specified but not used in flow: Prm5
##W TCOS 000049 11:47:47(011) <main_program> Parameter specified but not used in flow: Prm6
##W TCOS 000049 11:47:47(012) <main_program> Parameter specified but not used in flow: Prm7
##W TCOS 000049 11:47:47(013) <main_program> Parameter specified but not used in flow: Prm8
##W TCOS 000049 11:47:47(014) <main_program> Parameter specified but not used in flow: _APT_DISABLE_COMBINATION
##W TCOS 000049 11:47:47(015) <main_program> Parameter specified but not used in flow: _DS_ENABLE_RESERVED_CHAR_CONVERT
##W TCOS 000049 11:47:47(016) <main_program> Parameter specified but not used in flow: DSProjectMapName
##W TOIX 000170 11:53:24(000) <IDC_Output,0> Import consumed only 99bytes of the record's 100 bytes (no further warnings will be generated from this partition)
##W TOIX 000154 11:53:24(001) <IDC_Output,0> Import warning at record 0.
##I TOIX 000156 11:47:48(002) <IDC_Output,0> Progress: 10 percent.
##I TOIX 000156 11:47:48(003) <IDC_Output,0> Progress: 20 percent.
##I TOIX 000156 11:47:48(004) <IDC_Output,0> Progress: 30 percent.
##I TOIX 000156 11:47:48(005) <IDC_Output,0> Progress: 40 percent.
##I TOIX 000156 11:47:48(005) <IDC_Output,0> Progress: 50 percent.
**VIEWDATA ROW LIMITER HIT**
I have a file that is coming into Linux from Windows environment. Each line is 100 characters long (using awk). So for a file of 5 lines, number of characters should be 500. However, when I do a vi on it, I see 505 characters, that is, every line has 1 additional character. When I do a vi and go to the last character, it is character 99 so the file has 99 visible characters. However, whenever I get the length of the line from Unix using awk, it has 100 characters.
I have tried the following:
1)I set the Field Width and Field Max Width to length of the column on the Edit Column Metadata window.
2)Added a one character dummy column at the end and set "Drop on Input"
3)Went through every post in this forum. None of the problems are similar to mine.
4)Here is the interesting part. When I take the first 102 lines of the file, it loads fine. When I take first 103, it gives the error. When I do a tail on the last 5 lines of this 103 line file, it loads fine again! Exact same situation
on the tail side.
Any help would be appreciated.
##I TFCN 000001 11:47:47(000) <main_program>
Ascential DataStage(tm) Enterprise Edition 7.5.1A
Copyright (c) 2004, 1997-2004 Ascential Software Corporation.
All Rights Reserved
##I TUTL 000031 11:47:47(001) <main_program> The open files limit is 1024; raising to 65536.
##I TOSH 000002 11:47:47(002) <main_program> orchgeneral: loaded
##I TOSH 000002 11:47:47(003) <main_program> orchsort: loaded
##I TOSH 000002 11:47:47(004) <main_program> orchstats: loaded
##I TFSC 000001 11:47:47(005) <main_program> APT configuration file: /opt/Ascential/DataStage/Configurations/default.apt
##W TCOS 000049 11:47:47(006) <main_program> Parameter specified but not used in flow: Prm1
##W TCOS 000049 11:47:47(007) <main_program> Parameter specified but not used in flow: Prm2
##W TCOS 000049 11:47:47(008) <main_program> Parameter specified but not used in flow: Prm3
##W TCOS 000049 11:47:47(009) <main_program> Parameter specified but not used in flow: Prm4
##W TCOS 000049 11:47:47(010) <main_program> Parameter specified but not used in flow: Prm5
##W TCOS 000049 11:47:47(011) <main_program> Parameter specified but not used in flow: Prm6
##W TCOS 000049 11:47:47(012) <main_program> Parameter specified but not used in flow: Prm7
##W TCOS 000049 11:47:47(013) <main_program> Parameter specified but not used in flow: Prm8
##W TCOS 000049 11:47:47(014) <main_program> Parameter specified but not used in flow: _APT_DISABLE_COMBINATION
##W TCOS 000049 11:47:47(015) <main_program> Parameter specified but not used in flow: _DS_ENABLE_RESERVED_CHAR_CONVERT
##W TCOS 000049 11:47:47(016) <main_program> Parameter specified but not used in flow: DSProjectMapName
##W TOIX 000170 11:53:24(000) <IDC_Output,0> Import consumed only 99bytes of the record's 100 bytes (no further warnings will be generated from this partition)
##W TOIX 000154 11:53:24(001) <IDC_Output,0> Import warning at record 0.
##I TOIX 000156 11:47:48(002) <IDC_Output,0> Progress: 10 percent.
##I TOIX 000156 11:47:48(003) <IDC_Output,0> Progress: 20 percent.
##I TOIX 000156 11:47:48(004) <IDC_Output,0> Progress: 30 percent.
##I TOIX 000156 11:47:48(005) <IDC_Output,0> Progress: 40 percent.
##I TOIX 000156 11:47:48(005) <IDC_Output,0> Progress: 50 percent.
**VIEWDATA ROW LIMITER HIT**
I have a file that is coming into Linux from Windows environment. Each line is 100 characters long (using awk). So for a file of 5 lines, number of characters should be 500. However, when I do a vi on it, I see 505 characters, that is, every line has 1 additional character. When I do a vi and go to the last character, it is character 99 so the file has 99 visible characters. However, whenever I get the length of the line from Unix using awk, it has 100 characters.
I have tried the following:
1)I set the Field Width and Field Max Width to length of the column on the Edit Column Metadata window.
2)Added a one character dummy column at the end and set "Drop on Input"
3)Went through every post in this forum. None of the problems are similar to mine.
4)Here is the interesting part. When I take the first 102 lines of the file, it loads fine. When I take first 103, it gives the error. When I do a tail on the last 5 lines of this 103 line file, it loads fine again! Exact same situation
on the tail side.
Any help would be appreciated.
Even if you are joining the sequential file with other datasets, your problems arises in your source sequential file. As you have mentioned that the file is coming from Windows environment, I believe the error has something to do with how you are placing the file in UNIX. Are you transferring in the binary mode? Or can you try giving the line terminator as DOS line terminator and check if you are able to read the data?
Kris
Where's the "Any" key?-Homer Simpson
Where's the "Any" key?-Homer Simpson
Okay. If that doesn't seem to work we need to take a different approach. Can you provide the details as to what you have set everything on format tab of the sequential file? Also, if the file is a fixed width file, did you mention the datatype as CHAR for all the fields?
Kris
Where's the "Any" key?-Homer Simpson
Where's the "Any" key?-Homer Simpson
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact: