Trouble viewing source data

ady · Post by **ady** » Wed Dec 27, 2006 9:01 am

I am having trouble viewing the data from a sequential file. Here is the error

##I TFCN 000001 09:49:58(000) <main_program>
DataStage XE Parallel Extender V7.0.0
Copyright (C) 2003, 1997 - 2002 Ascential Software, Inc.
All Rights Reserved

##I TOSH 000002 09:49:58(001) <main_program> orchgeneral: loaded
##I TOSH 000002 09:49:58(002) <main_program> orchsort: loaded
##I TOSH 000002 09:49:58(003) <main_program> orchstats: loaded
##I TFSC 000001 09:49:58(004) <main_program> APT configuration file: /ascential/datastage/70/UND/Ascential/DataStage/Configurations/default.apt
##W TCOS 000049 09:49:59(000) <main_program> Parameter specified but not used in flow: _HashFilePath
##W TCOS 000049 09:49:59(001) <main_program> Parameter specified but not used in flow: _SeqFilePath
##W TCOS 000049 09:49:59(002) <main_program> Parameter specified but not used in flow: _APT_CONFIG_FILE
##W TCOS 000049 09:49:59(003) <main_program> Parameter specified but not used in flow: DSProjectMapName
##W TOIX 000000 09:50:00(000) <Prod,0> Input buffer overrun at field "CPW_UPDATE_DATE"
##W TOIX 000154 09:50:00(001) <Prod,0> Import warning at record 0:
##W TOIX 000018 09:50:00(002) <Prod,0> Import unsuccessful at record 0:
##W TOIX 000000 09:50:00(003) <Prod,0> Input buffer overrun at field "CPW_UPDATE_DATE"
##W TOIX 000154 09:50:00(004) <Prod,0> Import warning at record 1:
##W TOIX 000018 09:50:00(005) <Prod,0> Import unsuccessful at record 1:
##W TOIX 000000 09:50:00(006) <Prod,0> Input buffer overrun at field "CPW_UPDATE_DATE"
##W TOIX 000154 09:50:00(007) <Prod,0> Import warning at record 2:
##W TOIX 000018 09:50:00(008) <Prod,0> Import unsuccessful at record 2:
##W TOIX 000000 09:50:00(009) <Prod,0> Input buffer overrun at field "CPW_UPDATE_DATE"
##W TOIX 000154 09:50:00(010) <Prod,0> Import warning at record 3:
##W TOIX 000018 09:50:00(011) <Prod,0> Import unsuccessful at record 3:
##W TOIX 000000 09:50:00(012) <Prod,0> Input buffer overrun at field "CPW_UPDATE_DATE"
##W TOIX 000154 09:50:00(013) <Prod,0> Import warning at record 4:
##W TOIX 000018 09:50:00(014) <Prod,0> Import unsuccessful at record 4:
##I TOIX 000157 09:50:00(015) <Prod,0> (no further reports will be generated from this partition until successful import)
##I TOIX 000156 09:50:00(016) <Prod,0> Progress: 10 percent
##I TOIX 000156 09:50:00(017) <Prod,0> Progress: 20 percent
##I TOIX 000156 09:50:00(018) <Prod,0> Progress: 30 percent
##I TOIX 000156 09:50:00(019) <Prod,0> Progress: 40 percent
##I TOIX 000156 09:50:00(020) <Prod,0> Progress: 50 percent
##I TOIX 000156 09:50:00(021) <Prod,0> Progress: 60 percent
##I TOIX 000156 09:50:00(022) <Prod,0> Progress: 70 percent
##I TOIX 000156 09:50:01(000) <Prod,0> Progress: 80 percent
##I TOIX 000156 09:50:01(001) <Prod,0> Progress: 90 percent
##I TOIX 000163 09:50:01(002) <Prod,0> Import complete. 0 records imported successfully, 27749 rejected.
**VIEWDATA ROW LIMITER HIT**

I thought there is a datatype problem at field "CPW_UPDATE_DATE". The datatype is CHAR and I tried changing it to VARCHAR and it dosent work.

What could be the problem here ? ....

johnthomas · Post by **johnthomas** » Wed Dec 27, 2006 9:13 am

is the input data for CPW_UPDATE_DATE Date or timestamp ??

ady · Post by **ady** » Wed Dec 27, 2006 10:18 am

The Actual data is in the format "0001-01-01"

Its a date but we have it as char in the source. When we use it as char in server it works fine.

DSguru2B · Post by **DSguru2B** » Wed Dec 27, 2006 10:26 am

What is the length that you have specified for char? Change it to varchar and see what happens.

ady · Post by **ady** » Wed Dec 27, 2006 10:33 am

The Actual data is in the format "0001-01-01"

Its a date but we have it as char in the source. When we use it as char in server it works fine.

DSguru2B · Post by **DSguru2B** » Wed Dec 27, 2006 10:39 am

I read that before. You did not answer my question, what is the length that you specified for char? if its 10 then try using varchar.

ady · Post by **ady** » Wed Dec 27, 2006 11:09 am

Yes, the length is 10. I tried using VARCHAR but it gives out the same error

DSguru2B · Post by **DSguru2B** » Wed Dec 27, 2006 11:13 am

Do an exact search on 'Input buffer overrun at field'. This usually means that the data does not match its input schema.

us1aslam1us · Post by **us1aslam1us** » Wed Dec 27, 2006 12:35 pm

Use some ditor to view the data of that particular field in the input file. This will give you some idea.

Sam

ady · Post by **ady** » Wed Dec 27, 2006 12:52 pm

"|"2002-08-21"|"03"|"2"|"9378643000"|" "|"
0|0|" "|" "|"N"|"2002-08-21"|"SN4150"|" "|" "|"
"|" "|0|0|" "|" "|" "|
"|" "|" "|"2000-11-30"|" "|" "|"2000-11-30"
60535"|"000000"|"C"|"SPEEDWAY DISTRICT #114 "
"|"ENON "|"45323"|"1056"|"OH"|"4D
"|"2004-02-05"|"03"|"4"|"9378643000"|" "|"
0|0|" "|" "|"N"|"2004-02-05"|"SN4042"|" "|" "|"
"|" "|0|0|" "|" "|" "|
"|" "|" "|"2000-11-30"|" "|" "|"2000-11-30"
60536"|"000000"|"C"|"SPEEDWAY DISTRICT #115 "
"|"ENON "|"45323"|"1056"|"OH"|"4D
"|"2004-02-05"|"03"|"4"|"9378643000"|" "|"
0|0|" "|" "|"N"|"2004-02-05"|"SN4042"|" "|" "|"
"|" "|0|0|" "|" "|" "|
"|" "|" "|"2000-11-30"|" "|" "|"2000-11-30"

Thats how the data looks. The highlighted red filed is the one we are talkin bout.

DSguru2B · Post by **DSguru2B** » Wed Dec 27, 2006 12:56 pm

You sure there is no leading or trailing space. Just increase the size from 10 to maybe 12 and then see if it screams.

I_Server_Whale · Post by **I_Server_Whale** » Wed Dec 27, 2006 1:00 pm

beaditya wrote:"|"2002-08-21"|"03"|"2"|"9378643000"|" "|"
0|0|" "|" "|"N"|"2002-08-21"|"SN4150"|" "|" "|"
"|" "|0|0|" "|" "|" "|
"|" "|" "|"2000-11-30"|" "|" "|"2000-11-30"
60535"|"000000"|"C"|"SPEEDWAY DISTRICT #114 "
"|"ENON "|"45323"|"1056"|"OH"|"4D
"|"2004-02-05"|"03"|"4"|"9378643000"|" "|"
0|0|" "|" "|"N"|"2004-02-05"|"SN4042"|" "|" "|"
"|" "|0|0|" "|" "|" "|
"|" "|" "|"2000-11-30"|" "|" "|"2000-11-30"
60536"|"000000"|"C"|"SPEEDWAY DISTRICT #115 "
"|"ENON "|"45323"|"1056"|"OH"|"4D
"|"2004-02-05"|"03"|"4"|"9378643000"|" "|"
0|0|" "|" "|"N"|"2004-02-05"|"SN4042"|" "|" "|"
"|" "|0|0|" "|" "|" "|
"|" "|" "|"2000-11-30"|" "|" "|"2000-11-30"

Thats how the data looks. The highlighted red filed is the one we are talkin bout.

Is this the format that the sequential file exists in?? I mean, I can vaguely figure out the end of the record. It would be nice if you post the file in a well formatted way.

ady · Post by **ady** » Wed Dec 27, 2006 1:03 pm

Actually I have 2 sequential files with the same data but use different delimiters "," and "|" . Both of them give the 'Input buffer overrun at field' error.

Wen I change the datatype of that field to VARCHAR in the file with delimiter "," I am able to view the data. But it dosent work with the file with "|" as delimiter.

Is the delimiter causing this problem ??

DSguru2B · Post by **DSguru2B** » Wed Dec 27, 2006 1:12 pm

I doubt it. As Whale requested. Can we get a well formatted sample data. Enclose it in 'Code' I just created a file with a pipe as delimiter. Its working fine.

DSguru2B · Post by **DSguru2B** » Wed Dec 27, 2006 1:21 pm

Ok the input file looks really suspicious. Your quote character surrounds the delimiter and not the values. I was able to reproduce your error at my end. Its due to the same. Your data is messed up and thats why your getting errors. Build a server job and use a server sequential file stage to read this particular file with quote character set to 000 and delimiter to |. You will see what i am talking about.