Unable to view all records from sequential file

mac4rfree85 · Post by **mac4rfree85** » Mon Dec 05, 2011 3:18 am

Hi guys,

I have a sequential file. The file has 2500 rows but when i processed the file using datastage using sequential file, the job is processing only 253 rows.

Even viewdata is showing only 253 rows. No error is occuring.

Can somebody help me to process the whole file.

Cheers!!!

sanjayS · Post by **sanjayS** » Mon Dec 05, 2011 5:29 am

can you able to see all records in Source sequential file using View Data?

Thanks,
sanjay.

TPons · Post by **TPons** » Mon Dec 05, 2011 7:13 am

There is option of limiting the number of rows to be processed at run time.
if it has been set, change it to no limit option.

Pons

qt_ky · Post by **qt_ky** » Mon Dec 05, 2011 7:39 am

Check the detailed job log to see if the other rows are getting rejected.

mac4rfree85 · Post by **mac4rfree85** » Mon Dec 05, 2011 7:42 am

Yeah qt_ky. the Other rows are getting rejected saying delim=end is not available. But all the rows have same delimiter then how are those 253 rows are able to process.

chulett · Post by **chulett** » Mon Dec 05, 2011 8:32 am

Your metadata doesn't match your file's data, nothing more than that. What settings are you using in the stage? What is different between a row it will process and one it rejects? Can you post an example of each?

mac4rfree85 · Post by **mac4rfree85** » Tue Dec 06, 2011 5:06 am

chulett wrote:Your metadata doesn't match your file's data, nothing more than that. What settings are you using in the stage? What is different between a row it will process and one it rejects? Can you post an example of each?

Following two records got loaded.

Code: Select all

2011-05-03Aveo                                              RETAIL US        12CHEVROLET CARS            ENTRY CAR
 2011-05-03Aveo                                              FLEET  US       250CHEVROLET CARS            ENTRY CAR

Following records are not loaded to the table.

Code: Select all

2011-05-03DTS                                               RETAIL US         0CADILLAC CARS             LUXURY CAR
 2011-05-03DTS                                               FLEET  US        59CADILLAC CARS             LUXURY CAR

kamtammystuff · Post by **kamtammystuff** » Tue Dec 06, 2011 5:53 am

What is the metadata in the sequential file stage?

zulfi123786 · Post by **zulfi123786** » Tue Dec 06, 2011 5:55 am

perform cat -vte <filename> and filter out two loaded records and two records that are not loaded from the above command and list them.

Bascially the stage is unable to find the record delimiter.

ray.wurlod · Post by **ray.wurlod** » Tue Dec 06, 2011 2:58 pm

Maybe DataStage developers don't like Cadillacs?

Seriously, though, does the 253rd record or thereabouts contain an "end of file" character (Ctrl-D)?

mac4rfree85 · Post by **mac4rfree85** » Tue Dec 06, 2011 11:46 pm

This is what i got

Code: Select all

dsadm@catom-gmapdvl38:/appl/data/Input/US> cat -vte TESTING.txt
 2011-05-03DTS                                               RETAIL US         0CADILLAC CARS             LUXURY CAR$
 2011-05-03DTS                                               FLEET  US        59CADILLAC CARS             LUXURY CAR$
 2011-05-03HHR                                               RETAIL US         7CHEVROLET TRUCKS          VAN/MPV$
 2011-05-03HHR                                               FLEET  US       246CHEVROLET TRUCKS          VAN/MPV$
 2011-05-03STS                                               RETAIL US         2CADILLAC CARS             LUXURY CAR$
 2011-05-03Aveo                                              RETAIL US        12CHEVROLET CARS            ENTRY CAR$
 2011-05-03Aveo                                              FLEET  US       250CHEVROLET CARS            ENTRY CAR$

ray.wurlod · Post by **ray.wurlod** » Tue Dec 06, 2011 11:48 pm

Was this file sourced from a Windows system?

mac4rfree85 · Post by **mac4rfree85** » Tue Dec 06, 2011 11:50 pm

Yes ray, i took the file from Windows and ftpped to Unix box, since my datastage is installed in Unix... Also just to answer your question, it is not processing first 253 rows, but random 253 rows from the file.

ray.wurlod · Post by **ray.wurlod** » Tue Dec 06, 2011 11:56 pm

Could it be that there's a Ctrl-M at the end of the lines (shows as ^M in vi) that is interfering with the parsing? That is, in what mode did you perform the FTP and does your FTP tool handle translating line terminators?

mac4rfree85 · Post by **mac4rfree85** » Wed Dec 07, 2011 12:10 am

That was the first thing i checked. i did not see any Ctrl-M characters..