Fixed width Seq File from Server to Parallel

DSguru2B · Post by **DSguru2B** » Thu Dec 28, 2006 2:40 pm

us1aslam1us, some spaces are fine. They just mean that there is no value. Spaces within fields are ok, but the OP has to validate that and make sure that these spaces are the 'OK' spaces.

us1aslam1us · Post by **us1aslam1us** » Thu Dec 28, 2006 2:46 pm

beaditya wrote: I think the error is due to "Spaces between columns" value, its set to "0" in the server job when the file was created.
I tried giving "0" as the "Fill Char" value in parallen seq stage, it dosent work

Just curious to know,How you had done this in server job?

Sam

ady · Post by **ady** » Thu Dec 28, 2006 3:07 pm

Whats the best thing to do now ?

ady · Post by **ady** » Thu Dec 28, 2006 3:13 pm

us1aslam1us wrote:
beaditya wrote: I think the error is due to "Spaces between columns" value, its set to "0" in the server job when the file was created.
I tried giving "0" as the "Fill Char" value in parallen seq stage, it dosent work
Just curious to know,How you had done this in server job?

Sam

Which one are you talkin about?...... The "Spaces between columns" ??

us1aslam1us · Post by **us1aslam1us** » Thu Dec 28, 2006 3:13 pm

1. Analyze the server job which is creating this file and get the proper format of the file.
2. Using the Hex editor verify what exactly those space mean there and by someway need to remove those spaces from the file by considering them to be 'OK' space.

Sam

ady · Post by **ady** » Thu Dec 28, 2006 3:31 pm

Can I just create a server job to write the fixed width data into another normal seq file and then get it into parallel? Will that work ?

I never used HEX, so thats gonna be a problem

us1aslam1us · Post by **us1aslam1us** » Thu Dec 28, 2006 3:42 pm

Instead read the whole record in one column as varchar (6*)and break it using substring in the transformer depending on the required format.

Sam

DSguru2B · Post by **DSguru2B** » Thu Dec 28, 2006 3:47 pm

beaditya wrote:Can I just create a server job to write the fixed width data into another normal seq file and then get it into parallel? Will that work ?

I never used HEX, so thats gonna be a problem

Good idea. Sounds good.

ady · Post by **ady** » Thu Dec 28, 2006 3:52 pm

Actually I am testing some jobs rightnow, I have 1 file from the PROD and 1 from the DEV ENV and I compare these file record by record using a DIFFERENCE stage in parallel. Thats why I need these files in parallel.

I need to do this for most of the jobs which have different metadata.... so I dont want to spend much time editing the metadata and substringing every column.

Instead can I create a new seq file in server from the old "Fixed width" file and use it ? That should work right?

..... Performance is not an issue for us as of now because these job as just for checking data and will be discarded after few runs.

us1aslam1us · Post by **us1aslam1us** » Thu Dec 28, 2006 3:55 pm

That will work fine.

Sam