Joining 2 Files having different File formats

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
abhilasha.patil
Participant
Posts: 5
Joined: Tue Sep 18, 2007 4:04 am

Joining 2 Files having different File formats

Post by abhilasha.patil »

Hi,

I am joining data from 2 sequential files. I want to perform inner join on both the files.
1st input file is a comma delimited file and 2nd input file is a fixed width file.
Both the file has same metadata.

I want to sort the data of both the files and then I want to join them.

Since file formats of both the files is different, I cant join them.
Please suggest me some solution for joining the data of both the files.

I tried using sequential file stage, to first trim the data from both the files, and then join both of them, but I am not getting correct results even after this.
Also using 2 transformer stages is reducing the performance of my job.

Please reply quickly.

Thanks,
Abhilasha.
JoshGeorge
Participant
Posts: 612
Joined: Thu May 03, 2007 4:59 am
Location: Melbourne

Post by JoshGeorge »

Set the APT_PAD_CHARACTER to space, then trim on both links for key fields to join and try.
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
abhilasha.patil
Participant
Posts: 5
Joined: Tue Sep 18, 2007 4:04 am

Post by abhilasha.patil »

Am i suppose to use transformer stage for triming the data from both the inputs? If yes, Transformer stage is making my job more slow.

Can you please suggest me some other alternative, rather than using Transformer stage?

Also let me know where can I find this parameter: APT_PAD_CHARACTER and where i need to set it to space.

Thanks,
Abhilasha
JoshGeorge
Participant
Posts: 612
Joined: Thu May 03, 2007 4:59 am
Location: Melbourne

Post by JoshGeorge »

Use modify stage.
There is a function - 'string_trim' (Ref. Orchestrate Operators guide 7.5 version) which can be used in modify stage to trim string. But you cannot trim both leading and trailing spaces using this function.

This is the syntax :

string_trim[character,direction,justify](string)

This function can be used to trim occurrences of a character from the beginning or end of a string.
Ex:
Code:

string_trim[" "](string) --> Will trim all trailing spaces
string_trim[" ",begin](string) --> Will trim all leading spaces


Add APT_PAD_STRING in your job parameter and change it to 0x20 (Space).
Joshy George
<a href="http://www.linkedin.com/in/joshygeorge1" ><img src="http://www.linkedin.com/img/webpromo/bt ... _80x15.gif" width="80" height="15" border="0"></a>
abhilasha.patil
Participant
Posts: 5
Joined: Tue Sep 18, 2007 4:04 am

Post by abhilasha.patil »

Thanks....Your solution was helpful. I could solve my problem. :)
Mhasan
Participant
Posts: 38
Joined: Wed Apr 19, 2006 10:03 pm
Location: Bangalore

Post by Mhasan »

Can anybody tell where i can find this "Orchestrate Operators guide 7.5 version" Document
Thanks
M A hasan
Post Reply