Page 1 of 1
transformer
Posted: Wed Apr 14, 2010 1:26 pm
by kittu.raja
Hi Everyone,
My design is
FF -------- XFM-------------FF
In my transformer i am deriving some columns and in one of the column I need the count of the source file.Without using aggragater can i achieve my goal inside the transformer.
Can anybosy help me out in this.
Thanks,
Posted: Wed Apr 14, 2010 1:40 pm
by anbu
Why cant you use aggregator?
Posted: Wed Apr 14, 2010 1:55 pm
by chowdhury99
Use Execute Command stage in Sequence and use wc -l filename command to get the total count.
Thanks
Re: transformer
Posted: Wed Apr 14, 2010 4:48 pm
by vivekgadwal
kittu.raja wrote:Hi Everyone,
My design is
FF -------- XFM-------------FF
In my transformer i am deriving some columns and in one of the column I need the count of the source file.Without using aggragater can i achieve my goal inside the transformer.
Can anybosy help me out in this.
Thanks,
Get another Sequential file stage which has two columns defined in it. Let the datatype of one of the fields be Integer/Varchar and the length depends on how big your row count is and the other is Character.
Code: Select all
DmyCol1 - Char(1)
RowCount - Integer or Character.
In the job's before job sub-routine, execute the following command:
Code: Select all
echo "X|`wc -l {file name for which counts are required}`" > {another file to store this result}
Read this file from the second Seq. file stage. In your XFM, generate a dummy column with the value 'X' in it and after this, join it with the second file's data. You job design would look like this:
Code: Select all
Sequential File 1 ----------- XFM ----------- LKUP/JNR ----------- Sequential File output
|
Sequential File 2 ---------------------------------
Hope this helps...
Re: transformer
Posted: Thu Apr 15, 2010 12:15 am
by meet_deb85
Hi
Exact command is
Code: Select all
echo "X|`ls -l {file name for which counts are required}|wc -l|bc`"
vivekgadwal wrote:kittu.raja wrote:Hi Everyone,
My design is
FF -------- XFM-------------FF
In my transformer i am deriving some columns and in one of the column I need the count of the source file.Without using aggragater can i achieve my goal inside the transformer.
Can anybosy help me out in this.
Thanks,
Get another Sequential file stage which has two columns defined in it. Let the datatype of one of the fields be Integer/Varchar and the length depends on how big your row count is and the other is Character.
Code: Select all
DmyCol1 - Char(1)
RowCount - Integer or Character.
In the job's before job sub-routine, execute the following command:
Code: Select all
echo "X|`wc -l {file name for which counts are required}`" > {another file to store this result}
Read this file from the second Seq. file stage. In your XFM, generate a dummy column with the value 'X' in it and after this, join it with the second file's data. You job design would look like this:
Code: Select all
Sequential File 1 ----------- XFM ----------- LKUP/JNR ----------- Sequential File output
|
Sequential File 2 ---------------------------------
Hope this helps...
Re: transformer
Posted: Thu Apr 15, 2010 5:41 am
by chulett
meet_deb85 wrote:Exact command is
Code: Select all
echo "X|`ls -l {file name for which counts are required}|wc -l|bc`"
What? There's absolutely no need to "list" anything and I have no idea what role you think the "binary calculator" is playing here.
Re: transformer
Posted: Thu Apr 15, 2010 7:51 am
by vivekgadwal
chulett wrote:meet_deb85 wrote:Exact command is
Code: Select all
echo "X|`ls -l {file name for which counts are required}|wc -l|bc`"
What? There's absolutely no need to "list" anything and I have no idea what role you think the "binary calculator" is playing here.
Very true. The command that I gave should do the trick for the OP.
Re: transformer
Posted: Thu Apr 15, 2010 8:22 am
by kittu.raja
[quote="vivekgadwal"][quote="chulett"][quote="meet_deb85"]Exact command is
Code: Select all
Hi Everyone,
Thank you for the response.
I want it in transformer itself.
Can I do using stage variables if yes how can I achieve it.
If that it is not possible then I will use before sub routine.
Thanks,
Posted: Thu Apr 15, 2010 8:50 am
by Sainath.Srinivasan
What do you mean by 'count of source file' ?
If you have data from 5 files, will each data record carry 1,2,3,4,5 or will they all have 5 ?
Posted: Thu Apr 15, 2010 11:39 am
by kittu.raja
Sainath.Srinivasan wrote:What do you mean by 'count of source file' ?
If you have data from 5 files, will each data record carry 1,2,3,4,5 or will they all have 5 ?
I have only one file and I want to count the number of records in a file using transformer.
Thanks,
Posted: Thu Apr 15, 2010 12:59 pm
by vivekgadwal
kittu.raja wrote:Sainath.Srinivasan wrote:What do you mean by 'count of source file' ?
If you have data from 5 files, will each data record carry 1,2,3,4,5 or will they all have 5 ?
I have only one file and I want to count the number of records in a file using transformer.
Thanks,
Is there a problem creating a dummy file for the duration of the job's run? I guess what Sainath and even I want to know is, "why the insistence in having this within a transformer?" as there are other simple ways - the way I proposed for example - to do this!