I have searched through the forum and DS documentation but I couldn't find the similar example
I have a parallel job that reads the source file using the Sequential File stage with one varchar column, then splits the records into fields using
Column Import stage according to the dynamically passed schema file.
The source file is has comma separated variable length fields.
I managed to define a schema and I works OK.
Now, I would like to calculate the sum of one column, that I know the position of, for example the 5th field in the file. Is it possible to use aggregator for this?
How do I actually define a calculation to use if I cannot see the fields? The same question stands for a transformer.
I have experimented a bit with a partial schema, just to read the field I am interested in, but cannot make it work so far.
Aggregator and (maybe) partial schema
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Not at all. The Aggregator has special rules (the same rules as GROUP BY in SQL) - you can only pass through grouped columns or columns to which a set function is applied. You can't pass through any other columns, automatically or specifically.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
I see.
In this case, I will just extract the relevant portion of the input records into a defined column in a transformer and pass to the aggregator. It will be parametrized so no problem with making it generic.
I will still use the schema file to load the data into he database.
I will be dealing with lookups and SCDs in a not too distant future. But at least I know I should be able to utlilize schema then .
Thanks.
In this case, I will just extract the relevant portion of the input records into a defined column in a transformer and pass to the aggregator. It will be parametrized so no problem with making it generic.
I will still use the schema file to load the data into he database.
I will be dealing with lookups and SCDs in a not too distant future. But at least I know I should be able to utlilize schema then .
Thanks.