Dynamic Metadata
Moderators: chulett, rschirm, roy
Dynamic Metadata
We have a requirement is as below:
Table A:
Col1
Col2
Col3
Col4
Col5
Col6
Col7
Col8
Col9
Col10
All users require Col1 to Col5 to be loaded to the target table.
User1 need Col6 & Col7 also to be loaded in the target along with Col1 to Col5. This may change later as the user1 may require Col6, Col7 & Col9 to be loaded.
User2 need Col7 & Col10 to be loaded in addition to Col1 to Col5
User3 need Col8 to be loaded in addition to Col1 to Col5.
We have transformation defined only for Col1 to Col5. All other column values can be directly loaded into the target table.
Please let me know whether this can be achieved in DS using a single job.
Thanks in advance!
Regards,
Manju
Table A:
Col1
Col2
Col3
Col4
Col5
Col6
Col7
Col8
Col9
Col10
All users require Col1 to Col5 to be loaded to the target table.
User1 need Col6 & Col7 also to be loaded in the target along with Col1 to Col5. This may change later as the user1 may require Col6, Col7 & Col9 to be loaded.
User2 need Col7 & Col10 to be loaded in addition to Col1 to Col5
User3 need Col8 to be loaded in addition to Col1 to Col5.
We have transformation defined only for Col1 to Col5. All other column values can be directly loaded into the target table.
Please let me know whether this can be achieved in DS using a single job.
Thanks in advance!
Regards,
Manju
-
- Premium Member
- Posts: 138
- Joined: Wed Jul 16, 2008 9:51 pm
- Location: Kolkata
Hi Arulan,
As per my understanding of the post, the "Table A" which is the target table has already got 10 columns and Col1 to Col5 are using some transformation logic whereas Col6 to Col10 are just direct mappings.
So the confusing part is what's the problem in loading Col6 to Col10.
If there is an issue, then FDW_CITI hasn't said it yet.
As per my understanding of the post, the "Table A" which is the target table has already got 10 columns and Col1 to Col5 are using some transformation logic whereas Col6 to Col10 are just direct mappings.
So the confusing part is what's the problem in loading Col6 to Col10.
![Confused :?](./images/smilies/icon_confused.gif)
If there is an issue, then FDW_CITI hasn't said it yet.
TONY
ETL Manager
Infotrellis India
"Do what you can, with what you have, from where you are and to the best of your abilities."
ETL Manager
Infotrellis India
"Do what you can, with what you have, from where you are and to the best of your abilities."
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
The Target table is Single table with 3 different query requirements.ArndW wrote:Do users 1, 2 and 3 read different target tables? Or do you just have one target table but 3 different query requirements? ...
The users 1, 2 & 3 may select any columns from col6 to col10 for loading the target table.
The column selection is dynamic and this should be done using a single job
Regards,
Manju
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Why?FDW_CITI wrote:... and this should be done using a single job
To me the most eminently sensible, and most easily maintained, design is to use separate jobs with a sequence to make the decision about which one to run. These might be multi-instance so each user can run without needing to have regard to any other users who might be running it.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
-
- Premium Member
- Posts: 79
- Joined: Thu Mar 22, 2007 4:58 pm
- Location: USA
It sounds like there is a lack of separation in this problem. As I read it:
1. Different users need different query results for reports.
2. The defined solution is to perform the LOAD according to the ultimate output needed.
Wouldn't it make more sense to split up the Load and the Report requirements? Especially where the reporting requirements may change?
The other issue that comes to mind is what is the reason for the different needs? If it is just about what individual job responsibilities the single table/single job solution is good. If the difference is for security (maybe Col 7,8,9 are HIPAA or financially sensitive values...) then it would imply that the problem description is incomplete and something else needs to address the access.
1. Different users need different query results for reports.
2. The defined solution is to perform the LOAD according to the ultimate output needed.
Wouldn't it make more sense to split up the Load and the Report requirements? Especially where the reporting requirements may change?
The other issue that comes to mind is what is the reason for the different needs? If it is just about what individual job responsibilities the single table/single job solution is good. If the difference is for security (maybe Col 7,8,9 are HIPAA or financially sensitive values...) then it would imply that the problem description is incomplete and something else needs to address the access.
Jack Thornton
----------------
Spectacular achievement is always preceded by spectacular preparation - Robert H. Schuller
----------------
Spectacular achievement is always preceded by spectacular preparation - Robert H. Schuller