dynamic creation on metadata
Moderators: chulett, rschirm, roy
dynamic creation on metadata
Hi,
I have a task where my input text file could be in any format each time.
I number of coulmns wil be different each time, i was wondering if it is possible to create the number of coulmns dynamically if i know how many coulmns do i need.
Please advice
I have a task where my input text file could be in any format each time.
I number of coulmns wil be different each time, i was wondering if it is possible to create the number of coulmns dynamically if i know how many coulmns do i need.
Please advice
Okay... who in the heck is supplying you data in a file that "could be in any format each time"? Sheesh. Now we need a RTM stage.
Can you expalin what "any format" actually means? What might change from file to file?
RTM = Read Their Mind
Can you expalin what "any format" actually means? What might change from file to file?
RTM = Read Their Mind
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
[quote="chulett"]Okay... who in the heck is supplying you data in a file that "could be in any format each time"? Sheesh. Now we need a RTM stage.
Can you expalin what "any format" actually means? What might change from file to file?
[size=7]RTM = Read [i]Their[/i] Mind[/size][/quote]
Thanks for replying,
Only the number of fields can differ each time.it could be 10 one time and caould be 30 one.
I was wondering if we can do this without making use os schema files.
Please suggest
Can you expalin what "any format" actually means? What might change from file to file?
[size=7]RTM = Read [i]Their[/i] Mind[/size][/quote]
Thanks for replying,
Only the number of fields can differ each time.it could be 10 one time and caould be 30 one.
I was wondering if we can do this without making use os schema files.
Please suggest
[quote="chulett"]More details, please. Do you always want just the first 10 no matter how many come or do you always need to process all columns? ...[/quote]
No, i might need to process all 10 columns or all 15 or 20.
i believe i can do it by dynamically creating schema files since i know the metadata before running the jobs. but wondering if i can do this without schema files.
No, i might need to process all 10 columns or all 15 or 20.
i believe i can do it by dynamically creating schema files since i know the metadata before running the jobs. but wondering if i can do this without schema files.
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
-
- Premium Member
- Posts: 79
- Joined: Thu Mar 22, 2007 4:58 pm
- Location: USA
The use of 10, 15, 20 and 30 to describe the number of fields implies that what is coming in is going to be in multiples of 5 (5 key fields perhaps and 5 repeating fields?). That sounds a lot like a pivot, which has been discussed many times in these forums. A search on 'pivot' will give you more information than you can shake a stick at.
If it isn't a pivot operation, the more details you provide, the better an answer that you will get.
If it isn't a pivot operation, the more details you provide, the better an answer that you will get.
Jack Thornton
----------------
Spectacular achievement is always preceded by spectacular preparation - Robert H. Schuller
----------------
Spectacular achievement is always preceded by spectacular preparation - Robert H. Schuller
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
I got it resolved.
I making use of my current architecture, tables.
I am sure i can do it by making use of some tables that have been defined,i will know the format before running the jobs and each request can be in a different format.
I am parameterizing the whole logic that we put in the transformer derivation field.i tested it and its working.
I making use of my current architecture, tables.
I am sure i can do it by making use of some tables that have been defined,i will know the format before running the jobs and each request can be in a different format.
I am parameterizing the whole logic that we put in the transformer derivation field.i tested it and its working.