can i query on flat files
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 98
- Joined: Thu Feb 05, 2009 1:14 am
- Location: visakhapatnam
- Contact:
can i query on flat files
Hi,
I am using flat files as source ,can i do query on the flat files.
Thanks
Sunitha
I am using flat files as source ,can i do query on the flat files.
Thanks
Sunitha
-
- Participant
- Posts: 98
- Joined: Thu Feb 05, 2009 1:14 am
- Location: visakhapatnam
- Contact:
-
- Participant
- Posts: 98
- Joined: Thu Feb 05, 2009 1:14 am
- Location: visakhapatnam
- Contact:
-
- Participant
- Posts: 98
- Joined: Thu Feb 05, 2009 1:14 am
- Location: visakhapatnam
- Contact:
You don't "query" flat files, you read them. After that, you use whatever stage is appropriate for the "transformation", the original source doesn't matter at that point.
If you still need help, try explaining what exactly it is you need to do - without using the word "query". You might want to explain what "take wanted records" means, as well.
If you still need help, try explaining what exactly it is you need to do - without using the word "query". You might want to explain what "take wanted records" means, as well.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
If you really want to query on flat file, you need an application which will parse your query apply them to a flat file??
can you think about one?
yes there are some databases (no need to say they also have limitations)
like Oracle- You can FTP it to oracle server and then define an external table pointing to that file.
[Channelling Craig]
Then you can query on external tables (flat files) to your heart's content. However you cannot query on flat files directly. You just read them.
[/Channelling Craig]
Thats exactly what Oracle does with the external tables. And its just changes the perception of any one. The same logic can be applied in datastage too. Read files and do the transformations,filteration and get the desired result.
can you think about one?
yes there are some databases (no need to say they also have limitations)
like Oracle- You can FTP it to oracle server and then define an external table pointing to that file.
[Channelling Craig]
Then you can query on external tables (flat files) to your heart's content. However you cannot query on flat files directly. You just read them.
[/Channelling Craig]
Thats exactly what Oracle does with the external tables. And its just changes the perception of any one. The same logic can be applied in datastage too. Read files and do the transformations,filteration and get the desired result.
Priyadarshi Kunal
Genius may have its limitations, but stupidity is not thus handicapped.![Wink :wink:](./images/smilies/icon_wink.gif)
Genius may have its limitations, but stupidity is not thus handicapped.
![Wink :wink:](./images/smilies/icon_wink.gif)
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
Priyadarshi,
Using ODBC drivers for text files may be simpler option.
Creating external tables in Oracle is not straightforward. You have several hurdles to clear - such as create oracle directory, grant priv to the user, create external table ddl, grant select to the user.
Also this must be visible to the database rather than direct tns connection.
So not worth the hassle. I will recommend direct read and processing via DS.
Using ODBC drivers for text files may be simpler option.
Creating external tables in Oracle is not straightforward. You have several hurdles to clear - such as create oracle directory, grant priv to the user, create external table ddl, grant select to the user.
Also this must be visible to the database rather than direct tns connection.
So not worth the hassle. I will recommend direct read and processing via DS.
Last edited by Sainath.Srinivasan on Mon Jun 22, 2009 7:04 am, edited 1 time in total.
-
- Participant
- Posts: 98
- Joined: Thu Feb 05, 2009 1:14 am
- Location: visakhapatnam
- Contact:
Hi,
I am getting data in form of flat files , from as/400 where data is in form of flat files .This data has to be loded again into flat files in target ,but in between th source flat files and target flatfiles we have to perform transformation ,so how can we exactly apply the transformation ,if their is case we havwe to join two flat files ,how should we do.
Thanks
sunitha
I am getting data in form of flat files , from as/400 where data is in form of flat files .This data has to be loded again into flat files in target ,but in between th source flat files and target flatfiles we have to perform transformation ,so how can we exactly apply the transformation ,if their is case we havwe to join two flat files ,how should we do.
Thanks
sunitha
-
- Premium Member
- Posts: 1735
- Joined: Thu Mar 01, 2007 5:44 am
- Location: Troy, MI
Yes, using ODBC drivers is quite straight forward for text file specially csv files but oracle gives more flexibility. It was an example and there are many ways to achive it. But all depends on the requirement.Sainath.Srinivasan wrote:Priyadarshi,
Using ODBC drivers for text files may be simpler option.
Creating external tables in Oracle is not straightforward. You have several hurdles to clear
Also congrats for 2000 posts as well as premium poster logo with your name.
After reading flat files you can apply all kind of transformation on that data. As mentioned by Craig you can use join/lookup/merge.sunitha_cts wrote:I am getting data in form of flat files , from as/400 where data is in form of flat files .This data has to be loded again into flat files in target ,but in between th source flat files and target flatfiles we have to perform transformation ,so how can we exactly apply the transformation ,if their is case we havwe to join two flat files ,how should we do.
Priyadarshi Kunal
Genius may have its limitations, but stupidity is not thus handicapped.![Wink :wink:](./images/smilies/icon_wink.gif)
Genius may have its limitations, but stupidity is not thus handicapped.
![Wink :wink:](./images/smilies/icon_wink.gif)
-
- Premium Member
- Posts: 892
- Joined: Thu Oct 16, 2003 5:18 am
-
- Premium Member
- Posts: 892
- Joined: Thu Oct 16, 2003 5:18 am
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
You can use them anywhere.
As with every other link*, the data end up in a virtual data set.
The only possible input to and the only possible output from an Orchestrate operator is a data set (virtual or physical).
* The single exception is a sparse lookup.
As with every other link*, the data end up in a virtual data set.
The only possible input to and the only possible output from an Orchestrate operator is a data set (virtual or physical).
* The single exception is a sparse lookup.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.