Hi to all
I want to load 10 million records in to Sequential File ...
how many ways we load and which way can take very less time
(which options are available for above scenario in seq file)
Plz help me any one
Thanks in Advance
:D
Sequentioal File
Moderators: chulett, rschirm, roy
What is your data source?
You could place a column export stage in front of the sequential file stage to export your columns into one varchar column with the proper column delimiters (assuming your seqfile is text only), then write the one column with the sequential file stage using the proper record delimiter.
Regards,
You could place a column export stage in front of the sequential file stage to export your columns into one varchar column with the proper column delimiters (assuming your seqfile is text only), then write the one column with the sequential file stage using the proper record delimiter.
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.
And of course the other question to ask is: What have you tried?
Sequential File stage alone may provide the performance needed to meet your business requirements/SLAs. The only benefit of my earlier suggestion is if your file is quite large in terms of volume (number of bytes)--it could parallelize the conversion of your data into the output format, whereas Sequential File stage would do so only sequentially.
Regards,
Sequential File stage alone may provide the performance needed to meet your business requirements/SLAs. The only benefit of my earlier suggestion is if your file is quite large in terms of volume (number of bytes)--it could parallelize the conversion of your data into the output format, whereas Sequential File stage would do so only sequentially.
Regards,
- james wiles
All generalizations are false, including this one - Mark Twain.
All generalizations are false, including this one - Mark Twain.
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
Sequential File stage actually uses STREAMS I/O, so should not be much of a bottleneck in the export operator. As noted, the problem may be in your disk system itself.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.