I was wondering if anyone had any hints about how to do this.
We will have large numbers of Call Detail Record files coming at us each day. We are getting a file per switch each time a switch fills up it's 12MB file. We will put the files for each switch type into it's own directory. We have written a C++ decoder for the switches (they come at us in a binary unformatted format). The decoder will take the name of an input file, the switch type, and the name of the output file and decode the file.
We have written another program to search a directory which is a parameter and to pass the name of the input files to the first program to decode the switches. I'm considering combining the two programs into one so we just have one program call and it will read all the files in the directory, decode them, and then pass them along to the decoded file directory.
After this we must run datastage jobs to pick up the decoded files and load them into the staging area.
To do this last bit I was thinking that what I could do was to write C++ code to call the datastage job from the C++ API. It all looks emminently doable. And learning the DS C++ API would not be an all bad thing.
But the thought just struck me (some days I am slow
![Wink ;-)](./images/smilies/icon_wink.gif)
Has anyone else out there done something like this? Accept large volumes of CDR files (or any other types of files) decode them using something other than DS, then load the decoded file using a DS job?
I'd be very interested to know if anyone is willing to share something like this.
Thanks