Hi,
I have a scenario in which I receive files on hourly basis.
Sometimes I may receive file with data of mutiple hours.(ie., not only data of that hour alone, but also data of its previous hours).
Data will be in ascending order ( I mean, data of old hours at first and data of latest hour at last ). I can expect records in such a way that, records which i need to insert new (will be on old hour) and same records may need to update(it will be in latest hour) will be in same file.
The approach that I'm following is
while reading the file, I have created row number in sequential file stage.
In the loading job, I'm using single node config file and sorting the data based on Row num and based on primary column, i'm deciding whether this record need go to insert or update
Please guide me if the above approach is not good
how to identify older and new records from same file
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 23
- Joined: Tue May 14, 2013 6:34 am
From your description the row number you generate is enough to give an order by time.
<a href=http://www.worldcommunitygrid.org/team/ ... TZ9H4CGVP1 target="WCGWin">
</a>
</a>