How to differentiate a new file from the old one.

Marcus426 · Post by **Marcus426** » Sat Oct 24, 2009 9:59 pm

Hi guys,

I need some help with the logic. I have a monthly file as an input. Based on the incoming file i need to generate a sequential number( the incoming file has a date column). The sequential number increments by 1 for every next(next month's) file. If for some reason I wanted to run the same monthly file again, the sequential number shouldn't increment by 1. Can you guys help me out with this?

Marcus.![/code]

chulett · Post by **chulett** » Sun Oct 25, 2009 12:27 am

Keep track of the dates and numbers you've processed. A successful lookup would mean use the old number (I assume) else assign a new one then store both it and the date.

Marcus426 · Post by **Marcus426** » Sun Oct 25, 2009 9:56 am

criag! thanks for the response. But can you please elaborate a little bit on that.!

chulett · Post by **chulett** » Sun Oct 25, 2009 10:17 am

Sorry, what part is unclear? To track something, you need to store whatever makes it unique, which sounds like the date in your case. Then a lookup tells you if you've seen it before or not.

Marcus426 · Post by **Marcus426** » Sun Oct 25, 2009 10:28 am

Got you this time! Thanks a lot :D

Marcus426 · Post by **Marcus426** » Sun Oct 25, 2009 2:54 pm

Craig,

Here is an example of what i am supposed to do,

Code: Select all


source file                                       

ColA       ColB                        
May          a
May          a     
May          a
May          b
May          b
May          b

I need to genereate a source_ id for the source file ( a sequential number starting with 1) and for each different values in ColB ineed to generate ColB_id which is unique for each value in colB . for the next months source file the soirce_id should increment by 1( in this case 2) and the colB_id should increment by1 ( in this case to 3 since there are two different colB values). If for any reason I ran the same month file again the values shouldn't increment.

I got the point that we should store them and do a lookup , but I am still confused about how ti increment the values

Thanks!

chulett · Post by **chulett** » Sun Oct 25, 2009 9:49 pm

Stage variables are one way, only increment when ColB changes value. A Sort stage with the 'Create key change column' option would help, too - increment only when that column has a 1 in it.