DSXchange

Hi, I tried reading multiple xml files using External Source stage.. I mean I used the ls command to list out the files in the dir. But the issue is there are around 50000 files in the dir and I am getting the error like too many return values. I shall send the exact error by tomorrow. Seems like the list is large and not able to capture this list. When I tried say some 25000 files its working fine..Is there a way to read all the 50000 files at the same time.

Please start a new topic with this new and independent question....

So let it be written, so let it be done.

A quick google for "argument list too long" (which is what I assume your issue is) should help with alternatives to the ls you are doing.

Hi Thanks a lot for your advise, I got the solution for this. I used the below command in the External source stage and it worked fine..

find ./ -maxdepth 1 -name "*.txt"

Thanks again..