DSXchange

Tar command takes 35 mins on the command prompt.
But when run in the job ,the job is not completeing at all.

Thanks,
Chetan

Hi, I have a n external source stage with below command. tar -xOvf /home/LLS2TEALFA01_CSS997.tar This tar ball has 19k files and around 100,000 records. The job takes around 30 mins to finish. The performace of the stage is very slow.I'm just testing the load with the job having only 2 stages: Exter...

The issue with the external source stage about consuming more bytes than defined i resolved that error by increasing APT_MAX_DELIMITED_READ_SIZE and APT_DEFAULT_TRANSPORT_BLOCK_SIZE. That seems to have solved the Consuming more than 100000 bytes error. But still did not get why it says record delimi...

Thanks for the response.

But actually the client does not want to write these files to the disk.
So going on this route.


Thanks,
Chetan

Yes windows newline erros with no record delimiter. I did read a post here about the error of Consuming 100000 bytes for which the user increased the APT_MAX_DELIMITED_READ_SIZE and the job worked. But i dont want to do that atleast now without finding out why this is happening. Any thoughts? thanks...

hi Kryt0n, I just passed the file throught od -xc and i got the below output. The First two lines of the files looks like this. [iamie] TLTSID=E53F96B42B72102BB3D22D89C82DEAEC. The second portion in the code is with a blank line and next line which has data. So it is DOS style right ?As there is \r\...

Specify your record delimiter string property in the Sequential File stage as "DOS style". ... Hi Ray, I tried using DOS style but still the same problem. Also when i load a tar file which has around 7000 records,the job aborts saying consumed more than 100000 bytes . I read the posts her...

Hi ,
I'm getting a special character @ in files.
What is this character to remove it.

Like ^M is (ctrl-v ctrl-m).


Thanks,
Chetan.C

Actually there is ^M characters in the data.

When i use tar -xOvf /home/TL012938524.tar | sed 's/^M/ /g' on command prompt i'm getting the correct output but if i use it in external source stage its not getting removed.

Thanks

Hi ,

Is it a problem in the configuration?
Please guide me for a solution.

Thanks,
Chetan.C

Hi , Im using the below command to in the source program column in External source stage. tar -xOvf /home/files/Reques_folder.tar The output from the external source stage is skipping the Blank lines. However when i enter the same command in unix prompt it is giving the correct data from the files i...

Sorry actually len() works but my logic in stage variables was wrong, fixed it thanks.

Hi,

I tried len() but its not working either.
Empty line is just a blank line not any number of spaces or blanks.
Like
a

b

Thanks.

Hi,

I have data like this.
Col1
a
b
c
"Empty line"
d
e
f

My output should be
abc
def

I'm using stage variables to concatenate the rows.
But how to mention the constraint in Transformer as blank line?

I tried trim(col1)='' but it is not working.

Thanks
Chetan

Thanks ray.