Hi,
I have implemented encryption and decryption in parallel routines but the process runs slow as it has to pick up the key from disk also the transformer does not free the memory of returned pointer so the job aborts on very high volume. To enhance the speed I am planning to have a build stage where the per record disk read would be eliminated by a single read before the per record loop.
However there is a small challenge where I would need an expert advice.
1. Using the routines, the user may encrypt multiple columns in one transformer how to allow multiple columns to be encrypted in build stage as the metadata for the input port is static.
2. The encryption routine takes input argument of any datatype and the decryption routine give back exact same datatype and no explicit type conversion are required, how to have this in build stage as the metadata is predefined
Thanks
Need Advice on Build Stage
Moderators: chulett, rschirm, roy
-
- Premium Member
- Posts: 730
- Joined: Tue Nov 04, 2008 10:14 am
- Location: Bangalore
Need Advice on Build Stage
- Zulfi
Can you accomplkish what you need to via java? The java integration stage is far more generic in the ability to determine column names at run time.
Alternatively, a complete custom Operator, perhaps.....
Ernie
Alternatively, a complete custom Operator, perhaps.....
Ernie
Ernie Ostic
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
blogit!
<a href="https://dsrealtime.wordpress.com/2015/0 ... ere/">Open IGC is Here!</a>
-
- Premium Member
- Posts: 730
- Joined: Tue Nov 04, 2008 10:14 am
- Location: Bangalore