Dat Encryption in DataStage

dsuser1 · Post by **dsuser1** » Mon Dec 08, 2003 1:11 pm

Any approaches for encryption of data using datastage?

T.I.A

kcbland · Post by **kcbland** » Mon Dec 08, 2003 1:16 pm

Pssst, I've got a great encryption program you can have. I promise not to share it.

You can encode any string of data using any encryption program you want. You have to find out what your security requirements require. Simply scrambling a string may not be enough. Are you doing government or healthcare work? If so, you may have regulations to follow.

1stpoint · Post by **1stpoint** » Mon Dec 08, 2003 2:02 pm

Encryption is not only application specific, it is usually database specific.

There are many encryption algorithms available but the application usually determines which one.

mhester · Post by **mhester** » Mon Dec 08, 2003 2:43 pm

Here are some references that may help you in your research regarding encryption/decryption algorithms and methodologies.

1. E. Biham and A. Shamir, Differential Cryptanalysis of the Data Encryption Standard, Springer-Verlag, 1993.

2. T.W. Cusick and M.C. Wood, "The REDOC-II Cryptosystem," Advances in Cryptology--CRYPTO '90 Proceedings, Springer- Verlag, 1991, pp. 545-563.

3. J. Deamen, R. Govaerts, and J. Vandewalle, "Block Ciphers Based on Modular Arithmetic," Proceedings of the 3rd Symposium on State and Progress of Research in Cryptography, Rome, Italy, 15-16 Feb 1993, pp. 80-89.

4. J.-H. Evertse, "Linear Structures in Blockciphers," Advances in Cryptology--EUROCRPYT '87, Springer-Verlag, 1988, pp. 249- 266.

5. H. Feistel, "Cryptography and Computer Privacy," Scientific American, v. 228, n. 5, May 73, pp. 15-23.

6. GOST 28147-89, "Cryptographic Protection for Data Processing Systems," "Cryptographic Transformation Algorithm," Government Standard of the U.S.S.R., Inv. No. 3583, UDC 681.325.6:006.354. (in Russian)

7. X. Lai, J. Massey, and S. Murphy, "Markov Ciphers and Differential Cryptanalysis," Advances in Cryptology--EUROCRYPT '91 Proceedings, Springer-Verlag, 1991, pp. 17-38.

8. J.L. Massey and X. Lai, "Device for Converting a Digital Block and the Use Thereof," International Patent PCT/CH91/00117, 16 May 1991.

9. J.L. Massey and X. Lai, "Device for the Conversion of a Digital Block and Use of Same," U.S. Patent 5,214,703, 25 May 1993.

10. M. Matsui, "Linear Cryptanalysis Method for DES Cipher," Advances in Cryptology--CRYPTO '93 Proceedings, Springer- Verlag, 1994, in preparation.

11. R.C. Merkle, "Fast Software Encryption Functions," Advances in Cryptology--CRYPTO '90 Proceedings, Springer-Verlag, 1991, pp. 476-501.

12. R.C. Merkle, "Method and Apparatus for Data Encryption," U.S. Patent 5,003,597, 26 Mar 1991.

13. S. Miyaguchi, "The FEAL-8 Cryptosystem and Call for Attack," Advances in Cryptology--CRYPTO '89 Proceedings, Springer- Verlag, 1990, pp. 624-627.

14. S. Miyaguchi, "Expansion of the FEAL Cipher," NTT Review, v. 2, n. 6, Nov 1990.

15. S. Miyaguchi, "The FEAL Cipher Family," Advances in Cryptology--CRYPTO '90 Proceedings, Springer-Verlag, 1991, pp. 627-638.

16. National Bureau of Standards, Data Encryption Standard, U.S. Department of Commerce, FIPS Publication 46, Jan 1977.

17. National Institute of Standards and Technology, "Clipper Chip Technology," 30 Apr 1993.

18. RSA Laboratories, Answers to Frequently Asked Questions About Today's Cryptography, Revision 2.0, RSA Data Security Inc., 5 Oct 1993.

19. B. Schneier, "Data Guardians," MacWorld, Feb 1993, 145-151.

20. B. Schneier, Applied Cryptography, John Wiley & Sons, New York, 1994.

21. J.L Smith, The Design of Lucifer, A Cryptographic Device for Data Communication, RC 3326, White Plains: IBM Research.

22. M.J. Weiner, "Efficient DES Key Search," Advances in Cryptology--CRYPTO '93 Proceedings, Springer-Verlag, in preparation.

23. M.C. Wood, "Method of Cryptographically Transforming Electronic Digital Data from One Form to Another," U.S. Patent 5,003,596, 26 Mar 1991.

Regards,

Michael Hester

kduke · Post by **kduke** » Mon Dec 08, 2003 2:50 pm

Michael

Nice list but nobody likes a show off.

Kim.

mhester · Post by **mhester** » Mon Dec 08, 2003 2:57 pm

This list will certainly help someone understand that there is more to encryption/decryption than meets the eye. There are many products out there that can integrate seemlessly with many tools and I was able to find a couple that can be called from DS.

Regards,

Michael Hester

kcbland · Post by **kcbland** » Mon Dec 08, 2003 3:49 pm

I laughed. Mike hit the nail on the head. His point is that the simple question has significant ramifications to the implementation.

Besides, he got a lot of points for that post!

mhester · Post by **mhester** » Mon Dec 08, 2003 4:08 pm

Ken,

You are correct. That was the reason I posted so many references. I also wanted to point out that simple encryption like the following will work but it is the weakest algorithm by far. The string S2 should be unique in every aspect (no repeats), but as I can see in my S2 there is at least one dup which would make the cypher fail. This is a stream algorithm and probably would not be very good for bulk or large data streams and certainly would not be good for file encryption.

Code: Select all

      NEWSTR = ""
      ORIGSTR = ""
      I = 0
      S1 = ' AaBbCcDdEeFfGgHhIiJjKkLlMmNnOoPpQqRrSsTtUuVvWwXxYyZz0123456789~`!@#$%&*()_-+={[}]|\:;<>?/'
      S2 = ' AabBCDcEfheHFGgdjIJkPMioKlNSLqmnOpTQrsWRUuyXvzxZ1Y0tw9V6`@4538_!*~927$-#%<=[(]?f{7+a]&F(*'

      PRINT LEN(S1)
      PRINT LEN(S2)

      PRINT "ENTER STRING TO ENCRYPT ":
      INPUT ESTR

      FOR I = 1 TO LEN(ESTR)

         EPOS = INDEX(S1,ESTR[I,1],1)

         NEWSTR := S2[EPOS,1]

      NEXT I

      I = 0
      EPOS = 0
      FOR I = 1 TO LEN(NEWSTR)

         EPOS = INDEX(S2,NEWSTR[I,1],1)

         ORIGSTR := S1[EPOS,1]

      NEXT I


      PRINT ESTR
      PRINT NEWSTR
      PRINT ORIGSTR
      STOP

Regards,

Michael Hester

ray.wurlod · Post by **ray.wurlod** » Mon Dec 08, 2003 6:19 pm

mhester wrote:Here are some references that may help you in your research regarding encryption/decryption algorithm's and methodologies.

Aaaaarrrrggghhh!! The apostrophe's again!!!

dsuser1 · Post by **dsuser1** » Mon Dec 08, 2003 6:38 pm

I need to implement AES algorithm..

I was looking if somebody has implemented encryption in datstage while doing the data loading. Like having the algorithm in an external program (C, C++, etc) and calling it from ascential. Is this possible? Else is there anything that supports encryption(Oracle has some DBMS utilities for the same)? which is the best appoach?

Is there anybody out there having solid knowledge?

T.I.A

mhester · Post by **mhester** » Mon Dec 08, 2003 8:42 pm

Ray,

That message was actually sent to me by a friend many years ago and all I did was to cut and paste, but I will be sure to pass on your aggravation with his grammar.

Regards,

Michael Hester

dsuser1 · Post by **dsuser1** » Tue Dec 09, 2003 6:03 pm

still I didn't get any good answer :

I need to implement AES algorithm..

I was looking if somebody has implemented encryption in datstage while doing the data loading. Like having the algorithm in an external program (C, C++, etc) and calling it from ascential. Is this possible? Else is there anything that supports encryption(Oracle has some DBMS utilities for the same)? which is the best appoach?

Is there anybody out there having solid knowledge in encryption?

T.I.A

Teej · Post by **Teej** » Tue Dec 09, 2003 7:13 pm

Yes, you can build a custom stage to handle your encryption. Heck, you can use that custom stage to load data, write data, whatever. Do not expect it to be very optimized for PX, though.

Read the PDF included with your client installation of DataStage, "Parallel Job Developer's Guide" (parjdev.pdf), chapter 49, "Specifying Custom Parallel Stages."

If you have any specific questions about custom stages, post it here, and we'll explain a bit further. However, read the entire chapter first, before asking.

But on how to code AES encryption in C? Well, it's a secret.

We'll tell you, but we'll have to eliminate you afterward. Government policy.

-T.J.

mhester · Post by **mhester** » Wed Dec 10, 2003 7:32 am

T.I.A,

Can you create a C program called from a server routine to do this? The answer is yes, but is not for the faint of heart. I have done this a couple of times (for other things) and there can be serious ramifications and Ascential will not support the solution.

Can you create a C program or whatever and call it from PX? The answer would be yes, but I'm not sure how you would do it. As others have pointed out you would need to refer to your documentation and then post questions.

Is it better to let the DB handle this? I don't know the answer to that and maybe some of the Oracle forums might give you some insight.

You have not received a definitive answer because the truth is that there is a small pool of consultants that have this skill set in their bag. To paraphrase others - this is kind of a murky area and one that is sure to have many different or possible solutions.

It also matters what kind of cipher you want to use and whether you are doing stream, block or file encryption/decryption. Is 64 bit ok or do you require 128 bit? Is the key 64, 128 or 496 bit? The list is endless so this would be no trivial task.

Regards,

Michael Hester

chulett · Post by **chulett** » Thu Nov 10, 2005 10:49 am

mhester wrote:This list will certainly help someone understand that there is more to encryption/decryption than meets the eye. There are many products out there that can integrate seemlessly with many tools and I was able to find a couple that can be called from DS.

Sorry to resurrect this old thread - and I'm not sure if Michael shows up here too often any more - but I'm curious if anyone knows what "couple of products" he found that can be called from DS? Or what I might look for that would make something 'callable'?

Thanks.