**Question:**

What exactly does each of the above RSA methods do?

This post is for those wishing to understand internal structure of the outputs produced by each of these methods.

**EncryptBytes:**

- RSA encryption is computationally intensive, and therefore RSA is typically used for small bits of data, such as for encrypting symmetric encryption keys. Assuming there are not too many bytes to be encrypted, the output size is equal to the modulus bit length (i.e. the RSA key length). For example, if a 1024-bit key is used, the output is 128 bytes. (Otherwise the input is chunked into pieces, and each piece is RSA-encrypted and the output is a series of RSA encrypted chunks, each chunk having the size of the RSA key.)

- The first step of RSA encryption is to pad the input using either PKCS v1.5 or OAEP padding. Note: the padding involves randomly generated bytes, and thus the encrypted output of identical data will differ each time.
- The 2nd step is to do the RSA modular exponentiation using the desired key (public or private).
- The final step is to 4321 byte swap the output depending on whether little-endian or bit-endian output is desired.

**SignBytes:**

- First hash the input data using the chosen hash algorithm (such as SHA-1, SHA-256, etc.)
- Next pad the hash using either PKCS v1.5 or PSS. Note: PKCS v1.5 padding for signatures is different than for encryption. It does not use randomly generated bytes. The PKCS v1.5 padding algorithm first ASN.1 encodes the hash into the following ASN.1 format, and then pads it:

SEQUENCE { SEQUENCE { hashOid OID NULL } hash OCTET STRING }

3) The RSA modular exponentiation is performed on the padded data using the desired key (public or private).

4) The final step is to 4321 byte swap the output depending on whether little-endian or bit-endian output is desired.

**OpenSslSignBytes**

- With the OpenSsl* signature functions, it is assumed that the data passed in is already a hash. No automatic hashing is performed as w/ the Sign* methods. (Therefore, the number of bytes passed to an OpenSslSign* methods must be reasonably small (i.e. as small as a hash would typically be).)
- The data is PKCS v1.5 padded directly -- meaning that it is NOT embedded within an ASN.1 encoded structure first. There is no option for PSS padding.
- The RSA modular exponentiation is performed on the padded data using the desired key (public or private).
- The final step is to 4321 byte swap the output depending on whether little-endian or bit-endian output is desired.