Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I'm trying to decrypt a file encrypted with AES256 CBC, and here's my code to do that:

PaddedBufferedBlockCipher cipher = new PaddedBufferedBlockCipher(new CBCBlockCipher(new AESFastEngine()));
cipher.init(false, new ParametersWithIV(new KeyParameter(key), iv));
byte[] outBuf = new byte[cipher.getOutputSize(data.length)];
int processed = cipher.processBytes(data, 0, data.length, outBuf, 0);
processed += cipher.doFinal(outBuf, processed);
It's quite simple, however cipher.getOutputSize(data.length) would always return a value which is larger than needed. I always end up with some weird characters at the end of the buffer.
I have tried to decrypt two html files, and here's the end of them:
</HTML>����������������
// Processed: 9304 Size: 9312
</body></html>��������
// Processed: 636 Buffer size: 640
So the additional bytes size is not consistent, I cannot just decrement 'by some'.
What am I doing wrong when calculating the size of the output array?
                Because it has to return a worst-case value, so a buffer of the given size is guaranteed to be big enough.
– Andreas
                Mar 28, 2017 at 7:17
It's relevant to padding.
AES encrypts block by block, and every block must be 128 bits long.  Passing in some data that is not divisible into blocks of 128 bits means that the final block of plaintext must be padded until it is 128 bits in size.
getOutputSize is returning you the the size of the ciphertext should you encrypt.  Remember that the plaintext is padded and then encrypted, so the input size to output size is the same.  You can see this because 636 mod 16 = 12 and 636 - 12 + 16 = 640.  E.g. it is rounding up to the nearest multiple of 16 because this is how many extra padding bytes were included before encryption.
Since you are using this while decrypting, the array is larger than required because you are allocating the bytes that would be required.  getOutputSize can be used for both encryption and decryption provided you are aware that it is a "worst-case" scenario for buffer allocation.
                We actually use PKCS7 padding, I thought that  PaddedBufferedBlockCipher will handle that well. What should I do to remove those characters automatically?
– Krystian
                Mar 28, 2017 at 8:10
                I think your use of getOutputSize is probably wrong.  I imagine it is meant to be used for encryption only.  Can you verify if the question marks are actually null characters?
– Luke Joshua Park
                Mar 28, 2017 at 8:12
                @Krystian getOutputSize cannot be used for determining the decrypted plaintext size, because the padding is only accessible after actual decryption. You cannot know how long the plaintext will be before you try it.
– Artjom B.
                Mar 28, 2017 at 8:34
TL;DR: getOutputSize() lets you know what sized buffer to allocate (a worst case), return values from processBytes/doFinal tell you how much of the buffer was actually used (exact).
You are using PaddedBufferedBlockCipher for decryption (above code defaults to PKCS7 padding). getOutputSize() cannot determine the exact amount of output plaintext until it actually sees the decrypted data for the final block, so the returned value will be an upper bound, which is why the return value is there for processBytes/doFinal to let you know how much was actually output.
For simplicity, PaddedBufferedBlockCipher also ignores the details of the particular padding scheme used - it assumes only that the final block could contain some amount of padding that will be removed.
The unused bytes at the end of 'outBuf' are not written to by the cipher. In particular, they will not contain the padding. Presumably the "weird characters" are zero bytes from the array initialisation.
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.