Re: [Pki-devel] [PATCH] 4 Replaced byte-to-char & char-to-byte converters.

On 1/5/2012 3:00 PM, Endi Sukma Dewata wrote: To clarify, in patch #9 I wrote simple tests for measuring the encoding and decoding times for each converter (identified by the tag) to process a set of chars/bytes 10k times. for (int i=0; i<10000; i++) encode(tag, chars); for (int i=0; i<10000; i++) decode(tag, bytes); Each test is done twice, one using Mozilla JSS library and another using our internal library which is based on sun.io before the upgrade and java.nio (Charset) after the upgrade. Below is the result. The 2 numbers in each category correspond to the execution times before and after the upgrade. BMP String Encoding time: - JSS : 210 ms 212 ms - Internal: 281 ms 319 ms BMP String Decoding time: - JSS : 186 ms 196 ms - Internal: 212 ms 149 ms IA5 String Encoding time: - JSS : 42 ms 101 ms - Internal: 46 ms 110 ms IA5 String Decoding time: - JSS : 93 ms 118 ms - Internal: 84 ms 37 ms Printable String Encoding time: - JSS : 26 ms 113 ms - Internal: 48 ms 86 ms Printable String Decoding time: - JSS : 37 ms 49 ms - Internal: 40 ms 38 ms Teletex String Encoding time: - JSS : 18 ms 36 ms - Internal: 43 ms 103 ms Teletex String Decoding time: - JSS : 16 ms 16 ms - Internal: 23 ms 32 ms Universal String Encoding time: - JSS : 74 ms 71 ms - Internal: 30 ms 49 ms Universal String Decoding time: - JSS : 80 ms 76 ms - Internal: 42 ms 18 ms UTF-8 String Encoding time: - JSS : 13 ms 31 ms - Internal: 28 ms 59 ms UTF-8 String Decoding time: - JSS : 37 ms 51 ms - Internal: 46 ms 52 ms The JSS tests are supposed to produce similar numbers before and after the upgrade since they are executing the same code, but that's not always the case, see IA5 encoding and printable encoding times. This is probably due to the changing nature of machine load at the time of the execution. If we look at the internal numbers, some go up and some go down. We cannot really say whether it's an improvement or degradation since the JSS numbers that we use as a baseline aren't always stable. Also since different converter uses different code, there isn't a single number that can describe the overall behavior. So all I can conclude is that the upgrade to the internal library does not show any significant performance degradation that we should be concerned about at this point. If we want to get real statistical numbers (e.g. % improvement over the old library or JSS) we will have to define the test scope, generate better test data, and execute the tests systematically with varying loads. I'm not sure if we want to do that at this stage. -- Endi S. Dewata